2024-05-27T09:14:18.653 INFO:root:teuthology version: 0.0.1.dev291+ge7ae758 2024-05-27T09:14:18.654 DEBUG:teuthology.run:Teuthology command: teuthology --name vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi --owner scheduled_vshankar@teuthology --verbose --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} --archive /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727681 -- /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727681/orig.config.yaml 2024-05-27T09:14:18.702 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-27T09:14:18.791 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727681 branch: wip-vshankar-testing-20240521.105740-debug description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} email: vshankar@redhat.com first_in_suite: false job_id: '7727681' kernel: client: branch: testing flavor: default kdb: 1 sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240521.105740-debug ceph: cephfs: max_mds: 3 session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true mds_sleep_rank_change: 5000000.0 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - Replacing daemon mds sha1: a6507388dd5057528934822c0163b0c347ef1d5d ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: a6507388dd5057528934822c0163b0c347ef1d5d kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240521.105740-debug sha1: a6507388dd5057528934822c0163b0c347ef1d5d owner: scheduled_vshankar@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 2274 sha1: a6507388dd5057528934822c0163b0c347ef1d5d sleep_before_teardown: 0 subset: 442/1024 suite: fs suite_branch: wip-vshankar-testing-20240521.105740-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a6507388dd5057528934822c0163b0c347ef1d5d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi114.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKf/+L1STYw31rZGPcAo5FwUu6vxD7GBLONXQiG9T5rWTrlECbcLiomfEzVFwCNwwVsVZ+tmOWYoL5g+DnNAhC4= smithi163.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDHRf4tY2GXdVG2BTaLwqt8rpW1NZRe+/anqgZY1DI6ZTtB5MtdtrycJNc0mmZLgb/s5FNAdI8MfkNhPT5JOPJ4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - mds_thrash: null - workunit: clients: all: - fs/snaps timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e7ae7580c417d6ab27fda98fecadca5d9f85d63e timestamp: 2024-05-27_06:46:53 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-27T09:14:18.791 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa; will attempt to use it 2024-05-27T09:14:18.792 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks 2024-05-27T09:14:18.792 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-27T09:14:18.793 INFO:teuthology.task.internal:Checking packages... 2024-05-27T09:14:18.814 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a6507388dd5057528934822c0163b0c347ef1d5d' 2024-05-27T09:14:18.814 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-27T09:14:18.814 INFO:teuthology.packaging:ref: None 2024-05-27T09:14:18.814 INFO:teuthology.packaging:tag: None 2024-05-27T09:14:18.814 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240521.105740-debug 2024-05-27T09:14:18.814 INFO:teuthology.packaging:sha1: a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:14:18.814 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240521.105740-debug 2024-05-27T09:14:18.961 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3868.ga6507388 2024-05-27T09:14:18.962 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-27T09:14:18.969 INFO:teuthology.task.internal:no buildpackages task found 2024-05-27T09:14:18.969 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-27T09:14:18.979 INFO:teuthology.task.internal:Saving configuration 2024-05-27T09:14:18.996 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-27T09:14:19.004 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-27T09:14:19.028 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727681', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 09:08:41.099870', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKf/+L1STYw31rZGPcAo5FwUu6vxD7GBLONXQiG9T5rWTrlECbcLiomfEzVFwCNwwVsVZ+tmOWYoL5g+DnNAhC4='} 2024-05-27T09:14:19.050 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727681', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 09:08:41.100907', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDHRf4tY2GXdVG2BTaLwqt8rpW1NZRe+/anqgZY1DI6ZTtB5MtdtrycJNc0mmZLgb/s5FNAdI8MfkNhPT5JOPJ4='} 2024-05-27T09:14:19.050 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-27T09:14:19.056 INFO:teuthology.task.internal:roles: ubuntu@smithi114.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-05-27T09:14:19.057 INFO:teuthology.task.internal:roles: ubuntu@smithi163.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-05-27T09:14:19.057 INFO:teuthology.run_tasks:Running task console_log... 2024-05-27T09:14:19.113 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f382ddcd000>, signals=[15]) 2024-05-27T09:14:19.113 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-27T09:14:19.119 INFO:teuthology.task.internal:Opening connections... 2024-05-27T09:14:19.119 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi114.front.sepia.ceph.com 2024-05-27T09:14:19.121 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:14:19.195 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi163.front.sepia.ceph.com 2024-05-27T09:14:19.196 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:14:19.268 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-27T09:14:19.275 DEBUG:teuthology.orchestra.run.smithi114:> uname -m 2024-05-27T09:14:19.292 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64 2024-05-27T09:14:19.293 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release 2024-05-27T09:14:19.350 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="CentOS Stream" 2024-05-27T09:14:19.350 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="9" 2024-05-27T09:14:19.350 INFO:teuthology.orchestra.run.smithi114.stdout:ID="centos" 2024-05-27T09:14:19.350 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE="rhel fedora" 2024-05-27T09:14:19.351 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="9" 2024-05-27T09:14:19.351 INFO:teuthology.orchestra.run.smithi114.stdout:PLATFORM_ID="platform:el9" 2024-05-27T09:14:19.351 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T09:14:19.351 INFO:teuthology.orchestra.run.smithi114.stdout:ANSI_COLOR="0;31" 2024-05-27T09:14:19.351 INFO:teuthology.orchestra.run.smithi114.stdout:LOGO="fedora-logo-icon" 2024-05-27T09:14:19.351 INFO:teuthology.orchestra.run.smithi114.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T09:14:19.351 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://centos.org/" 2024-05-27T09:14:19.351 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T09:14:19.351 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T09:14:19.351 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T09:14:19.352 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2024-05-27T09:14:19.370 DEBUG:teuthology.orchestra.run.smithi163:> uname -m 2024-05-27T09:14:19.388 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64 2024-05-27T09:14:19.388 DEBUG:teuthology.orchestra.run.smithi163:> cat /etc/os-release 2024-05-27T09:14:19.445 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="CentOS Stream" 2024-05-27T09:14:19.445 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="9" 2024-05-27T09:14:19.445 INFO:teuthology.orchestra.run.smithi163.stdout:ID="centos" 2024-05-27T09:14:19.445 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="rhel fedora" 2024-05-27T09:14:19.445 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="9" 2024-05-27T09:14:19.445 INFO:teuthology.orchestra.run.smithi163.stdout:PLATFORM_ID="platform:el9" 2024-05-27T09:14:19.445 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T09:14:19.445 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31" 2024-05-27T09:14:19.445 INFO:teuthology.orchestra.run.smithi163.stdout:LOGO="fedora-logo-icon" 2024-05-27T09:14:19.445 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T09:14:19.445 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://centos.org/" 2024-05-27T09:14:19.446 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T09:14:19.446 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T09:14:19.446 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T09:14:19.446 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2024-05-27T09:14:19.470 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-27T09:14:19.479 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-27T09:14:19.485 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-27T09:14:19.485 DEBUG:teuthology.orchestra.run.smithi114:> test '!' -e /home/ubuntu/cephtest 2024-05-27T09:14:19.488 DEBUG:teuthology.orchestra.run.smithi163:> test '!' -e /home/ubuntu/cephtest 2024-05-27T09:14:19.504 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-27T09:14:19.510 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-27T09:14:19.510 DEBUG:teuthology.orchestra.run.smithi114:> test -z $(ls -A /var/lib/ceph) 2024-05-27T09:14:19.544 DEBUG:teuthology.orchestra.run.smithi163:> test -z $(ls -A /var/lib/ceph) 2024-05-27T09:14:19.644 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-27T09:14:19.700 INFO:teuthology.run_tasks:Running task kernel... 2024-05-27T09:14:19.717 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'} 2024-05-27T09:14:19.717 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-05-27T09:14:19.717 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-05-27T09:14:19.717 DEBUG:teuthology.orchestra.run.smithi114:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T09:14:19.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:14:19.734 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-05-27T09:14:20.047 DEBUG:teuthology.task.kernel:sha1 for client.0 is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:14:20.047 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-27T09:14:20.047 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-05-27T09:14:20.064 INFO:teuthology.orchestra.run.smithi114.stdout:5.14.0-437.el9.x86_64 2024-05-27T09:14:20.065 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:14:20.065 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-27T09:14:20.066 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum upgrade -y linux-firmware 2024-05-27T09:14:26.428 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.1 MB 00:04 2024-05-27T09:14:29.612 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - AppStream 9.0 MB/s | 19 MB 00:02 2024-05-27T09:14:35.470 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 6.2 MB 00:02 2024-05-27T09:14:37.296 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 17 kB 00:00 2024-05-27T09:14:38.148 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 29 MB/s | 21 MB 00:00 2024-05-27T09:14:43.315 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-05-27T09:14:45.262 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-05-27T09:14:45.263 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-05-27T09:14:45.263 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-05-27T09:14:45.383 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on client.0... 2024-05-27T09:14:45.383 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:14:45.604 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-27T09:14:45.604 DEBUG:teuthology.orchestra.run.smithi114:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-27T09:14:45.606 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi114.front.sepia.ceph.com to complete... 2024-05-27T09:14:57.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27 09:14:57 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-27T09:14:57.776 INFO:teuthology.orchestra.run.smithi114.stderr:FINISHED --2024-05-27 09:14:57-- 2024-05-27T09:14:57.776 INFO:teuthology.orchestra.run.smithi114.stderr:Total wall clock time: 12s 2024-05-27T09:14:57.776 INFO:teuthology.orchestra.run.smithi114.stderr:Downloaded: 1 files, 340M in 12s (28.7 MB/s) 2024-05-27T09:14:57.780 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on client.0... 2024-05-27T09:14:57.780 DEBUG:teuthology.orchestra.run.smithi114:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-27T09:14:59.862 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying... ######################################## 2024-05-27T09:15:00.057 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing... ######################################## 2024-05-27T09:15:00.177 INFO:teuthology.orchestra.run.smithi114.stdout:Updating / installing... 2024-05-27T09:15:08.051 INFO:teuthology.orchestra.run.smithi114.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-27T09:15:47.480 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi114.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-27T09:15:47.480 DEBUG:teuthology.orchestra.run.smithi114:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-27T09:15:47.560 INFO:teuthology.orchestra.run.smithi114.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-27T09:15:47.560 INFO:teuthology.orchestra.run.smithi114.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-27T09:15:47.560 INFO:teuthology.orchestra.run.smithi114.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-27T09:15:47.560 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-27T09:15:47.560 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-27T09:15:47.560 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-27T09:15:47.560 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-27T09:15:47.560 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-27T09:15:47.560 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-27T09:15:47.560 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-27T09:15:47.560 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-27T09:15:47.561 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-27T09:15:47.561 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-27T09:15:47.561 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-27T09:15:47.561 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-27T09:15:47.561 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-27T09:15:47.561 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-27T09:15:47.561 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-27T09:15:47.561 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-27T09:15:47.561 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-27T09:15:47.561 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-27T09:15:47.561 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-27T09:15:47.561 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-27T09:15:47.562 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-27T09:15:47.562 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-27T09:15:47.562 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-27T09:15:47.562 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-27T09:15:47.562 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-27T09:15:47.562 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-27T09:15:47.562 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-27T09:15:47.562 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-27T09:15:47.562 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-27T09:15:47.562 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-27T09:15:47.562 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-27T09:15:47.563 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-27T09:15:47.563 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-27T09:15:47.563 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-27T09:15:47.563 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-27T09:15:47.563 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-27T09:15:47.563 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-27T09:15:47.563 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-27T09:15:47.563 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-27T09:15:47.563 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-27T09:15:47.563 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-27T09:15:47.563 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-27T09:15:47.563 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-27T09:15:47.563 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-27T09:15:47.563 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-27T09:15:47.564 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-27T09:15:47.564 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-27T09:15:47.564 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-27T09:15:47.564 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-27T09:15:47.564 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-27T09:15:47.564 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-27T09:15:47.564 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-27T09:15:47.564 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-27T09:15:47.564 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-27T09:15:47.564 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-27T09:15:47.564 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-27T09:15:47.564 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-27T09:15:47.564 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-27T09:15:47.565 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-27T09:15:47.565 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-27T09:15:47.565 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-27T09:15:47.565 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-27T09:15:47.565 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-27T09:15:47.565 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-27T09:15:47.565 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-27T09:15:47.565 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-27T09:15:47.565 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-27T09:15:47.565 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-27T09:15:47.565 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-27T09:15:47.565 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-27T09:15:47.565 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-27T09:15:47.565 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-27T09:15:47.566 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-27T09:15:47.566 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-27T09:15:47.566 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-27T09:15:47.566 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-27T09:15:47.566 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-27T09:15:47.566 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-27T09:15:47.566 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-27T09:15:47.566 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-27T09:15:47.566 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-27T09:15:47.566 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-27T09:15:47.566 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-27T09:15:47.566 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-27T09:15:47.566 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-27T09:15:47.566 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-27T09:15:47.566 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-27T09:15:47.567 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-27T09:15:47.567 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-27T09:15:47.567 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-27T09:15:47.567 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-27T09:15:47.567 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-27T09:15:47.567 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-27T09:15:47.567 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-27T09:15:47.567 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-27T09:15:47.567 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-27T09:15:47.567 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-27T09:15:47.567 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-27T09:15:47.567 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-27T09:15:47.567 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-27T09:15:47.568 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-27T09:15:47.568 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-27T09:15:47.568 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-27T09:15:47.568 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-27T09:15:47.568 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-27T09:15:47.568 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-27T09:15:47.568 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-27T09:15:47.568 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-27T09:15:47.568 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-27T09:15:47.568 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-27T09:15:47.568 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-27T09:15:47.568 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-27T09:15:47.568 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-27T09:15:47.568 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-27T09:15:47.569 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-27T09:15:47.569 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-27T09:15:47.569 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-27T09:15:47.569 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-27T09:15:47.569 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-27T09:15:47.569 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-27T09:15:47.569 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-27T09:15:47.569 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-27T09:15:47.569 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-27T09:15:47.569 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-27T09:15:47.569 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-27T09:15:47.569 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-27T09:15:47.569 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-27T09:15:47.569 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-27T09:15:47.569 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-27T09:15:47.570 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-27T09:15:47.570 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-27T09:15:47.570 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-27T09:15:47.570 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-27T09:15:47.570 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-27T09:15:47.570 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-27T09:15:47.570 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-27T09:15:47.570 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-27T09:15:47.570 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-27T09:15:47.570 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-27T09:15:47.570 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-27T09:15:47.570 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-27T09:15:47.570 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-27T09:15:47.571 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-27T09:15:47.571 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-27T09:15:47.571 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-27T09:15:47.571 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-27T09:15:47.571 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-27T09:15:47.571 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-27T09:15:47.571 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-27T09:15:47.571 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-27T09:15:47.571 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-27T09:15:47.571 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-27T09:15:47.571 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-27T09:15:47.571 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-27T09:15:47.571 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-27T09:15:47.572 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-27T09:15:47.572 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-27T09:15:47.572 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-27T09:15:47.572 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-27T09:15:47.572 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-27T09:15:47.572 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-27T09:15:47.572 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-27T09:15:47.572 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-27T09:15:47.572 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-27T09:15:47.572 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-27T09:15:47.572 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-27T09:15:47.572 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-27T09:15:47.572 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-27T09:15:47.572 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-27T09:15:47.573 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-27T09:15:47.573 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-27T09:15:47.573 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-27T09:15:47.573 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-27T09:15:47.573 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-27T09:15:47.573 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-27T09:15:47.573 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-27T09:15:47.573 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-27T09:15:47.573 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-27T09:15:47.573 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-27T09:15:47.573 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-27T09:15:47.573 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-27T09:15:47.573 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-27T09:15:47.574 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-27T09:15:47.574 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-27T09:15:47.574 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-27T09:15:47.574 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-27T09:15:47.574 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-27T09:15:47.574 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-27T09:15:47.574 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-27T09:15:47.574 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-27T09:15:47.574 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-27T09:15:47.574 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-27T09:15:47.574 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-27T09:15:47.574 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-27T09:15:47.574 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-27T09:15:47.575 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-27T09:15:47.575 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-27T09:15:47.575 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-27T09:15:47.575 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-27T09:15:47.575 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-27T09:15:47.575 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-27T09:15:47.575 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-27T09:15:47.575 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-27T09:15:47.575 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-27T09:15:47.575 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-27T09:15:47.575 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-27T09:15:47.575 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-27T09:15:47.575 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-27T09:15:47.576 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-27T09:15:47.576 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-27T09:15:47.576 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-27T09:15:47.576 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-27T09:15:47.576 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-27T09:15:47.576 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-27T09:15:47.576 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-27T09:15:47.576 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-27T09:15:47.576 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-27T09:15:47.576 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-27T09:15:47.576 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-27T09:15:47.576 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-27T09:15:47.576 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-27T09:15:47.576 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-27T09:15:47.576 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-27T09:15:47.577 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-27T09:15:47.577 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-27T09:15:47.577 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-27T09:15:47.577 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-27T09:15:47.577 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-27T09:15:47.577 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-27T09:15:47.577 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-27T09:15:47.577 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-27T09:15:47.577 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-27T09:15:47.577 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-27T09:15:47.577 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-27T09:15:47.577 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-27T09:15:47.577 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-27T09:15:47.577 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-27T09:15:47.578 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-27T09:15:47.578 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-27T09:15:47.578 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-27T09:15:47.578 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-27T09:15:47.578 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-27T09:15:47.578 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-27T09:15:47.578 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-27T09:15:47.578 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-27T09:15:47.578 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-27T09:15:47.578 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-27T09:15:47.578 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-27T09:15:47.578 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-27T09:15:47.578 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-27T09:15:47.579 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-27T09:15:47.579 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-27T09:15:47.579 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-27T09:15:47.579 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-27T09:15:47.579 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-27T09:15:47.579 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-27T09:15:47.579 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-27T09:15:47.579 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-27T09:15:47.579 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-27T09:15:47.579 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-27T09:15:47.579 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-27T09:15:47.579 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-27T09:15:47.579 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-27T09:15:47.580 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-27T09:15:47.580 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-27T09:15:47.580 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-27T09:15:47.580 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-27T09:15:47.580 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-27T09:15:47.580 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-27T09:15:47.580 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-27T09:15:47.580 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-27T09:15:47.580 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-27T09:15:47.580 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-27T09:15:47.580 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-27T09:15:47.580 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-27T09:15:47.580 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-27T09:15:47.580 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-27T09:15:47.581 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-27T09:15:47.581 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-27T09:15:47.581 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-27T09:15:47.581 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-27T09:15:47.581 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-27T09:15:47.581 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-27T09:15:47.581 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-27T09:15:47.581 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-27T09:15:47.581 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-27T09:15:47.581 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-27T09:15:47.581 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-27T09:15:47.581 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-27T09:15:47.581 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-27T09:15:47.582 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-27T09:15:47.582 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-27T09:15:47.582 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-27T09:15:47.582 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-27T09:15:47.582 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-27T09:15:47.582 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-27T09:15:47.582 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-27T09:15:47.582 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-27T09:15:47.582 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-27T09:15:47.582 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-27T09:15:47.583 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-27T09:15:47.583 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-27T09:15:47.583 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-27T09:15:47.583 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-27T09:15:47.583 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-27T09:15:47.583 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-27T09:15:47.583 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-27T09:15:47.583 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-27T09:15:47.583 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-27T09:15:47.583 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-27T09:15:47.585 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-27T09:15:47.585 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-27T09:15:47.586 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-27T09:15:47.586 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-27T09:15:47.586 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-27T09:15:47.586 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-27T09:15:47.586 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-27T09:15:47.586 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-27T09:15:47.586 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-27T09:15:47.586 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-27T09:15:47.586 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-27T09:15:47.586 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-27T09:15:47.586 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-27T09:15:47.586 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-27T09:15:47.587 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-27T09:15:47.587 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-27T09:15:47.587 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-27T09:15:47.587 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-27T09:15:47.587 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-27T09:15:47.587 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-27T09:15:47.587 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-27T09:15:47.587 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-27T09:15:47.587 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-27T09:15:47.587 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-27T09:15:47.587 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-27T09:15:47.587 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-27T09:15:47.587 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-27T09:15:47.587 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-27T09:15:47.588 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-27T09:15:47.588 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-27T09:15:47.588 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-27T09:15:47.588 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-27T09:15:47.588 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-27T09:15:47.588 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-27T09:15:47.588 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-27T09:15:47.588 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-27T09:15:47.588 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-27T09:15:47.588 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-27T09:15:47.588 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-27T09:15:47.588 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-27T09:15:47.588 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-27T09:15:47.588 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-27T09:15:47.589 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-27T09:15:47.589 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-27T09:15:47.589 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-27T09:15:47.589 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-27T09:15:47.589 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-27T09:15:47.589 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-27T09:15:47.589 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-27T09:15:47.589 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-27T09:15:47.589 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-27T09:15:47.589 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-27T09:15:47.589 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-27T09:15:47.589 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-27T09:15:47.589 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-27T09:15:47.590 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-27T09:15:47.590 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-27T09:15:47.590 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-27T09:15:47.590 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-27T09:15:47.590 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-27T09:15:47.590 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-27T09:15:47.590 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-27T09:15:47.590 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-27T09:15:47.590 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-27T09:15:47.590 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-27T09:15:47.590 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-27T09:15:47.590 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-27T09:15:47.591 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-27T09:15:47.591 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-27T09:15:47.591 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-27T09:15:47.591 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-27T09:15:47.591 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-27T09:15:47.591 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-27T09:15:47.591 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-27T09:15:47.591 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-27T09:15:47.591 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-27T09:15:47.591 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-27T09:15:47.591 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-27T09:15:47.591 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-27T09:15:47.591 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-27T09:15:47.592 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-27T09:15:47.592 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-27T09:15:47.592 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-27T09:15:47.592 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-27T09:15:47.592 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-27T09:15:47.592 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-27T09:15:47.592 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-27T09:15:47.592 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-27T09:15:47.592 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-27T09:15:47.592 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-27T09:15:47.592 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-27T09:15:47.592 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-27T09:15:47.592 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-27T09:15:47.592 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-27T09:15:47.592 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-27T09:15:47.593 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-27T09:15:47.593 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-27T09:15:47.593 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-27T09:15:47.593 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-27T09:15:47.593 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-27T09:15:47.593 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-27T09:15:47.593 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-27T09:15:47.593 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-27T09:15:47.593 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-27T09:15:47.593 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-27T09:15:47.593 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-27T09:15:47.593 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-27T09:15:47.594 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-27T09:15:47.594 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-27T09:15:47.594 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-27T09:15:47.594 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-27T09:15:47.594 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-27T09:15:47.594 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-27T09:15:47.594 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-27T09:15:47.594 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-27T09:15:47.594 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-27T09:15:47.594 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-27T09:15:47.594 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-27T09:15:47.594 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-27T09:15:47.594 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-27T09:15:47.595 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-27T09:15:47.595 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-27T09:15:47.595 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-27T09:15:47.595 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-27T09:15:47.595 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-27T09:15:47.595 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-27T09:15:47.595 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-27T09:15:47.595 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-27T09:15:47.595 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-27T09:15:47.595 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-27T09:15:47.595 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-27T09:15:47.595 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-27T09:15:47.595 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-27T09:15:47.595 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-27T09:15:47.596 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-27T09:15:47.596 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-27T09:15:47.596 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-27T09:15:47.596 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-27T09:15:47.596 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-27T09:15:47.596 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-27T09:15:47.596 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-27T09:15:47.596 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-27T09:15:47.596 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-27T09:15:47.596 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-27T09:15:47.596 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-27T09:15:47.596 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-27T09:15:47.596 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-27T09:15:47.597 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-27T09:15:47.597 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-27T09:15:47.597 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-27T09:15:47.597 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-27T09:15:47.597 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-27T09:15:47.597 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-27T09:15:47.597 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-27T09:15:47.597 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-27T09:15:47.597 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-27T09:15:47.597 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-27T09:15:47.597 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-27T09:15:47.597 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-27T09:15:47.597 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-27T09:15:47.597 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-27T09:15:47.598 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-27T09:15:47.598 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-27T09:15:47.598 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-27T09:15:47.598 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-27T09:15:47.598 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-27T09:15:47.598 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-27T09:15:47.598 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-27T09:15:47.598 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-27T09:15:47.598 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-27T09:15:47.598 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-27T09:15:47.598 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-27T09:15:47.598 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-27T09:15:47.599 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-27T09:15:47.599 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-27T09:15:47.599 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-27T09:15:47.599 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-27T09:15:47.599 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-27T09:15:47.599 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-27T09:15:47.599 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-27T09:15:47.599 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-27T09:15:47.599 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-27T09:15:47.599 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-27T09:15:47.599 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-27T09:15:47.599 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-27T09:15:47.599 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-27T09:15:47.599 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-27T09:15:47.600 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-27T09:15:47.600 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-27T09:15:47.600 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-27T09:15:47.600 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-27T09:15:47.600 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-27T09:15:47.600 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-27T09:15:47.600 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-27T09:15:47.600 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-27T09:15:47.600 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-27T09:15:47.600 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-27T09:15:47.600 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-27T09:15:47.600 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-27T09:15:47.600 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-27T09:15:47.600 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-27T09:15:47.601 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-27T09:15:47.601 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-27T09:15:47.601 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-27T09:15:47.601 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-27T09:15:47.601 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-27T09:15:47.601 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-27T09:15:47.601 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-27T09:15:47.601 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-27T09:15:47.601 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-27T09:15:47.601 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-27T09:15:47.601 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-27T09:15:47.601 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-27T09:15:47.601 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-27T09:15:47.601 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-27T09:15:47.602 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-27T09:15:47.602 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-27T09:15:47.602 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-27T09:15:47.602 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-27T09:15:47.602 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-27T09:15:47.602 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-27T09:15:47.602 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-27T09:15:47.602 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-27T09:15:47.602 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-27T09:15:47.602 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-27T09:15:47.602 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-27T09:15:47.602 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-27T09:15:47.602 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-27T09:15:47.603 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-27T09:15:47.603 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-27T09:15:47.603 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-27T09:15:47.603 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-27T09:15:47.603 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-27T09:15:47.603 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-27T09:15:47.603 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-27T09:15:47.603 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-27T09:15:47.603 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-27T09:15:47.603 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-27T09:15:47.603 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-27T09:15:47.603 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-27T09:15:47.603 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-27T09:15:47.603 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-27T09:15:47.604 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-27T09:15:47.604 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-27T09:15:47.604 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-27T09:15:47.604 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-27T09:15:47.604 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-27T09:15:47.604 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-27T09:15:47.604 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-27T09:15:47.604 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-27T09:15:47.604 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-27T09:15:47.604 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-27T09:15:47.604 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-27T09:15:47.604 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-27T09:15:47.604 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-27T09:15:47.604 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-27T09:15:47.604 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-27T09:15:47.605 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-27T09:15:47.605 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-27T09:15:47.605 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-27T09:15:47.605 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-27T09:15:47.605 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-27T09:15:47.605 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-27T09:15:47.605 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-27T09:15:47.605 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-27T09:15:47.605 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-27T09:15:47.605 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-27T09:15:47.605 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-27T09:15:47.605 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-27T09:15:47.605 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-27T09:15:47.605 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-27T09:15:47.605 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-27T09:15:47.606 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-27T09:15:47.606 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-27T09:15:47.606 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-27T09:15:47.606 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-27T09:15:47.606 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-27T09:15:47.606 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-27T09:15:47.606 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-27T09:15:47.606 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-27T09:15:47.606 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-27T09:15:47.606 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-27T09:15:47.606 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-27T09:15:47.606 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-27T09:15:47.606 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-27T09:15:47.607 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-27T09:15:47.607 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-27T09:15:47.607 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-27T09:15:47.607 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-27T09:15:47.607 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-27T09:15:47.607 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-27T09:15:47.607 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-27T09:15:47.607 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-27T09:15:47.607 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-27T09:15:47.607 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-27T09:15:47.607 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-27T09:15:47.607 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-27T09:15:47.607 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-27T09:15:47.607 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-27T09:15:47.607 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-27T09:15:47.608 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-27T09:15:47.608 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-27T09:15:47.608 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-27T09:15:47.608 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-27T09:15:47.608 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-27T09:15:47.608 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-27T09:15:47.608 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-27T09:15:47.608 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-27T09:15:47.608 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-27T09:15:47.608 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-27T09:15:47.608 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-27T09:15:47.608 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-27T09:15:47.608 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-27T09:15:47.609 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-27T09:15:47.609 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-27T09:15:47.609 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-27T09:15:47.609 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-27T09:15:47.609 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-27T09:15:47.609 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-27T09:15:47.609 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-27T09:15:47.609 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-27T09:15:47.609 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-27T09:15:47.609 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-27T09:15:47.609 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-27T09:15:47.609 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-27T09:15:47.609 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-27T09:15:47.610 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-27T09:15:47.610 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-27T09:15:47.610 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-27T09:15:47.610 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-27T09:15:47.610 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-27T09:15:47.610 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-27T09:15:47.610 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-27T09:15:47.610 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-27T09:15:47.610 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-27T09:15:47.610 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-27T09:15:47.610 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-27T09:15:47.610 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-27T09:15:47.610 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-27T09:15:47.610 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-27T09:15:47.611 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-27T09:15:47.611 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-27T09:15:47.611 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-27T09:15:47.611 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-27T09:15:47.611 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-27T09:15:47.611 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-27T09:15:47.611 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-27T09:15:47.611 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-27T09:15:47.611 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-27T09:15:47.611 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-27T09:15:47.611 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-27T09:15:47.611 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-27T09:15:47.611 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-27T09:15:47.611 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-27T09:15:47.612 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-27T09:15:47.612 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-27T09:15:47.612 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-27T09:15:47.612 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-27T09:15:47.612 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-27T09:15:47.612 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-27T09:15:47.612 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-27T09:15:47.612 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-27T09:15:47.612 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-27T09:15:47.612 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-27T09:15:47.612 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-27T09:15:47.612 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-27T09:15:47.612 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-27T09:15:47.612 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-27T09:15:47.612 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-27T09:15:47.613 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-27T09:15:47.613 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-27T09:15:47.613 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-27T09:15:47.613 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-27T09:15:47.613 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-27T09:15:47.613 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-27T09:15:47.613 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-27T09:15:47.613 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-27T09:15:47.613 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-27T09:15:47.613 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-27T09:15:47.613 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-27T09:15:47.613 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-27T09:15:47.613 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-27T09:15:47.613 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-27T09:15:47.613 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-27T09:15:47.614 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-27T09:15:47.614 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-27T09:15:47.614 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-27T09:15:47.614 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-27T09:15:47.614 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-27T09:15:47.614 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-27T09:15:47.614 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-27T09:15:47.614 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-27T09:15:47.614 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-27T09:15:47.614 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-27T09:15:47.614 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-27T09:15:47.614 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-27T09:15:47.614 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-27T09:15:47.615 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-27T09:15:47.615 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-27T09:15:47.615 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-27T09:15:47.615 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-27T09:15:47.615 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-27T09:15:47.615 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-27T09:15:47.615 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-27T09:15:47.615 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-27T09:15:47.615 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-27T09:15:47.615 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-27T09:15:47.615 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-27T09:15:47.615 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-27T09:15:47.615 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-27T09:15:47.615 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-27T09:15:47.616 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-27T09:15:47.616 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-27T09:15:47.616 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-27T09:15:47.616 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-27T09:15:47.616 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-27T09:15:47.616 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-27T09:15:47.616 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-27T09:15:47.616 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-27T09:15:47.616 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-27T09:15:47.616 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-27T09:15:47.616 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-27T09:15:47.616 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-27T09:15:47.616 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-27T09:15:47.616 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-27T09:15:47.617 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-27T09:15:47.617 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-27T09:15:47.617 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-27T09:15:47.617 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-27T09:15:47.617 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-27T09:15:47.617 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-27T09:15:47.617 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-27T09:15:47.617 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-27T09:15:47.617 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-27T09:15:47.617 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-27T09:15:47.617 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-27T09:15:47.617 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-27T09:15:47.618 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-27T09:15:47.618 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-27T09:15:47.618 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-27T09:15:47.618 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-27T09:15:47.618 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-27T09:15:47.618 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-27T09:15:47.618 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-27T09:15:47.618 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-27T09:15:47.618 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-27T09:15:47.618 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-27T09:15:47.618 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-27T09:15:47.618 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-27T09:15:47.618 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-27T09:15:47.619 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-27T09:15:47.619 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-27T09:15:47.619 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-27T09:15:47.619 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-27T09:15:47.619 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-27T09:15:47.619 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-27T09:15:47.619 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-27T09:15:47.619 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-27T09:15:47.619 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-27T09:15:47.619 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-27T09:15:47.619 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-27T09:15:47.619 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-27T09:15:47.619 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-27T09:15:47.620 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-27T09:15:47.620 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-27T09:15:47.620 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-27T09:15:47.620 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-27T09:15:47.620 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-27T09:15:47.620 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-27T09:15:47.620 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-27T09:15:47.620 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-27T09:15:47.620 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-27T09:15:47.620 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-27T09:15:47.620 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-27T09:15:47.620 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-27T09:15:47.620 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-27T09:15:47.621 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-27T09:15:47.621 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-27T09:15:47.621 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-27T09:15:47.621 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-27T09:15:47.621 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-27T09:15:47.621 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-27T09:15:47.621 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-27T09:15:47.621 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-27T09:15:47.621 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-27T09:15:47.621 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-27T09:15:47.621 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-27T09:15:47.621 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-27T09:15:47.621 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-27T09:15:47.622 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-27T09:15:47.622 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-27T09:15:47.622 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-27T09:15:47.622 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-27T09:15:47.622 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-27T09:15:47.622 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-27T09:15:47.622 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-27T09:15:47.622 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-27T09:15:47.622 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-27T09:15:47.622 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-27T09:15:47.622 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-27T09:15:47.622 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-27T09:15:47.623 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-27T09:15:47.623 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-27T09:15:47.623 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-27T09:15:47.623 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-27T09:15:47.623 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-27T09:15:47.623 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-27T09:15:47.623 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-27T09:15:47.623 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-27T09:15:47.623 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-27T09:15:47.623 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-27T09:15:47.623 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-27T09:15:47.623 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-27T09:15:47.623 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-27T09:15:47.624 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-27T09:15:47.624 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-27T09:15:47.624 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-27T09:15:47.624 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-27T09:15:47.624 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-27T09:15:47.624 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-27T09:15:47.624 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-27T09:15:47.624 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-27T09:15:47.624 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-27T09:15:47.624 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-27T09:15:47.624 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-27T09:15:47.624 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-27T09:15:47.624 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-27T09:15:47.624 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-27T09:15:47.625 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-27T09:15:47.625 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-27T09:15:47.625 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-27T09:15:47.625 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-27T09:15:47.625 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-27T09:15:47.625 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-27T09:15:47.625 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-27T09:15:47.625 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-27T09:15:47.625 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-27T09:15:47.625 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-27T09:15:47.625 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-27T09:15:47.625 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-27T09:15:47.625 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-27T09:15:47.626 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-27T09:15:47.626 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-27T09:15:47.626 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-27T09:15:47.626 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-27T09:15:47.626 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-27T09:15:47.626 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-27T09:15:47.626 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-27T09:15:47.626 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-27T09:15:47.626 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-27T09:15:47.626 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-27T09:15:47.626 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-27T09:15:47.626 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-27T09:15:47.626 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-27T09:15:47.627 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-27T09:15:47.627 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-27T09:15:47.627 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-27T09:15:47.627 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-27T09:15:47.627 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-27T09:15:47.627 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-27T09:15:47.627 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-27T09:15:47.627 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-27T09:15:47.627 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-27T09:15:47.627 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-27T09:15:47.627 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-27T09:15:47.627 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-27T09:15:47.628 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-27T09:15:47.628 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-27T09:15:47.628 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-27T09:15:47.628 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-27T09:15:47.628 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-27T09:15:47.628 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-27T09:15:47.628 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-27T09:15:47.628 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-27T09:15:47.628 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-27T09:15:47.628 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-27T09:15:47.628 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-27T09:15:47.628 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-27T09:15:47.628 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-27T09:15:47.628 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-27T09:15:47.629 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-27T09:15:47.629 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-27T09:15:47.629 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-27T09:15:47.629 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-27T09:15:47.629 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-27T09:15:47.629 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-27T09:15:47.629 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-27T09:15:47.629 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-27T09:15:47.629 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-27T09:15:47.629 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-27T09:15:47.629 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-27T09:15:47.629 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-27T09:15:47.629 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-27T09:15:47.629 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-27T09:15:47.630 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-27T09:15:47.630 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-27T09:15:47.630 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-27T09:15:47.630 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-27T09:15:47.630 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-27T09:15:47.630 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-27T09:15:47.630 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-27T09:15:47.630 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-27T09:15:47.630 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-27T09:15:47.630 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-27T09:15:47.630 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-27T09:15:47.630 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-27T09:15:47.630 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-27T09:15:47.631 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-27T09:15:47.631 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-27T09:15:47.631 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-27T09:15:47.631 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-27T09:15:47.631 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-27T09:15:47.631 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-27T09:15:47.631 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-27T09:15:47.631 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-27T09:15:47.631 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-27T09:15:47.631 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-27T09:15:47.631 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-27T09:15:47.631 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-27T09:15:47.631 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-27T09:15:47.632 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-27T09:15:47.632 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-27T09:15:47.632 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-27T09:15:47.632 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-27T09:15:47.632 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-27T09:15:47.632 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-27T09:15:47.632 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-27T09:15:47.632 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-27T09:15:47.632 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-27T09:15:47.632 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-27T09:15:47.632 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-27T09:15:47.632 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-27T09:15:47.632 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-27T09:15:47.632 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-27T09:15:47.633 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-27T09:15:47.633 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-27T09:15:47.633 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-27T09:15:47.633 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-27T09:15:47.633 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-27T09:15:47.633 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-27T09:15:47.633 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-27T09:15:47.633 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-27T09:15:47.633 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-27T09:15:47.633 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-27T09:15:47.633 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-27T09:15:47.633 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-27T09:15:47.633 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-27T09:15:47.633 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-27T09:15:47.634 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-27T09:15:47.634 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-27T09:15:47.634 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-27T09:15:47.634 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-27T09:15:47.634 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-27T09:15:47.634 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-27T09:15:47.634 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-27T09:15:47.634 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-27T09:15:47.634 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-27T09:15:47.634 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-27T09:15:47.634 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-27T09:15:47.634 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-27T09:15:47.634 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-27T09:15:47.635 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-27T09:15:47.635 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-27T09:15:47.635 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-27T09:15:47.635 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-27T09:15:47.635 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-27T09:15:47.635 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-27T09:15:47.635 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-27T09:15:47.635 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-27T09:15:47.635 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-27T09:15:47.635 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-27T09:15:47.635 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-27T09:15:47.635 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-27T09:15:47.635 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-27T09:15:47.635 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-27T09:15:47.636 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-27T09:15:47.636 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-27T09:15:47.636 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-27T09:15:47.636 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-27T09:15:47.636 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-27T09:15:47.636 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-27T09:15:47.636 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-27T09:15:47.636 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-27T09:15:47.636 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-27T09:15:47.636 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-27T09:15:47.636 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-27T09:15:47.636 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-27T09:15:47.636 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-27T09:15:47.637 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-27T09:15:47.637 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-27T09:15:47.637 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-27T09:15:47.637 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-27T09:15:47.637 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-27T09:15:47.637 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-27T09:15:47.637 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-27T09:15:47.637 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-27T09:15:47.637 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-27T09:15:47.637 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-27T09:15:47.637 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-27T09:15:47.637 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-27T09:15:47.637 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-27T09:15:47.637 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-27T09:15:47.638 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-27T09:15:47.638 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-27T09:15:47.638 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-27T09:15:47.638 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-27T09:15:47.638 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-27T09:15:47.638 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-27T09:15:47.638 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-27T09:15:47.638 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-27T09:15:47.638 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-27T09:15:47.638 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-27T09:15:47.638 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-27T09:15:47.638 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-27T09:15:47.638 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-27T09:15:47.639 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-27T09:15:47.639 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-27T09:15:47.639 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-27T09:15:47.639 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-27T09:15:47.639 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-27T09:15:47.639 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-27T09:15:47.639 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-27T09:15:47.639 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-27T09:15:47.639 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-27T09:15:47.639 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-27T09:15:47.639 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-27T09:15:47.639 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-27T09:15:47.639 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-27T09:15:47.639 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-27T09:15:47.640 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-27T09:15:47.640 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-27T09:15:47.640 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-27T09:15:47.640 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-27T09:15:47.640 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-27T09:15:47.640 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-27T09:15:47.640 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-27T09:15:47.640 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-27T09:15:47.640 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-27T09:15:47.640 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-27T09:15:47.640 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-27T09:15:47.640 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-27T09:15:47.640 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-27T09:15:47.640 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-27T09:15:47.641 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-27T09:15:47.641 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-27T09:15:47.641 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-27T09:15:47.641 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-27T09:15:47.641 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-27T09:15:47.641 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-27T09:15:47.641 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-27T09:15:47.641 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-27T09:15:47.641 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-27T09:15:47.641 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-27T09:15:47.641 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-27T09:15:47.641 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-27T09:15:47.641 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-27T09:15:47.641 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-27T09:15:47.642 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-27T09:15:47.642 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-27T09:15:47.642 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-27T09:15:47.642 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-27T09:15:47.642 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-27T09:15:47.642 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-27T09:15:47.642 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-27T09:15:47.642 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-27T09:15:47.642 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-27T09:15:47.642 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-27T09:15:47.642 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-27T09:15:47.642 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-27T09:15:47.642 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-27T09:15:47.643 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-27T09:15:47.643 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-27T09:15:47.643 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-27T09:15:47.643 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-27T09:15:47.643 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-27T09:15:47.643 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-27T09:15:47.643 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-27T09:15:47.643 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-27T09:15:47.643 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-27T09:15:47.643 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-27T09:15:47.643 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-27T09:15:47.643 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-27T09:15:47.643 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-27T09:15:47.643 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-27T09:15:47.644 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-27T09:15:47.644 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-27T09:15:47.644 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-27T09:15:47.644 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-27T09:15:47.644 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-27T09:15:47.644 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-27T09:15:47.644 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-27T09:15:47.644 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-27T09:15:47.644 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-27T09:15:47.644 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-27T09:15:47.644 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-27T09:15:47.644 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-27T09:15:47.644 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-27T09:15:47.644 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-27T09:15:47.644 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-27T09:15:47.645 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-27T09:15:47.645 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-27T09:15:47.645 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-27T09:15:47.645 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-27T09:15:47.645 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-27T09:15:47.645 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-27T09:15:47.645 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-27T09:15:47.645 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-27T09:15:47.645 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-27T09:15:47.645 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-27T09:15:47.645 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-27T09:15:47.645 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-27T09:15:47.645 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-27T09:15:47.645 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-27T09:15:47.645 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-27T09:15:47.646 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-27T09:15:47.646 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-27T09:15:47.646 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-27T09:15:47.646 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-27T09:15:47.646 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-27T09:15:47.646 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-27T09:15:47.646 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-27T09:15:47.646 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-27T09:15:47.646 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-27T09:15:47.646 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-27T09:15:47.646 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-27T09:15:47.646 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-27T09:15:47.646 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-27T09:15:47.647 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-27T09:15:47.647 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-27T09:15:47.647 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-27T09:15:47.647 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-27T09:15:47.647 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-27T09:15:47.647 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-27T09:15:47.647 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-27T09:15:47.647 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-27T09:15:47.647 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-27T09:15:47.647 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-27T09:15:47.647 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-27T09:15:47.647 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-27T09:15:47.647 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-27T09:15:47.647 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-27T09:15:47.648 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-27T09:15:47.648 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-27T09:15:47.648 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-27T09:15:47.648 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-27T09:15:47.648 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-27T09:15:47.648 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-27T09:15:47.648 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-27T09:15:47.648 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-27T09:15:47.648 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-27T09:15:47.648 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-27T09:15:47.648 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-27T09:15:47.648 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-27T09:15:47.649 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-27T09:15:47.649 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-27T09:15:47.649 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-27T09:15:47.649 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-27T09:15:47.649 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-27T09:15:47.649 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-27T09:15:47.649 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-27T09:15:47.649 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-27T09:15:47.649 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-27T09:15:47.649 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-27T09:15:47.649 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-27T09:15:47.649 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-27T09:15:47.649 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-27T09:15:47.649 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-27T09:15:47.650 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-27T09:15:47.650 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-27T09:15:47.650 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-27T09:15:47.650 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-27T09:15:47.650 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-27T09:15:47.650 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-27T09:15:47.650 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-27T09:15:47.650 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-27T09:15:47.650 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-27T09:15:47.650 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-27T09:15:47.650 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-27T09:15:47.650 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-27T09:15:47.650 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-27T09:15:47.651 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-27T09:15:47.651 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-27T09:15:47.651 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-27T09:15:47.651 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-27T09:15:47.651 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-27T09:15:47.651 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-27T09:15:47.651 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-27T09:15:47.651 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-27T09:15:47.651 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-27T09:15:47.651 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-27T09:15:47.651 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-27T09:15:47.651 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-27T09:15:47.651 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-27T09:15:47.652 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-27T09:15:47.652 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-27T09:15:47.652 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-27T09:15:47.652 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-27T09:15:47.652 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-27T09:15:47.652 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-27T09:15:47.652 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-27T09:15:47.652 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-27T09:15:47.652 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-27T09:15:47.652 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-27T09:15:47.652 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-27T09:15:47.652 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-27T09:15:47.653 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-27T09:15:47.653 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-27T09:15:47.653 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-27T09:15:47.653 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-27T09:15:47.653 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-27T09:15:47.653 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-27T09:15:47.653 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-27T09:15:47.653 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-27T09:15:47.653 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-27T09:15:47.653 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-27T09:15:47.653 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-27T09:15:47.653 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-27T09:15:47.653 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-27T09:15:47.653 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-27T09:15:47.654 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-27T09:15:47.654 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-27T09:15:47.654 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-27T09:15:47.654 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-27T09:15:47.654 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-27T09:15:47.654 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-27T09:15:47.654 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-27T09:15:47.654 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-27T09:15:47.654 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-27T09:15:47.654 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-27T09:15:47.654 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-27T09:15:47.655 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-27T09:15:47.655 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-27T09:15:47.655 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-27T09:15:47.655 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-27T09:15:47.655 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-27T09:15:47.655 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-27T09:15:47.655 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-27T09:15:47.655 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-27T09:15:47.655 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-27T09:15:47.655 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-27T09:15:47.655 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-27T09:15:47.655 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-27T09:15:47.655 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-27T09:15:47.656 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-27T09:15:47.656 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-27T09:15:47.656 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-27T09:15:47.656 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-27T09:15:47.656 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-27T09:15:47.656 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-27T09:15:47.656 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-27T09:15:47.656 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-27T09:15:47.656 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-27T09:15:47.656 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-27T09:15:47.656 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-27T09:15:47.656 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-27T09:15:47.656 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-27T09:15:47.657 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-27T09:15:47.657 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-27T09:15:47.657 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-27T09:15:47.657 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-27T09:15:47.657 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-27T09:15:47.657 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-27T09:15:47.657 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-27T09:15:47.657 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-27T09:15:47.657 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-27T09:15:47.657 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-27T09:15:47.657 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-27T09:15:47.657 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-27T09:15:47.657 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-27T09:15:47.657 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-27T09:15:47.658 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-27T09:15:47.658 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-27T09:15:47.658 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-27T09:15:47.658 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-27T09:15:47.658 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-27T09:15:47.658 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-27T09:15:47.658 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-27T09:15:47.658 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-27T09:15:47.658 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-27T09:15:47.658 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-27T09:15:47.658 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-27T09:15:47.658 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-27T09:15:47.658 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-27T09:15:47.658 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-27T09:15:47.659 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-27T09:15:47.659 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-27T09:15:47.659 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-27T09:15:47.659 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-27T09:15:47.659 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-27T09:15:47.659 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-27T09:15:47.659 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-27T09:15:47.659 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-27T09:15:47.659 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-27T09:15:47.659 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-27T09:15:47.659 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-27T09:15:47.659 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-27T09:15:47.659 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-27T09:15:47.659 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-27T09:15:47.659 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-27T09:15:47.660 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-27T09:15:47.660 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-27T09:15:47.660 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-27T09:15:47.660 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-27T09:15:47.660 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-27T09:15:47.660 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-27T09:15:47.660 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-27T09:15:47.660 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-27T09:15:47.660 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-27T09:15:47.660 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-27T09:15:47.660 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-27T09:15:47.660 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-27T09:15:47.660 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-27T09:15:47.661 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-27T09:15:47.661 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-27T09:15:47.661 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-27T09:15:47.661 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-27T09:15:47.661 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-27T09:15:47.661 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-27T09:15:47.661 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-27T09:15:47.661 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-27T09:15:47.661 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-27T09:15:47.661 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-27T09:15:47.661 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-27T09:15:47.661 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-27T09:15:47.661 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-27T09:15:47.661 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-27T09:15:47.662 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-27T09:15:47.662 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-27T09:15:47.662 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-27T09:15:47.662 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-27T09:15:47.662 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-27T09:15:47.662 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-27T09:15:47.662 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-27T09:15:47.662 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-27T09:15:47.662 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-27T09:15:47.662 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-27T09:15:47.662 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-27T09:15:47.662 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-27T09:15:47.663 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-27T09:15:47.663 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-27T09:15:47.663 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-27T09:15:47.663 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-27T09:15:47.663 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-27T09:15:47.663 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-27T09:15:47.663 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-27T09:15:47.663 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-27T09:15:47.663 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-27T09:15:47.663 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-27T09:15:47.663 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-27T09:15:47.664 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-27T09:15:47.664 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-27T09:15:47.664 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-27T09:15:47.664 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-27T09:15:47.664 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-27T09:15:47.664 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-27T09:15:47.664 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-27T09:15:47.664 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-27T09:15:47.664 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-27T09:15:47.664 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-27T09:15:47.664 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-27T09:15:47.664 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-27T09:15:47.664 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-27T09:15:47.664 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-27T09:15:47.665 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-27T09:15:47.665 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-27T09:15:47.665 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-27T09:15:47.665 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-27T09:15:47.665 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-27T09:15:47.665 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-27T09:15:47.665 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-27T09:15:47.665 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-27T09:15:47.665 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-27T09:15:47.665 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-27T09:15:47.665 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-27T09:15:47.665 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-27T09:15:47.665 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-27T09:15:47.665 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-27T09:15:47.666 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-27T09:15:47.666 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-27T09:15:47.666 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-27T09:15:47.666 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-27T09:15:47.666 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-27T09:15:47.666 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-27T09:15:47.666 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-27T09:15:47.666 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-27T09:15:47.666 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-27T09:15:47.666 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-27T09:15:47.666 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-27T09:15:47.666 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-27T09:15:47.666 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-27T09:15:47.667 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-27T09:15:47.667 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-27T09:15:47.667 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-27T09:15:47.667 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-27T09:15:47.667 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-27T09:15:47.667 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-27T09:15:47.667 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-27T09:15:47.667 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-27T09:15:47.667 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-27T09:15:47.667 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-27T09:15:47.667 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-27T09:15:47.667 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-27T09:15:47.667 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-27T09:15:47.667 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-27T09:15:47.667 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-27T09:15:47.668 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-27T09:15:47.668 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-27T09:15:47.668 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-27T09:15:47.668 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-27T09:15:47.668 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-27T09:15:47.668 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-27T09:15:47.668 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-27T09:15:47.668 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-27T09:15:47.668 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-27T09:15:47.668 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-27T09:15:47.668 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-27T09:15:47.668 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-27T09:15:47.668 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-27T09:15:47.668 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-27T09:15:47.668 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-27T09:15:47.669 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-27T09:15:47.669 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-27T09:15:47.669 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-27T09:15:47.669 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-27T09:15:47.669 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-27T09:15:47.669 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-27T09:15:47.669 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-27T09:15:47.669 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-27T09:15:47.669 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-27T09:15:47.669 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-27T09:15:47.669 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-27T09:15:47.669 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-27T09:15:47.669 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-27T09:15:47.669 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-27T09:15:47.669 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-27T09:15:47.669 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-27T09:15:47.670 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-27T09:15:47.670 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-27T09:15:47.670 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-27T09:15:47.670 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-27T09:15:47.670 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-27T09:15:47.670 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-27T09:15:47.670 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-27T09:15:47.670 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-27T09:15:47.670 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-27T09:15:47.670 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-27T09:15:47.670 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-27T09:15:47.670 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-27T09:15:47.670 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-27T09:15:47.670 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-27T09:15:47.671 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-27T09:15:47.671 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-27T09:15:47.671 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-27T09:15:47.671 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-27T09:15:47.671 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-27T09:15:47.671 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-27T09:15:47.671 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-27T09:15:47.671 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-27T09:15:47.671 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-27T09:15:47.671 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-27T09:15:47.671 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-27T09:15:47.671 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-27T09:15:47.671 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-27T09:15:47.671 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-27T09:15:47.672 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-27T09:15:47.672 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-27T09:15:47.672 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-27T09:15:47.672 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-27T09:15:47.672 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-27T09:15:47.672 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-27T09:15:47.672 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-27T09:15:47.672 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-27T09:15:47.672 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-27T09:15:47.672 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-27T09:15:47.672 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-27T09:15:47.672 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-27T09:15:47.672 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-27T09:15:47.672 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-27T09:15:47.672 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-27T09:15:47.673 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-27T09:15:47.673 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-27T09:15:47.673 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-27T09:15:47.673 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-27T09:15:47.673 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-27T09:15:47.673 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-27T09:15:47.673 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-27T09:15:47.673 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-27T09:15:47.673 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-27T09:15:47.673 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-27T09:15:47.673 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-27T09:15:47.673 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-27T09:15:47.673 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-27T09:15:47.673 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-27T09:15:47.673 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-27T09:15:47.674 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-27T09:15:47.674 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-27T09:15:47.674 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-27T09:15:47.674 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-27T09:15:47.674 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-27T09:15:47.674 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-27T09:15:47.674 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-27T09:15:47.674 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-27T09:15:47.674 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-27T09:15:47.674 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-27T09:15:47.674 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-27T09:15:47.674 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-27T09:15:47.674 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-27T09:15:47.674 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-27T09:15:47.675 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-27T09:15:47.675 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-27T09:15:47.675 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-27T09:15:47.675 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-27T09:15:47.675 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-27T09:15:47.675 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-27T09:15:47.675 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-27T09:15:47.675 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-27T09:15:47.675 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-27T09:15:47.675 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-27T09:15:47.675 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-27T09:15:47.675 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-27T09:15:47.675 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-27T09:15:47.675 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-27T09:15:47.676 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-27T09:15:47.676 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-27T09:15:47.676 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-27T09:15:47.676 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-27T09:15:47.676 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-27T09:15:47.676 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-27T09:15:47.676 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-27T09:15:47.676 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-27T09:15:47.676 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-27T09:15:47.676 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-27T09:15:47.676 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-27T09:15:47.676 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-27T09:15:47.676 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-27T09:15:47.676 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-27T09:15:47.676 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-27T09:15:47.677 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-27T09:15:47.677 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-27T09:15:47.677 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-27T09:15:47.677 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-27T09:15:47.677 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-27T09:15:47.677 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-27T09:15:47.677 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-27T09:15:47.677 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-27T09:15:47.677 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-27T09:15:47.677 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-27T09:15:47.677 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-27T09:15:47.677 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-27T09:15:47.677 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-27T09:15:47.677 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-27T09:15:47.677 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-27T09:15:47.678 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-27T09:15:47.678 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-27T09:15:47.678 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-27T09:15:47.678 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-27T09:15:47.678 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-27T09:15:47.678 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-27T09:15:47.678 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-27T09:15:47.678 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-27T09:15:47.678 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-27T09:15:47.678 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-27T09:15:47.678 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-27T09:15:47.678 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-27T09:15:47.678 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-27T09:15:47.679 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-27T09:15:47.679 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-27T09:15:47.679 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-27T09:15:47.679 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-27T09:15:47.679 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-27T09:15:47.679 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-27T09:15:47.679 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-27T09:15:47.679 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-27T09:15:47.679 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-27T09:15:47.679 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-27T09:15:47.679 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-27T09:15:47.679 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-27T09:15:47.679 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-27T09:15:47.680 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-27T09:15:47.680 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-27T09:15:47.680 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-27T09:15:47.680 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-27T09:15:47.680 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-27T09:15:47.680 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-27T09:15:47.680 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-27T09:15:47.680 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-27T09:15:47.680 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-27T09:15:47.680 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-27T09:15:47.680 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-27T09:15:47.680 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-27T09:15:47.680 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-27T09:15:47.680 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-27T09:15:47.681 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-27T09:15:47.681 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-27T09:15:47.681 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-27T09:15:47.681 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-27T09:15:47.681 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-27T09:15:47.681 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-27T09:15:47.681 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-27T09:15:47.681 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-27T09:15:47.681 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-27T09:15:47.681 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-27T09:15:47.681 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-27T09:15:47.681 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-27T09:15:47.681 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-27T09:15:47.681 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-27T09:15:47.682 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-27T09:15:47.682 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-27T09:15:47.682 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-27T09:15:47.682 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-27T09:15:47.682 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-27T09:15:47.682 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-27T09:15:47.682 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-27T09:15:47.682 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-27T09:15:47.682 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-27T09:15:47.682 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-27T09:15:47.682 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-27T09:15:47.683 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-27T09:15:47.683 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-27T09:15:47.683 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-27T09:15:47.683 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-27T09:15:47.683 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-27T09:15:47.683 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-27T09:15:47.683 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-27T09:15:47.683 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-27T09:15:47.683 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-27T09:15:47.683 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-27T09:15:47.683 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-27T09:15:47.683 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-27T09:15:47.683 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-27T09:15:47.684 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-27T09:15:47.684 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-27T09:15:47.684 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-27T09:15:47.684 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-27T09:15:47.684 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-27T09:15:47.684 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-27T09:15:47.684 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-27T09:15:47.684 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-27T09:15:47.684 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-27T09:15:47.684 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-27T09:15:47.684 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-27T09:15:47.684 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-27T09:15:47.684 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-27T09:15:47.685 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-27T09:15:47.685 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-27T09:15:47.685 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-27T09:15:47.685 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-27T09:15:47.685 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-27T09:15:47.685 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-27T09:15:47.685 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-27T09:15:47.685 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-27T09:15:47.685 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-27T09:15:47.685 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-27T09:15:47.685 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-27T09:15:47.685 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-27T09:15:47.685 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-27T09:15:47.685 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-27T09:15:47.686 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-27T09:15:47.686 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-27T09:15:47.686 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-27T09:15:47.686 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-27T09:15:47.686 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-27T09:15:47.686 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-27T09:15:47.686 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-27T09:15:47.686 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-27T09:15:47.686 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-27T09:15:47.686 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-27T09:15:47.686 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-27T09:15:47.686 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-27T09:15:47.686 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-27T09:15:47.686 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-27T09:15:47.687 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-27T09:15:47.687 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-27T09:15:47.687 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-27T09:15:47.687 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-27T09:15:47.687 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-27T09:15:47.687 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-27T09:15:47.687 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-27T09:15:47.687 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-27T09:15:47.687 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-27T09:15:47.687 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-27T09:15:47.687 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-27T09:15:47.687 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-27T09:15:47.687 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-27T09:15:47.687 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-27T09:15:47.688 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-27T09:15:47.688 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-27T09:15:47.688 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-27T09:15:47.688 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-27T09:15:47.688 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-27T09:15:47.688 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-27T09:15:47.688 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-27T09:15:47.688 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-27T09:15:47.688 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-27T09:15:47.688 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-27T09:15:47.688 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-27T09:15:47.688 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-27T09:15:47.688 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-27T09:15:47.689 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-27T09:15:47.689 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-27T09:15:47.689 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-27T09:15:47.689 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-27T09:15:47.689 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-27T09:15:47.689 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-27T09:15:47.689 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-27T09:15:47.689 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-27T09:15:47.689 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-27T09:15:47.689 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-27T09:15:47.689 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-27T09:15:47.689 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-27T09:15:47.689 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-27T09:15:47.689 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-27T09:15:47.690 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-27T09:15:47.690 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-27T09:15:47.690 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-27T09:15:47.690 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-27T09:15:47.690 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-27T09:15:47.690 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-27T09:15:47.690 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-27T09:15:47.690 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-27T09:15:47.690 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-27T09:15:47.690 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-27T09:15:47.690 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-27T09:15:47.690 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-27T09:15:47.690 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-27T09:15:47.691 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-27T09:15:47.691 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-27T09:15:47.691 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-27T09:15:47.691 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-27T09:15:47.691 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-27T09:15:47.691 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-27T09:15:47.691 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-27T09:15:47.691 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-27T09:15:47.691 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-27T09:15:47.691 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-27T09:15:47.691 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-27T09:15:47.691 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-27T09:15:47.691 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-27T09:15:47.691 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-27T09:15:47.692 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-27T09:15:47.692 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-27T09:15:47.692 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-27T09:15:47.692 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-27T09:15:47.692 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-27T09:15:47.692 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-27T09:15:47.692 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-27T09:15:47.692 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-27T09:15:47.692 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-27T09:15:47.692 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-27T09:15:47.692 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-27T09:15:47.692 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-27T09:15:47.692 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-27T09:15:47.692 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-27T09:15:47.692 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-27T09:15:47.693 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-27T09:15:47.693 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-27T09:15:47.693 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-27T09:15:47.693 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-27T09:15:47.693 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-27T09:15:47.693 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-27T09:15:47.693 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-27T09:15:47.693 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-27T09:15:47.693 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-27T09:15:47.693 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-27T09:15:47.693 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-27T09:15:47.693 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-27T09:15:47.693 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-27T09:15:47.693 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-27T09:15:47.694 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-27T09:15:47.694 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-27T09:15:47.694 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-27T09:15:47.694 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-27T09:15:47.694 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-27T09:15:47.694 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-27T09:15:47.694 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-27T09:15:47.694 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-27T09:15:47.694 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-27T09:15:47.694 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-27T09:15:47.694 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-27T09:15:47.694 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-27T09:15:47.694 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-27T09:15:47.695 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-27T09:15:47.695 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-27T09:15:47.695 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-27T09:15:47.695 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-27T09:15:47.695 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-27T09:15:47.695 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-27T09:15:47.695 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-27T09:15:47.695 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-27T09:15:47.695 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-27T09:15:47.695 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-27T09:15:47.695 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-27T09:15:47.695 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-27T09:15:47.695 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-27T09:15:47.695 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-27T09:15:47.695 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-27T09:15:47.696 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-27T09:15:47.696 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-27T09:15:47.696 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-27T09:15:47.696 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-27T09:15:47.696 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-27T09:15:47.696 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-27T09:15:47.696 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-27T09:15:47.696 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-27T09:15:47.696 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-27T09:15:47.696 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-27T09:15:47.696 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-27T09:15:47.696 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-27T09:15:47.696 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-27T09:15:47.696 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-27T09:15:47.696 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-27T09:15:47.697 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-27T09:15:47.697 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-27T09:15:47.697 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-27T09:15:47.697 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-27T09:15:47.697 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-27T09:15:47.697 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-27T09:15:47.697 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-27T09:15:47.697 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-27T09:15:47.697 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-27T09:15:47.697 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-27T09:15:47.697 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-27T09:15:47.697 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-27T09:15:47.697 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-27T09:15:47.697 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-27T09:15:47.698 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-27T09:15:47.698 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-27T09:15:47.698 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-27T09:15:47.698 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-27T09:15:47.698 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-27T09:15:47.698 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-27T09:15:47.698 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-27T09:15:47.698 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-27T09:15:47.698 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-27T09:15:47.698 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-27T09:15:47.698 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-27T09:15:47.698 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-27T09:15:47.698 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-27T09:15:47.698 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-27T09:15:47.699 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-27T09:15:47.699 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-27T09:15:47.699 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-27T09:15:47.699 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-27T09:15:47.699 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-27T09:15:47.699 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-27T09:15:47.699 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-27T09:15:47.699 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-27T09:15:47.699 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-27T09:15:47.699 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-27T09:15:47.699 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-27T09:15:47.699 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-27T09:15:47.699 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-27T09:15:47.699 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-27T09:15:47.700 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-27T09:15:47.700 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-27T09:15:47.700 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-27T09:15:47.700 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-27T09:15:47.700 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-27T09:15:47.700 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-27T09:15:47.700 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-27T09:15:47.700 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-27T09:15:47.700 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-27T09:15:47.700 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-27T09:15:47.700 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-27T09:15:47.700 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-27T09:15:47.700 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-27T09:15:47.700 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-27T09:15:47.701 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-27T09:15:47.701 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-27T09:15:47.701 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-27T09:15:47.701 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-27T09:15:47.701 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-27T09:15:47.701 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-27T09:15:47.701 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-27T09:15:47.701 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-27T09:15:47.701 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-27T09:15:47.701 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-27T09:15:47.701 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-27T09:15:47.701 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-27T09:15:47.701 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-27T09:15:47.701 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-27T09:15:47.701 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-27T09:15:47.702 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-27T09:15:47.702 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-27T09:15:47.702 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-27T09:15:47.702 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-27T09:15:47.702 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-27T09:15:47.702 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-27T09:15:47.702 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-27T09:15:47.702 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-27T09:15:47.702 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-27T09:15:47.702 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-27T09:15:47.702 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-27T09:15:47.702 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-27T09:15:47.702 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-27T09:15:47.703 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-27T09:15:47.703 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-27T09:15:47.703 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-27T09:15:47.703 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-27T09:15:47.703 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-27T09:15:47.703 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-27T09:15:47.703 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-27T09:15:47.703 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-27T09:15:47.703 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-27T09:15:47.703 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-27T09:15:47.703 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-27T09:15:47.703 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-27T09:15:47.703 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-27T09:15:47.703 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-27T09:15:47.704 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-27T09:15:47.704 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-27T09:15:47.704 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-27T09:15:47.704 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-27T09:15:47.704 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-27T09:15:47.704 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-27T09:15:47.704 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-27T09:15:47.704 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-27T09:15:47.704 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-27T09:15:47.704 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-27T09:15:47.704 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-27T09:15:47.704 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-27T09:15:47.704 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-27T09:15:47.704 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-27T09:15:47.705 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-27T09:15:47.705 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-27T09:15:47.705 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-27T09:15:47.705 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-27T09:15:47.705 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-27T09:15:47.705 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-27T09:15:47.705 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-27T09:15:47.705 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-27T09:15:47.705 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-27T09:15:47.705 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-27T09:15:47.705 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T09:15:47.709 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-27T09:15:47.709 DEBUG:teuthology.orchestra.run.smithi114:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-27T09:15:47.732 INFO:teuthology.orchestra.run.smithi114.stdout:postinstall scriptlet (using /bin/sh): 2024-05-27T09:15:47.732 INFO:teuthology.orchestra.run.smithi114.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T09:15:47.732 INFO:teuthology.orchestra.run.smithi114.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T09:15:47.732 INFO:teuthology.orchestra.run.smithi114.stdout:fi 2024-05-27T09:15:47.732 INFO:teuthology.orchestra.run.smithi114.stdout:for file in vmlinuz System.map config; do 2024-05-27T09:15:47.732 INFO:teuthology.orchestra.run.smithi114.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-27T09:15:47.732 INFO:teuthology.orchestra.run.smithi114.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-27T09:15:47.732 INFO:teuthology.orchestra.run.smithi114.stdout: fi 2024-05-27T09:15:47.732 INFO:teuthology.orchestra.run.smithi114.stdout:done 2024-05-27T09:15:47.732 INFO:teuthology.orchestra.run.smithi114.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-27T09:15:47.733 INFO:teuthology.orchestra.run.smithi114.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-27T09:15:47.733 INFO:teuthology.orchestra.run.smithi114.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T09:15:47.733 INFO:teuthology.orchestra.run.smithi114.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T09:15:47.733 INFO:teuthology.orchestra.run.smithi114.stdout:fi 2024-05-27T09:15:47.733 INFO:teuthology.orchestra.run.smithi114.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-27T09:15:47.733 INFO:teuthology.orchestra.run.smithi114.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-27T09:15:47.733 INFO:teuthology.orchestra.run.smithi114.stdout:fi 2024-05-27T09:15:47.733 INFO:teuthology.orchestra.run.smithi114.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-27T09:15:47.733 INFO:teuthology.orchestra.run.smithi114.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T09:15:47.733 INFO:teuthology.orchestra.run.smithi114.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-27T09:15:47.733 INFO:teuthology.orchestra.run.smithi114.stdout:fi 2024-05-27T09:15:47.734 DEBUG:teuthology.orchestra.run.smithi114:> sudo rpm -qi grub2-tools 2024-05-27T09:15:47.807 INFO:teuthology.orchestra.run.smithi114.stdout:Name : grub2-tools 2024-05-27T09:15:47.807 INFO:teuthology.orchestra.run.smithi114.stdout:Epoch : 1 2024-05-27T09:15:47.807 INFO:teuthology.orchestra.run.smithi114.stdout:Version : 2.06 2024-05-27T09:15:47.807 INFO:teuthology.orchestra.run.smithi114.stdout:Release : 68.el9 2024-05-27T09:15:47.807 INFO:teuthology.orchestra.run.smithi114.stdout:Architecture: x86_64 2024-05-27T09:15:47.807 INFO:teuthology.orchestra.run.smithi114.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-27T09:15:47.807 INFO:teuthology.orchestra.run.smithi114.stdout:Group : Unspecified 2024-05-27T09:15:47.807 INFO:teuthology.orchestra.run.smithi114.stdout:Size : 8264254 2024-05-27T09:15:47.807 INFO:teuthology.orchestra.run.smithi114.stdout:License : GPLv3+ 2024-05-27T09:15:47.808 INFO:teuthology.orchestra.run.smithi114.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-27T09:15:47.808 INFO:teuthology.orchestra.run.smithi114.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-27T09:15:47.808 INFO:teuthology.orchestra.run.smithi114.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-27T09:15:47.808 INFO:teuthology.orchestra.run.smithi114.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-27T09:15:47.808 INFO:teuthology.orchestra.run.smithi114.stdout:Packager : builder@centos.org 2024-05-27T09:15:47.808 INFO:teuthology.orchestra.run.smithi114.stdout:Vendor : CentOS 2024-05-27T09:15:47.808 INFO:teuthology.orchestra.run.smithi114.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-27T09:15:47.808 INFO:teuthology.orchestra.run.smithi114.stdout:Summary : Support tools for GRUB. 2024-05-27T09:15:47.808 INFO:teuthology.orchestra.run.smithi114.stdout:Description : 2024-05-27T09:15:47.808 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:15:47.808 INFO:teuthology.orchestra.run.smithi114.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-27T09:15:47.808 INFO:teuthology.orchestra.run.smithi114.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-27T09:15:47.808 INFO:teuthology.orchestra.run.smithi114.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-27T09:15:47.809 INFO:teuthology.orchestra.run.smithi114.stdout:hardware devices. 2024-05-27T09:15:47.809 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:15:47.809 INFO:teuthology.orchestra.run.smithi114.stdout:This subpackage provides tools for support of all platforms. 2024-05-27T09:15:47.809 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-27T09:15:47.810 INFO:teuthology.task.kernel:Updating grub on smithi114 to boot 6.9.0-gb9459a26b088 2024-05-27T09:15:47.810 DEBUG:teuthology.orchestra.run.smithi114:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-27T09:15:48.366 INFO:teuthology.orchestra.run.smithi114.stderr:Generating grub configuration file ... 2024-05-27T09:15:49.741 INFO:teuthology.orchestra.run.smithi114.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-27T09:15:49.776 INFO:teuthology.orchestra.run.smithi114.stderr:done 2024-05-27T09:15:49.778 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2024-05-27T09:15:49.817 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.H5VlIvHvWb 2024-05-27T09:15:49.817 DEBUG:teuthology.orchestra.run.smithi114:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.H5VlIvHvWb 2024-05-27T09:15:49.882 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.H5VlIvHvWb 2024-05-27T09:15:50.017 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.H5VlIvHvWb is 6KB 2024-05-27T09:15:50.065 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.H5VlIvHvWb 2024-05-27T09:15:50.079 DEBUG:teuthology.orchestra.run.smithi114:> sudo /bin/ls /boot/loader/entries || true 2024-05-27T09:15:50.140 INFO:teuthology.orchestra.run.smithi114.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-27T09:15:50.140 INFO:teuthology.orchestra.run.smithi114.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-27T09:15:50.140 INFO:teuthology.orchestra.run.smithi114.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-27T09:15:50.141 INFO:teuthology.orchestra.run.smithi114.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-27T09:15:50.142 DEBUG:teuthology.orchestra.run.smithi114:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-27T09:15:50.288 DEBUG:teuthology.orchestra.run.smithi114:> sudo shutdown -r now 2024-05-27T09:16:20.291 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-27T09:16:20.292 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-05-27T09:16:20.293 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:16:38.814 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-05-27T09:16:47.815 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-05-27T09:16:47.816 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:16:50.878 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-05-27T09:17:02.879 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-05-27T09:17:02.880 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:17:34.726 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-05-27T09:17:35.331 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-05-27T09:17:35.331 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-gb9459a26b088"... 2024-05-27T09:17:35.331 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-05-27T09:17:35.368 INFO:teuthology.orchestra.run.smithi114.stdout:6.9.0-gb9459a26b088 2024-05-27T09:17:35.368 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-27T09:17:35.369 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-27T09:17:35.369 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-27T09:17:36.370 DEBUG:teuthology.parallel:result is None 2024-05-27T09:17:36.370 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-27T09:17:36.378 INFO:teuthology.task.internal:Creating test directory... 2024-05-27T09:17:36.378 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T09:17:36.383 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T09:17:36.427 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-27T09:17:36.433 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-27T09:17:36.443 INFO:teuthology.task.internal:Creating archive directory... 2024-05-27T09:17:36.443 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T09:17:36.447 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T09:17:36.499 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-27T09:17:36.506 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-27T09:17:36.506 DEBUG:teuthology.orchestra.run.smithi163:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T09:17:36.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:17:36.545 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T09:17:36.550 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T09:17:36.629 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T09:17:36.665 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T09:17:36.669 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T09:17:36.709 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T09:17:36.713 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-27T09:17:36.719 INFO:teuthology.task.internal:Configuring sudo... 2024-05-27T09:17:36.720 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T09:17:36.724 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T09:17:36.789 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-27T09:17:36.797 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-27T09:17:36.798 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T09:17:36.802 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T09:17:36.843 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T09:17:36.958 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T09:17:37.036 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T09:17:37.107 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T09:17:37.195 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:17:37.195 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T09:17:37.277 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T09:17:37.325 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T09:17:37.378 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T09:17:37.463 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T09:17:37.506 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-27T09:17:37.506 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T09:17:37.571 DEBUG:teuthology.orchestra.run.smithi114:> sudo service rsyslog restart 2024-05-27T09:17:37.579 DEBUG:teuthology.orchestra.run.smithi163:> sudo service rsyslog restart 2024-05-27T09:17:37.649 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T09:17:37.691 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T09:17:38.082 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-27T09:17:38.093 INFO:teuthology.task.internal:Starting timer... 2024-05-27T09:17:38.093 INFO:teuthology.run_tasks:Running task pcp... 2024-05-27T09:17:38.103 INFO:teuthology.run_tasks:Running task selinux... 2024-05-27T09:17:38.111 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-27T09:17:38.111 DEBUG:teuthology.orchestra.run.smithi114:> sudo service auditd rotate 2024-05-27T09:17:38.212 INFO:teuthology.orchestra.run.smithi114.stdout:Rotating logs: 2024-05-27T09:17:38.216 DEBUG:teuthology.orchestra.run.smithi163:> sudo service auditd rotate 2024-05-27T09:17:38.271 INFO:teuthology.orchestra.run.smithi163.stdout:Rotating logs: 2024-05-27T09:17:38.273 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-27T09:17:38.273 DEBUG:teuthology.orchestra.run.smithi114:> /usr/sbin/getenforce 2024-05-27T09:17:38.331 INFO:teuthology.orchestra.run.smithi114.stdout:Permissive 2024-05-27T09:17:38.331 DEBUG:teuthology.orchestra.run.smithi163:> /usr/sbin/getenforce 2024-05-27T09:17:38.351 INFO:teuthology.orchestra.run.smithi163.stdout:Permissive 2024-05-27T09:17:38.351 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi114.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi163.front.sepia.ceph.com': 'permissive'} 2024-05-27T09:17:38.351 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T09:17:38.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:17:38.428 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T09:17:38.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:17:38.455 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-27T09:17:38.455 DEBUG:teuthology.orchestra.run.smithi114:> sudo /usr/sbin/setenforce permissive 2024-05-27T09:17:38.653 DEBUG:teuthology.orchestra.run.smithi163:> sudo /usr/sbin/setenforce permissive 2024-05-27T09:17:38.679 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-27T09:17:38.689 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-27T09:17:38.701 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-27T09:17:38.701 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi114.front.sepia.ceph.com,smithi163.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-27T09:24:06.663 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi114.front.sepia.ceph.com'), Remote(name='ubuntu@smithi163.front.sepia.ceph.com')] 2024-05-27T09:24:06.664 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-05-27T09:24:06.666 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:24:06.774 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-05-27T09:24:06.921 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-05-27T09:24:06.921 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-05-27T09:24:06.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:24:06.996 DEBUG:teuthology.orchestra.run.smithi163:> true 2024-05-27T09:24:07.030 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-05-27T09:24:07.031 INFO:teuthology.run_tasks:Running task clock... 2024-05-27T09:24:07.041 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-27T09:24:07.041 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T09:24:07.042 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T09:24:07.049 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T09:24:07.049 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T09:24:07.099 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T09:24:07.113 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T09:24:07.136 INFO:teuthology.orchestra.run.smithi163.stderr:sudo: ntpd: command not found 2024-05-27T09:24:07.136 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T09:24:07.146 INFO:teuthology.orchestra.run.smithi163.stdout:506 Cannot talk to daemon 2024-05-27T09:24:07.159 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T09:24:07.172 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T09:24:07.175 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T09:24:07.230 INFO:teuthology.orchestra.run.smithi163.stderr:bash: line 1: ntpq: command not found 2024-05-27T09:24:07.234 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T09:24:07.234 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2024-05-27T09:24:07.234 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:24:07.234 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:24:07.235 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:24:07.235 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:24:07.246 INFO:teuthology.orchestra.run.smithi114.stderr:sudo: ntpd: command not found 2024-05-27T09:24:07.278 INFO:teuthology.orchestra.run.smithi114.stdout:506 Cannot talk to daemon 2024-05-27T09:24:07.320 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T09:24:07.362 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T09:24:07.499 INFO:teuthology.orchestra.run.smithi114.stderr:bash: line 1: ntpq: command not found 2024-05-27T09:24:07.509 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T09:24:07.509 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2024-05-27T09:24:07.509 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:24:07.510 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:24:07.510 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:24:07.510 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:24:07.511 INFO:teuthology.run_tasks:Running task install... 2024-05-27T09:24:07.520 DEBUG:teuthology.task.install:project ceph 2024-05-27T09:24:07.520 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d'}} 2024-05-27T09:24:07.520 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d'} 2024-05-27T09:24:07.520 INFO:teuthology.task.install:Using flavor: default 2024-05-27T09:24:07.525 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-27T09:24:07.526 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-27T09:24:07.526 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d', 'tag': None, 'wait_for_package': False} 2024-05-27T09:24:07.527 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:24:07.529 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d', 'tag': None, 'wait_for_package': False} 2024-05-27T09:24:07.529 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:24:07.751 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/ 2024-05-27T09:24:07.751 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3868.ga6507388 2024-05-27T09:24:07.760 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/ 2024-05-27T09:24:07.760 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3868.ga6507388 2024-05-27T09:24:07.864 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T09:24:07.864 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:24:07.864 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T09:24:07.929 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T09:24:07.930 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-27T09:24:07.930 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T09:24:07.953 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-27T09:24:07.953 DEBUG:teuthology.orchestra.run.smithi114:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a6507388dd5057528934822c0163b0c347ef1d5d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T09:24:07.956 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-27T09:24:07.956 DEBUG:teuthology.orchestra.run.smithi163:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a6507388dd5057528934822c0163b0c347ef1d5d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T09:24:08.022 DEBUG:teuthology.orchestra.run.smithi163:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T09:24:08.083 DEBUG:teuthology.orchestra.run.smithi114:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T09:24:08.096 DEBUG:teuthology.orchestra.run.smithi163:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T09:24:08.159 INFO:teuthology.orchestra.run.smithi163.stdout:check_obsoletes = 1 2024-05-27T09:24:08.160 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum clean all 2024-05-27T09:24:08.194 DEBUG:teuthology.orchestra.run.smithi114:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T09:24:08.270 INFO:teuthology.orchestra.run.smithi114.stdout:check_obsoletes = 1 2024-05-27T09:24:08.274 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum clean all 2024-05-27T09:24:08.387 INFO:teuthology.orchestra.run.smithi163.stdout:45 files removed 2024-05-27T09:24:08.414 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-27T09:24:08.594 INFO:teuthology.orchestra.run.smithi114.stdout:45 files removed 2024-05-27T09:24:08.632 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-27T09:24:09.098 INFO:teuthology.orchestra.run.smithi163.stdout:ceph packages for x86_64 198 kB/s | 84 kB 00:00 2024-05-27T09:24:09.448 INFO:teuthology.orchestra.run.smithi163.stdout:ceph noarch packages 38 kB/s | 12 kB 00:00 2024-05-27T09:24:09.522 INFO:teuthology.orchestra.run.smithi114.stdout:ceph packages for x86_64 183 kB/s | 84 kB 00:00 2024-05-27T09:24:09.799 INFO:teuthology.orchestra.run.smithi163.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-05-27T09:24:10.005 INFO:teuthology.orchestra.run.smithi114.stdout:ceph noarch packages 28 kB/s | 12 kB 00:00 2024-05-27T09:24:10.482 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-05-27T09:24:10.581 INFO:teuthology.orchestra.run.smithi114.stdout:ceph source packages 3.8 kB/s | 1.9 kB 00:00 2024-05-27T09:24:12.062 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.1 MB 00:01 2024-05-27T09:24:13.667 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - AppStream 9.0 MB/s | 19 MB 00:02 2024-05-27T09:24:15.954 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - AppStream 7.2 MB/s | 19 MB 00:02 2024-05-27T09:24:17.926 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - CRB 9.3 MB/s | 6.2 MB 00:00 2024-05-27T09:24:19.551 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 17 kB 00:00 2024-05-27T09:24:20.505 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - CRB 9.5 MB/s | 6.2 MB 00:00 2024-05-27T09:24:20.981 INFO:teuthology.orchestra.run.smithi163.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-05-27T09:24:22.056 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - Extras packages 90 kB/s | 17 kB 00:00 2024-05-27T09:24:23.983 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-27T09:24:26.228 INFO:teuthology.orchestra.run.smithi163.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-05-27T09:24:28.412 INFO:teuthology.orchestra.run.smithi163.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T09:24:28.413 INFO:teuthology.orchestra.run.smithi163.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T09:24:28.424 INFO:teuthology.orchestra.run.smithi163.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-27T09:24:28.425 INFO:teuthology.orchestra.run.smithi163.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-27T09:24:28.426 INFO:teuthology.orchestra.run.smithi163.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-27T09:24:28.427 INFO:teuthology.orchestra.run.smithi163.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T09:24:28.428 INFO:teuthology.orchestra.run.smithi163.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-27T09:24:28.430 INFO:teuthology.orchestra.run.smithi163.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-27T09:24:28.432 INFO:teuthology.orchestra.run.smithi163.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-27T09:24:28.494 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-05-27T09:24:28.517 INFO:teuthology.orchestra.run.smithi163.stdout:======================================================================================= 2024-05-27T09:24:28.517 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2024-05-27T09:24:28.517 INFO:teuthology.orchestra.run.smithi163.stdout:======================================================================================= 2024-05-27T09:24:28.517 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-05-27T09:24:28.517 INFO:teuthology.orchestra.run.smithi163.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-27T09:24:28.517 INFO:teuthology.orchestra.run.smithi163.stdout: ceph x86_64 2:19.0.0-3868.ga6507388.el9 ceph 6.6 k 2024-05-27T09:24:28.518 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base x86_64 2:19.0.0-3868.ga6507388.el9 ceph 5.4 M 2024-05-27T09:24:28.518 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse x86_64 2:19.0.0-3868.ga6507388.el9 ceph 863 k 2024-05-27T09:24:28.518 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3868.ga6507388.el9 ceph 147 k 2024-05-27T09:24:28.518 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr x86_64 2:19.0.0-3868.ga6507388.el9 ceph 1.5 M 2024-05-27T09:24:28.518 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 143 k 2024-05-27T09:24:28.518 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 3.5 M 2024-05-27T09:24:28.518 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 7.4 M 2024-05-27T09:24:28.518 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 49 k 2024-05-27T09:24:28.518 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw x86_64 2:19.0.0-3868.ga6507388.el9 ceph 10 M 2024-05-27T09:24:28.518 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test x86_64 2:19.0.0-3868.ga6507388.el9 ceph 43 M 2024-05-27T09:24:28.518 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-volume noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 273 k 2024-05-27T09:24:28.518 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 764 k 2024-05-27T09:24:28.518 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-mirror x86_64 2:19.0.0-3868.ga6507388.el9 ceph 224 k 2024-05-27T09:24:28.519 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-shell x86_64 2:19.0.0-3868.ga6507388.el9 ceph 27 k 2024-05-27T09:24:28.519 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-top noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 18 k 2024-05-27T09:24:28.519 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-27T09:24:28.519 INFO:teuthology.orchestra.run.smithi163.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-27T09:24:28.519 INFO:teuthology.orchestra.run.smithi163.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-27T09:24:28.519 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel x86_64 2:19.0.0-3868.ga6507388.el9 ceph 32 k 2024-05-27T09:24:28.519 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 756 k 2024-05-27T09:24:28.519 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel x86_64 2:19.0.0-3868.ga6507388.el9 ceph 127 k 2024-05-27T09:24:28.519 INFO:teuthology.orchestra.run.smithi163.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-27T09:24:28.519 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-27T09:24:28.519 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-27T09:24:28.519 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-27T09:24:28.520 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-27T09:24:28.520 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-27T09:24:28.520 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs x86_64 2:19.0.0-3868.ga6507388.el9 ceph 161 k 2024-05-27T09:24:28.520 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados x86_64 2:19.0.0-3868.ga6507388.el9 ceph 321 k 2024-05-27T09:24:28.520 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd x86_64 2:19.0.0-3868.ga6507388.el9 ceph 301 k 2024-05-27T09:24:28.520 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw x86_64 2:19.0.0-3868.ga6507388.el9 ceph 100 k 2024-05-27T09:24:28.520 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse x86_64 2:19.0.0-3868.ga6507388.el9 ceph 86 k 2024-05-27T09:24:28.520 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror x86_64 2:19.0.0-3868.ga6507388.el9 ceph 3.0 M 2024-05-27T09:24:28.520 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd x86_64 2:19.0.0-3868.ga6507388.el9 ceph 170 k 2024-05-27T09:24:28.520 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-27T09:24:28.520 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrading: 2024-05-27T09:24:28.520 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-27T09:24:28.521 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-27T09:24:28.521 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-27T09:24:28.521 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-27T09:24:28.521 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-27T09:24:28.521 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-27T09:24:28.521 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-27T09:24:28.521 INFO:teuthology.orchestra.run.smithi163.stdout: librados2 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 3.3 M 2024-05-27T09:24:28.521 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 3.1 M 2024-05-27T09:24:28.521 INFO:teuthology.orchestra.run.smithi163.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-27T09:24:28.521 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-27T09:24:28.521 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2024-05-27T09:24:28.521 INFO:teuthology.orchestra.run.smithi163.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-27T09:24:28.521 INFO:teuthology.orchestra.run.smithi163.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-27T09:24:28.522 INFO:teuthology.orchestra.run.smithi163.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T09:24:28.522 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common x86_64 2:19.0.0-3868.ga6507388.el9 ceph 20 M 2024-05-27T09:24:28.522 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 34 k 2024-05-27T09:24:28.522 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds x86_64 2:19.0.0-3868.ga6507388.el9 ceph 2.3 M 2024-05-27T09:24:28.522 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 274 k 2024-05-27T09:24:28.522 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon x86_64 2:19.0.0-3868.ga6507388.el9 ceph 4.9 M 2024-05-27T09:24:28.522 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd x86_64 2:19.0.0-3868.ga6507388.el9 ceph 18 M 2024-05-27T09:24:28.522 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 16 k 2024-05-27T09:24:28.522 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux x86_64 2:19.0.0-3868.ga6507388.el9 ceph 25 k 2024-05-27T09:24:28.522 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T09:24:28.522 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T09:24:28.522 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T09:24:28.523 INFO:teuthology.orchestra.run.smithi163.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T09:24:28.523 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T09:24:28.523 INFO:teuthology.orchestra.run.smithi163.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-27T09:24:28.523 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T09:24:28.523 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T09:24:28.523 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite x86_64 2:19.0.0-3868.ga6507388.el9 ceph 157 k 2024-05-27T09:24:28.523 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T09:24:28.523 INFO:teuthology.orchestra.run.smithi163.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T09:24:28.523 INFO:teuthology.orchestra.run.smithi163.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T09:24:28.523 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T09:24:28.523 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T09:24:28.523 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 497 k 2024-05-27T09:24:28.523 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T09:24:28.524 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 5.2 M 2024-05-27T09:24:28.524 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T09:24:28.524 INFO:teuthology.orchestra.run.smithi163.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-27T09:24:28.524 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T09:24:28.524 INFO:teuthology.orchestra.run.smithi163.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-27T09:24:28.524 INFO:teuthology.orchestra.run.smithi163.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-27T09:24:28.524 INFO:teuthology.orchestra.run.smithi163.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-27T09:24:28.524 INFO:teuthology.orchestra.run.smithi163.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T09:24:28.524 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T09:24:28.524 INFO:teuthology.orchestra.run.smithi163.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T09:24:28.524 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-27T09:24:28.524 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-27T09:24:28.524 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-27T09:24:28.525 INFO:teuthology.orchestra.run.smithi163.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T09:24:28.525 INFO:teuthology.orchestra.run.smithi163.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-27T09:24:28.525 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T09:24:28.525 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T09:24:28.525 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T09:24:28.525 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse x86_64 2:19.0.0-3868.ga6507388.el9 ceph 45 k 2024-05-27T09:24:28.525 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common x86_64 2:19.0.0-3868.ga6507388.el9 ceph 136 k 2024-05-27T09:24:28.525 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T09:24:28.525 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T09:24:28.525 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T09:24:28.525 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T09:24:28.525 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-27T09:24:28.525 INFO:teuthology.orchestra.run.smithi163.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-27T09:24:28.525 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T09:24:28.526 INFO:teuthology.orchestra.run.smithi163.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T09:24:28.526 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T09:24:28.526 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-27T09:24:28.526 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-27T09:24:28.526 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T09:24:28.526 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T09:24:28.526 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T09:24:28.526 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T09:24:28.526 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T09:24:28.526 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T09:24:28.526 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T09:24:28.526 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T09:24:28.527 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T09:24:28.527 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T09:24:28.527 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T09:24:28.527 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T09:24:28.527 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T09:24:28.527 INFO:teuthology.orchestra.run.smithi163.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T09:24:28.527 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T09:24:28.527 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T09:24:28.527 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T09:24:28.527 INFO:teuthology.orchestra.run.smithi163.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-27T09:24:28.527 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T09:24:28.527 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T09:24:28.528 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T09:24:28.528 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T09:24:28.528 INFO:teuthology.orchestra.run.smithi163.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-27T09:24:28.528 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T09:24:28.528 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T09:24:28.528 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T09:24:28.528 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T09:24:28.528 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-27T09:24:28.528 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T09:24:28.528 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T09:24:28.528 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T09:24:28.528 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T09:24:28.528 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T09:24:28.528 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T09:24:28.529 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T09:24:28.529 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T09:24:28.529 INFO:teuthology.orchestra.run.smithi163.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T09:24:28.529 INFO:teuthology.orchestra.run.smithi163.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T09:24:28.529 INFO:teuthology.orchestra.run.smithi163.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T09:24:28.529 INFO:teuthology.orchestra.run.smithi163.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-27T09:24:28.529 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T09:24:28.529 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T09:24:28.529 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T09:24:28.529 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T09:24:28.529 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-27T09:24:28.529 INFO:teuthology.orchestra.run.smithi163.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-27T09:24:28.529 INFO:teuthology.orchestra.run.smithi163.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T09:24:28.530 INFO:teuthology.orchestra.run.smithi163.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T09:24:28.530 INFO:teuthology.orchestra.run.smithi163.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T09:24:28.530 INFO:teuthology.orchestra.run.smithi163.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-27T09:24:28.530 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T09:24:28.530 INFO:teuthology.orchestra.run.smithi163.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-27T09:24:28.530 INFO:teuthology.orchestra.run.smithi163.stdout:Installing weak dependencies: 2024-05-27T09:24:28.530 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-27T09:24:28.530 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:24:28.530 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-05-27T09:24:28.530 INFO:teuthology.orchestra.run.smithi163.stdout:======================================================================================= 2024-05-27T09:24:28.530 INFO:teuthology.orchestra.run.smithi163.stdout:Install 148 Packages 2024-05-27T09:24:28.530 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrade 11 Packages 2024-05-27T09:24:28.530 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:24:28.531 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 219 M 2024-05-27T09:24:28.531 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-05-27T09:24:29.387 INFO:teuthology.orchestra.run.smithi163.stdout:(1/159): ceph-19.0.0-3868.ga6507388.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-05-27T09:24:29.912 INFO:teuthology.orchestra.run.smithi163.stdout:(2/159): ceph-base-19.0.0-3868.ga6507388.el9.x8 7.0 MB/s | 5.4 MB 00:00 2024-05-27T09:24:29.979 INFO:teuthology.orchestra.run.smithi163.stdout:(3/159): ceph-immutable-object-cache-19.0.0-386 2.2 MB/s | 147 kB 00:00 2024-05-27T09:24:30.158 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-05-27T09:24:30.221 INFO:teuthology.orchestra.run.smithi163.stdout:(4/159): ceph-mds-19.0.0-3868.ga6507388.el9.x86 9.4 MB/s | 2.3 MB 00:00 2024-05-27T09:24:30.255 INFO:teuthology.orchestra.run.smithi163.stdout:(5/159): ceph-fuse-19.0.0-3868.ga6507388.el9.x8 995 kB/s | 863 kB 00:00 2024-05-27T09:24:30.397 INFO:teuthology.orchestra.run.smithi163.stdout:(6/159): ceph-mgr-19.0.0-3868.ga6507388.el9.x86 8.6 MB/s | 1.5 MB 00:00 2024-05-27T09:24:30.622 INFO:teuthology.orchestra.run.smithi163.stdout:(7/159): ceph-common-19.0.0-3868.ga6507388.el9. 13 MB/s | 20 MB 00:01 2024-05-27T09:24:31.481 INFO:teuthology.orchestra.run.smithi163.stdout:(8/159): ceph-radosgw-19.0.0-3868.ga6507388.el9 12 MB/s | 10 MB 00:00 2024-05-27T09:24:31.531 INFO:teuthology.orchestra.run.smithi163.stdout:(9/159): ceph-selinux-19.0.0-3868.ga6507388.el9 506 kB/s | 25 kB 00:00 2024-05-27T09:24:31.790 INFO:teuthology.orchestra.run.smithi163.stdout:(10/159): ceph-osd-19.0.0-3868.ga6507388.el9.x8 13 MB/s | 18 MB 00:01 2024-05-27T09:24:31.840 INFO:teuthology.orchestra.run.smithi163.stdout:(11/159): cephfs-mirror-19.0.0-3868.ga6507388.e 4.4 MB/s | 224 kB 00:00 2024-05-27T09:24:31.915 INFO:teuthology.orchestra.run.smithi163.stdout:(12/159): cephfs-shell-19.0.0-3868.ga6507388.el 361 kB/s | 27 kB 00:00 2024-05-27T09:24:31.990 INFO:teuthology.orchestra.run.smithi163.stdout:(13/159): libcephfs-devel-19.0.0-3868.ga6507388 424 kB/s | 32 kB 00:00 2024-05-27T09:24:32.074 INFO:teuthology.orchestra.run.smithi163.stdout:(14/159): ceph-mon-19.0.0-3868.ga6507388.el9.x8 2.7 MB/s | 4.9 MB 00:01 2024-05-27T09:24:32.108 INFO:teuthology.orchestra.run.smithi163.stdout:(15/159): libcephfs2-19.0.0-3868.ga6507388.el9. 6.3 MB/s | 756 kB 00:00 2024-05-27T09:24:32.150 INFO:teuthology.orchestra.run.smithi163.stdout:(16/159): libcephsqlite-19.0.0-3868.ga6507388.e 2.0 MB/s | 157 kB 00:00 2024-05-27T09:24:32.183 INFO:teuthology.orchestra.run.smithi163.stdout:(17/159): librados-devel-19.0.0-3868.ga6507388. 1.6 MB/s | 127 kB 00:00 2024-05-27T09:24:32.325 INFO:teuthology.orchestra.run.smithi163.stdout:(18/159): libradosstriper1-19.0.0-3868.ga650738 2.8 MB/s | 497 kB 00:00 2024-05-27T09:24:32.442 INFO:teuthology.orchestra.run.smithi163.stdout:(19/159): python3-ceph-argparse-19.0.0-3868.ga6 388 kB/s | 45 kB 00:00 2024-05-27T09:24:32.559 INFO:teuthology.orchestra.run.smithi163.stdout:(20/159): python3-ceph-common-19.0.0-3868.ga650 1.1 MB/s | 136 kB 00:00 2024-05-27T09:24:32.563 INFO:teuthology.orchestra.run.smithi114.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T09:24:32.564 INFO:teuthology.orchestra.run.smithi114.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T09:24:32.583 INFO:teuthology.orchestra.run.smithi114.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-27T09:24:32.584 INFO:teuthology.orchestra.run.smithi114.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-27T09:24:32.585 INFO:teuthology.orchestra.run.smithi114.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-27T09:24:32.586 INFO:teuthology.orchestra.run.smithi114.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T09:24:32.589 INFO:teuthology.orchestra.run.smithi114.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-27T09:24:32.592 INFO:teuthology.orchestra.run.smithi114.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-27T09:24:32.593 INFO:teuthology.orchestra.run.smithi114.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-27T09:24:32.634 INFO:teuthology.orchestra.run.smithi163.stdout:(21/159): librgw2-19.0.0-3868.ga6507388.el9.x86 11 MB/s | 5.2 MB 00:00 2024-05-27T09:24:32.660 INFO:teuthology.orchestra.run.smithi163.stdout:(22/159): python3-cephfs-19.0.0-3868.ga6507388. 1.6 MB/s | 161 kB 00:00 2024-05-27T09:24:32.663 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-05-27T09:24:32.681 INFO:teuthology.orchestra.run.smithi114.stdout:======================================================================================= 2024-05-27T09:24:32.681 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-05-27T09:24:32.682 INFO:teuthology.orchestra.run.smithi114.stdout:======================================================================================= 2024-05-27T09:24:32.682 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-05-27T09:24:32.682 INFO:teuthology.orchestra.run.smithi114.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-27T09:24:32.682 INFO:teuthology.orchestra.run.smithi114.stdout: ceph x86_64 2:19.0.0-3868.ga6507388.el9 ceph 6.6 k 2024-05-27T09:24:32.682 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base x86_64 2:19.0.0-3868.ga6507388.el9 ceph 5.4 M 2024-05-27T09:24:32.682 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse x86_64 2:19.0.0-3868.ga6507388.el9 ceph 863 k 2024-05-27T09:24:32.682 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3868.ga6507388.el9 ceph 147 k 2024-05-27T09:24:32.682 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr x86_64 2:19.0.0-3868.ga6507388.el9 ceph 1.5 M 2024-05-27T09:24:32.682 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 143 k 2024-05-27T09:24:32.682 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 3.5 M 2024-05-27T09:24:32.683 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 7.4 M 2024-05-27T09:24:32.683 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 49 k 2024-05-27T09:24:32.683 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw x86_64 2:19.0.0-3868.ga6507388.el9 ceph 10 M 2024-05-27T09:24:32.683 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test x86_64 2:19.0.0-3868.ga6507388.el9 ceph 43 M 2024-05-27T09:24:32.683 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-volume noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 273 k 2024-05-27T09:24:32.683 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 764 k 2024-05-27T09:24:32.683 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-mirror x86_64 2:19.0.0-3868.ga6507388.el9 ceph 224 k 2024-05-27T09:24:32.683 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-shell x86_64 2:19.0.0-3868.ga6507388.el9 ceph 27 k 2024-05-27T09:24:32.683 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-top noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 18 k 2024-05-27T09:24:32.683 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-27T09:24:32.683 INFO:teuthology.orchestra.run.smithi114.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-27T09:24:32.683 INFO:teuthology.orchestra.run.smithi114.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-27T09:24:32.683 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel x86_64 2:19.0.0-3868.ga6507388.el9 ceph 32 k 2024-05-27T09:24:32.684 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 756 k 2024-05-27T09:24:32.684 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel x86_64 2:19.0.0-3868.ga6507388.el9 ceph 127 k 2024-05-27T09:24:32.684 INFO:teuthology.orchestra.run.smithi114.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-27T09:24:32.684 INFO:teuthology.orchestra.run.smithi114.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-27T09:24:32.684 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-27T09:24:32.684 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-27T09:24:32.684 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-27T09:24:32.684 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-27T09:24:32.684 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs x86_64 2:19.0.0-3868.ga6507388.el9 ceph 161 k 2024-05-27T09:24:32.684 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados x86_64 2:19.0.0-3868.ga6507388.el9 ceph 321 k 2024-05-27T09:24:32.684 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd x86_64 2:19.0.0-3868.ga6507388.el9 ceph 301 k 2024-05-27T09:24:32.684 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw x86_64 2:19.0.0-3868.ga6507388.el9 ceph 100 k 2024-05-27T09:24:32.684 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse x86_64 2:19.0.0-3868.ga6507388.el9 ceph 86 k 2024-05-27T09:24:32.685 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror x86_64 2:19.0.0-3868.ga6507388.el9 ceph 3.0 M 2024-05-27T09:24:32.685 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd x86_64 2:19.0.0-3868.ga6507388.el9 ceph 170 k 2024-05-27T09:24:32.685 INFO:teuthology.orchestra.run.smithi114.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-27T09:24:32.685 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrading: 2024-05-27T09:24:32.685 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-27T09:24:32.685 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-27T09:24:32.685 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-27T09:24:32.685 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-27T09:24:32.685 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-27T09:24:32.685 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-27T09:24:32.685 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-27T09:24:32.685 INFO:teuthology.orchestra.run.smithi114.stdout: librados2 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 3.3 M 2024-05-27T09:24:32.685 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 3.1 M 2024-05-27T09:24:32.686 INFO:teuthology.orchestra.run.smithi114.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-27T09:24:32.686 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-27T09:24:32.686 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-05-27T09:24:32.686 INFO:teuthology.orchestra.run.smithi114.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-27T09:24:32.686 INFO:teuthology.orchestra.run.smithi114.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-27T09:24:32.686 INFO:teuthology.orchestra.run.smithi114.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T09:24:32.686 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common x86_64 2:19.0.0-3868.ga6507388.el9 ceph 20 M 2024-05-27T09:24:32.686 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 34 k 2024-05-27T09:24:32.686 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds x86_64 2:19.0.0-3868.ga6507388.el9 ceph 2.3 M 2024-05-27T09:24:32.686 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 274 k 2024-05-27T09:24:32.686 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon x86_64 2:19.0.0-3868.ga6507388.el9 ceph 4.9 M 2024-05-27T09:24:32.686 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd x86_64 2:19.0.0-3868.ga6507388.el9 ceph 18 M 2024-05-27T09:24:32.687 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3868.ga6507388.el9 ceph-noarch 16 k 2024-05-27T09:24:32.687 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux x86_64 2:19.0.0-3868.ga6507388.el9 ceph 25 k 2024-05-27T09:24:32.687 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T09:24:32.687 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T09:24:32.687 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T09:24:32.687 INFO:teuthology.orchestra.run.smithi114.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T09:24:32.687 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T09:24:32.687 INFO:teuthology.orchestra.run.smithi114.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-27T09:24:32.687 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T09:24:32.687 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T09:24:32.687 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite x86_64 2:19.0.0-3868.ga6507388.el9 ceph 157 k 2024-05-27T09:24:32.687 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T09:24:32.687 INFO:teuthology.orchestra.run.smithi114.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T09:24:32.688 INFO:teuthology.orchestra.run.smithi114.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T09:24:32.688 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T09:24:32.688 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T09:24:32.688 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 497 k 2024-05-27T09:24:32.688 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T09:24:32.688 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2 x86_64 2:19.0.0-3868.ga6507388.el9 ceph 5.2 M 2024-05-27T09:24:32.688 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T09:24:32.688 INFO:teuthology.orchestra.run.smithi114.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-27T09:24:32.688 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T09:24:32.688 INFO:teuthology.orchestra.run.smithi114.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-27T09:24:32.688 INFO:teuthology.orchestra.run.smithi114.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-27T09:24:32.688 INFO:teuthology.orchestra.run.smithi114.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-27T09:24:32.688 INFO:teuthology.orchestra.run.smithi114.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T09:24:32.688 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T09:24:32.689 INFO:teuthology.orchestra.run.smithi114.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T09:24:32.689 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-27T09:24:32.689 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-27T09:24:32.689 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-27T09:24:32.689 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T09:24:32.689 INFO:teuthology.orchestra.run.smithi114.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-27T09:24:32.689 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T09:24:32.689 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T09:24:32.689 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T09:24:32.689 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse x86_64 2:19.0.0-3868.ga6507388.el9 ceph 45 k 2024-05-27T09:24:32.689 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common x86_64 2:19.0.0-3868.ga6507388.el9 ceph 136 k 2024-05-27T09:24:32.689 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T09:24:32.690 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T09:24:32.690 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T09:24:32.690 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T09:24:32.690 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-27T09:24:32.690 INFO:teuthology.orchestra.run.smithi114.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-27T09:24:32.690 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T09:24:32.690 INFO:teuthology.orchestra.run.smithi114.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T09:24:32.690 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T09:24:32.690 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-27T09:24:32.690 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-27T09:24:32.690 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T09:24:32.690 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T09:24:32.690 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T09:24:32.691 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T09:24:32.691 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T09:24:32.691 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T09:24:32.691 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T09:24:32.691 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T09:24:32.691 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T09:24:32.691 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T09:24:32.691 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T09:24:32.691 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T09:24:32.691 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T09:24:32.691 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T09:24:32.691 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T09:24:32.691 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T09:24:32.692 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T09:24:32.692 INFO:teuthology.orchestra.run.smithi114.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-27T09:24:32.692 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T09:24:32.692 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T09:24:32.692 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T09:24:32.692 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T09:24:32.692 INFO:teuthology.orchestra.run.smithi114.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-27T09:24:32.692 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T09:24:32.692 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T09:24:32.692 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T09:24:32.692 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T09:24:32.692 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-27T09:24:32.692 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T09:24:32.693 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T09:24:32.693 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T09:24:32.693 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T09:24:32.693 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T09:24:32.693 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T09:24:32.693 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T09:24:32.693 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T09:24:32.693 INFO:teuthology.orchestra.run.smithi114.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T09:24:32.693 INFO:teuthology.orchestra.run.smithi114.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T09:24:32.693 INFO:teuthology.orchestra.run.smithi114.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T09:24:32.693 INFO:teuthology.orchestra.run.smithi114.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-27T09:24:32.693 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T09:24:32.693 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T09:24:32.694 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T09:24:32.694 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T09:24:32.694 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-27T09:24:32.694 INFO:teuthology.orchestra.run.smithi114.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-27T09:24:32.694 INFO:teuthology.orchestra.run.smithi114.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T09:24:32.694 INFO:teuthology.orchestra.run.smithi114.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T09:24:32.694 INFO:teuthology.orchestra.run.smithi114.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T09:24:32.694 INFO:teuthology.orchestra.run.smithi114.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-27T09:24:32.694 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T09:24:32.694 INFO:teuthology.orchestra.run.smithi114.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-27T09:24:32.694 INFO:teuthology.orchestra.run.smithi114.stdout:Installing weak dependencies: 2024-05-27T09:24:32.694 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-27T09:24:32.694 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:24:32.695 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-05-27T09:24:32.695 INFO:teuthology.orchestra.run.smithi114.stdout:======================================================================================= 2024-05-27T09:24:32.695 INFO:teuthology.orchestra.run.smithi114.stdout:Install 148 Packages 2024-05-27T09:24:32.695 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrade 11 Packages 2024-05-27T09:24:32.695 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:24:32.695 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 219 M 2024-05-27T09:24:32.696 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-05-27T09:24:32.710 INFO:teuthology.orchestra.run.smithi163.stdout:(23/159): python3-rados-19.0.0-3868.ga6507388.e 4.2 MB/s | 321 kB 00:00 2024-05-27T09:24:32.752 INFO:teuthology.orchestra.run.smithi163.stdout:(24/159): python3-rbd-19.0.0-3868.ga6507388.el9 3.2 MB/s | 301 kB 00:00 2024-05-27T09:24:32.777 INFO:teuthology.orchestra.run.smithi163.stdout:(25/159): python3-rgw-19.0.0-3868.ga6507388.el9 1.5 MB/s | 100 kB 00:00 2024-05-27T09:24:32.836 INFO:teuthology.orchestra.run.smithi163.stdout:(26/159): rbd-fuse-19.0.0-3868.ga6507388.el9.x8 1.0 MB/s | 86 kB 00:00 2024-05-27T09:24:32.944 INFO:teuthology.orchestra.run.smithi163.stdout:(27/159): rbd-nbd-19.0.0-3868.ga6507388.el9.x86 1.5 MB/s | 170 kB 00:00 2024-05-27T09:24:33.061 INFO:teuthology.orchestra.run.smithi163.stdout:(28/159): rbd-mirror-19.0.0-3868.ga6507388.el9. 11 MB/s | 3.0 MB 00:00 2024-05-27T09:24:33.087 INFO:teuthology.orchestra.run.smithi163.stdout:(29/159): ceph-grafana-dashboards-19.0.0-3868.g 241 kB/s | 34 kB 00:00 2024-05-27T09:24:33.112 INFO:teuthology.orchestra.run.smithi163.stdout:(30/159): ceph-mgr-cephadm-19.0.0-3868.ga650738 2.8 MB/s | 143 kB 00:00 2024-05-27T09:24:33.787 INFO:teuthology.orchestra.run.smithi163.stdout:(31/159): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-05-27T09:24:33.851 INFO:teuthology.orchestra.run.smithi114.stdout:(1/159): ceph-19.0.0-3868.ga6507388.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-05-27T09:24:33.854 INFO:teuthology.orchestra.run.smithi163.stdout:(32/159): ceph-mgr-dashboard-19.0.0-3868.ga6507 4.6 MB/s | 3.5 MB 00:00 2024-05-27T09:24:33.879 INFO:teuthology.orchestra.run.smithi163.stdout:(33/159): ceph-mgr-modules-core-19.0.0-3868.ga6 2.9 MB/s | 274 kB 00:00 2024-05-27T09:24:33.930 INFO:teuthology.orchestra.run.smithi163.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3868.ga 327 kB/s | 16 kB 00:00 2024-05-27T09:24:33.955 INFO:teuthology.orchestra.run.smithi163.stdout:(35/159): ceph-mgr-rook-19.0.0-3868.ga6507388.e 490 kB/s | 49 kB 00:00 2024-05-27T09:24:33.997 INFO:teuthology.orchestra.run.smithi163.stdout:(36/159): ceph-volume-19.0.0-3868.ga6507388.el9 4.0 MB/s | 273 kB 00:00 2024-05-27T09:24:34.097 INFO:teuthology.orchestra.run.smithi163.stdout:(37/159): cephfs-top-19.0.0-3868.ga6507388.el9. 182 kB/s | 18 kB 00:00 2024-05-27T09:24:34.124 INFO:teuthology.orchestra.run.smithi163.stdout:(38/159): cephadm-19.0.0-3868.ga6507388.el9.noa 4.5 MB/s | 764 kB 00:00 2024-05-27T09:24:34.426 INFO:teuthology.orchestra.run.smithi114.stdout:(2/159): ceph-fuse-19.0.0-3868.ga6507388.el9.x8 1.5 MB/s | 863 kB 00:00 2024-05-27T09:24:34.490 INFO:teuthology.orchestra.run.smithi163.stdout:(39/159): ceph-test-19.0.0-3868.ga6507388.el9.x 15 MB/s | 43 MB 00:02 2024-05-27T09:24:34.515 INFO:teuthology.orchestra.run.smithi163.stdout:(40/159): fuse-2.9.9-15.el9.x86_64.rpm 192 kB/s | 80 kB 00:00 2024-05-27T09:24:34.535 INFO:teuthology.orchestra.run.smithi114.stdout:(3/159): ceph-immutable-object-cache-19.0.0-386 1.3 MB/s | 147 kB 00:00 2024-05-27T09:24:34.598 INFO:teuthology.orchestra.run.smithi163.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-05-27T09:24:34.632 INFO:teuthology.orchestra.run.smithi163.stdout:(42/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-05-27T09:24:34.666 INFO:teuthology.orchestra.run.smithi163.stdout:(43/159): libgfortran-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 803 kB 00:00 2024-05-27T09:24:34.691 INFO:teuthology.orchestra.run.smithi163.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-05-27T09:24:34.725 INFO:teuthology.orchestra.run.smithi163.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-27T09:24:34.750 INFO:teuthology.orchestra.run.smithi163.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-27T09:24:34.775 INFO:teuthology.orchestra.run.smithi163.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-27T09:24:34.817 INFO:teuthology.orchestra.run.smithi163.stdout:(48/159): python3-cryptography-36.0.1-4.el9.x86 5.6 MB/s | 1.2 MB 00:00 2024-05-27T09:24:34.851 INFO:teuthology.orchestra.run.smithi163.stdout:(49/159): automake-1.16.2-8.el9.noarch.rpm 6.7 MB/s | 689 kB 00:00 2024-05-27T09:24:34.876 INFO:teuthology.orchestra.run.smithi163.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-05-27T09:24:34.901 INFO:teuthology.orchestra.run.smithi163.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-05-27T09:24:34.926 INFO:teuthology.orchestra.run.smithi163.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-27T09:24:34.952 INFO:teuthology.orchestra.run.smithi163.stdout:(53/159): bison-3.7.4-5.el9.x86_64.rpm 5.2 MB/s | 943 kB 00:00 2024-05-27T09:24:34.969 INFO:teuthology.orchestra.run.smithi163.stdout:(54/159): flexiblas-3.0.4-8.el9.x86_64.rpm 487 kB/s | 33 kB 00:00 2024-05-27T09:24:35.002 INFO:teuthology.orchestra.run.smithi163.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 338 kB/s | 11 kB 00:00 2024-05-27T09:24:35.019 INFO:teuthology.orchestra.run.smithi163.stdout:(56/159): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-05-27T09:24:35.069 INFO:teuthology.orchestra.run.smithi163.stdout:(57/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-05-27T09:24:35.086 INFO:teuthology.orchestra.run.smithi163.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 676 kB/s | 45 kB 00:00 2024-05-27T09:24:35.112 INFO:teuthology.orchestra.run.smithi163.stdout:(59/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-05-27T09:24:35.146 INFO:teuthology.orchestra.run.smithi163.stdout:(60/159): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-05-27T09:24:35.170 INFO:teuthology.orchestra.run.smithi163.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 436 kB/s | 25 kB 00:00 2024-05-27T09:24:35.202 INFO:teuthology.orchestra.run.smithi114.stdout:(4/159): ceph-base-19.0.0-3868.ga6507388.el9.x8 3.1 MB/s | 5.4 MB 00:01 2024-05-27T09:24:35.204 INFO:teuthology.orchestra.run.smithi163.stdout:(62/159): libtool-2.4.6-45.el9.x86_64.rpm 4.8 MB/s | 582 kB 00:00 2024-05-27T09:24:35.230 INFO:teuthology.orchestra.run.smithi163.stdout:(63/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 565 kB/s | 47 kB 00:00 2024-05-27T09:24:35.255 INFO:teuthology.orchestra.run.smithi163.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-27T09:24:35.269 INFO:teuthology.orchestra.run.smithi114.stdout:(5/159): ceph-mds-19.0.0-3868.ga6507388.el9.x86 3.1 MB/s | 2.3 MB 00:00 2024-05-27T09:24:35.280 INFO:teuthology.orchestra.run.smithi163.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 700 kB/s | 35 kB 00:00 2024-05-27T09:24:35.305 INFO:teuthology.orchestra.run.smithi163.stdout:(66/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-05-27T09:24:35.364 INFO:teuthology.orchestra.run.smithi163.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 27 MB/s | 1.6 MB 00:00 2024-05-27T09:24:35.431 INFO:teuthology.orchestra.run.smithi163.stdout:(68/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 29 MB/s | 4.4 MB 00:00 2024-05-27T09:24:35.465 INFO:teuthology.orchestra.run.smithi163.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-05-27T09:24:35.477 INFO:teuthology.orchestra.run.smithi114.stdout:(6/159): ceph-mgr-19.0.0-3868.ga6507388.el9.x86 5.5 MB/s | 1.5 MB 00:00 2024-05-27T09:24:35.490 INFO:teuthology.orchestra.run.smithi163.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-27T09:24:35.557 INFO:teuthology.orchestra.run.smithi163.stdout:(71/159): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-05-27T09:24:35.599 INFO:teuthology.orchestra.run.smithi163.stdout:(72/159): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-05-27T09:24:35.624 INFO:teuthology.orchestra.run.smithi163.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-27T09:24:35.666 INFO:teuthology.orchestra.run.smithi163.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-05-27T09:24:35.750 INFO:teuthology.orchestra.run.smithi163.stdout:(75/159): postgresql-server-13.14-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-05-27T09:24:35.775 INFO:teuthology.orchestra.run.smithi163.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-05-27T09:24:35.800 INFO:teuthology.orchestra.run.smithi163.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-05-27T09:24:35.817 INFO:teuthology.orchestra.run.smithi163.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 610 kB/s | 26 kB 00:00 2024-05-27T09:24:35.834 INFO:teuthology.orchestra.run.smithi163.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 571 kB/s | 19 kB 00:00 2024-05-27T09:24:35.859 INFO:teuthology.orchestra.run.smithi163.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-05-27T09:24:35.876 INFO:teuthology.orchestra.run.smithi163.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 830 kB/s | 35 kB 00:00 2024-05-27T09:24:35.960 INFO:teuthology.orchestra.run.smithi163.stdout:(82/159): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-05-27T09:24:35.985 INFO:teuthology.orchestra.run.smithi163.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-05-27T09:24:36.010 INFO:teuthology.orchestra.run.smithi163.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-05-27T09:24:36.036 INFO:teuthology.orchestra.run.smithi163.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-27T09:24:36.061 INFO:teuthology.orchestra.run.smithi163.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-05-27T09:24:36.095 INFO:teuthology.orchestra.run.smithi163.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-05-27T09:24:36.120 INFO:teuthology.orchestra.run.smithi163.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-05-27T09:24:36.145 INFO:teuthology.orchestra.run.smithi163.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-27T09:24:36.171 INFO:teuthology.orchestra.run.smithi163.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-27T09:24:36.204 INFO:teuthology.orchestra.run.smithi163.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 919 kB/s | 54 kB 00:00 2024-05-27T09:24:36.210 INFO:teuthology.orchestra.run.smithi114.stdout:(7/159): ceph-mon-19.0.0-3868.ga6507388.el9.x86 5.2 MB/s | 4.9 MB 00:00 2024-05-27T09:24:36.280 INFO:teuthology.orchestra.run.smithi163.stdout:(92/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-05-27T09:24:36.305 INFO:teuthology.orchestra.run.smithi163.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 414 kB/s | 42 kB 00:00 2024-05-27T09:24:36.330 INFO:teuthology.orchestra.run.smithi163.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 880 kB/s | 44 kB 00:00 2024-05-27T09:24:36.355 INFO:teuthology.orchestra.run.smithi163.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-27T09:24:36.381 INFO:teuthology.orchestra.run.smithi163.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-05-27T09:24:36.406 INFO:teuthology.orchestra.run.smithi163.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-05-27T09:24:36.431 INFO:teuthology.orchestra.run.smithi163.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-05-27T09:24:36.448 INFO:teuthology.orchestra.run.smithi163.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-05-27T09:24:36.478 INFO:teuthology.orchestra.run.smithi114.stdout:(8/159): ceph-common-19.0.0-3868.ga6507388.el9. 6.5 MB/s | 20 MB 00:03 2024-05-27T09:24:36.499 INFO:teuthology.orchestra.run.smithi163.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-05-27T09:24:36.524 INFO:teuthology.orchestra.run.smithi163.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 845 kB/s | 64 kB 00:00 2024-05-27T09:24:36.549 INFO:teuthology.orchestra.run.smithi163.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-05-27T09:24:36.553 INFO:teuthology.orchestra.run.smithi114.stdout:(9/159): ceph-selinux-19.0.0-3868.ga6507388.el9 341 kB/s | 25 kB 00:00 2024-05-27T09:24:36.791 INFO:teuthology.orchestra.run.smithi163.stdout:(103/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-05-27T09:24:37.233 INFO:teuthology.orchestra.run.smithi163.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-05-27T09:24:37.317 INFO:teuthology.orchestra.run.smithi163.stdout:(105/159): lua-devel-5.4.4-4.el9.x86_64.rpm 28 kB/s | 22 kB 00:00 2024-05-27T09:24:37.350 INFO:teuthology.orchestra.run.smithi163.stdout:(106/159): gperftools-libs-2.9.1-2.el9.x86_64.r 2.6 MB/s | 309 kB 00:00 2024-05-27T09:24:37.403 INFO:teuthology.orchestra.run.smithi114.stdout:(10/159): ceph-osd-19.0.0-3868.ga6507388.el9.x8 9.2 MB/s | 18 MB 00:01 2024-05-27T09:24:37.495 INFO:teuthology.orchestra.run.smithi114.stdout:(11/159): cephfs-mirror-19.0.0-3868.ga6507388.e 2.4 MB/s | 224 kB 00:00 2024-05-27T09:24:37.550 INFO:teuthology.orchestra.run.smithi163.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 83 kB/s | 19 kB 00:00 2024-05-27T09:24:37.617 INFO:teuthology.orchestra.run.smithi163.stdout:(108/159): libarrow-9.0.0-11.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-05-27T09:24:37.634 INFO:teuthology.orchestra.run.smithi163.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 311 kB/s | 26 kB 00:00 2024-05-27T09:24:37.660 INFO:teuthology.orchestra.run.smithi163.stdout:(110/159): protobuf-compiler-3.14.0-13.el9.x86_ 778 kB/s | 863 kB 00:01 2024-05-27T09:24:37.677 INFO:teuthology.orchestra.run.smithi163.stdout:(111/159): liboath-2.6.7-2.el9.x86_64.rpm 825 kB/s | 49 kB 00:00 2024-05-27T09:24:37.687 INFO:teuthology.orchestra.run.smithi114.stdout:(12/159): ceph-radosgw-19.0.0-3868.ga6507388.el 6.9 MB/s | 10 MB 00:01 2024-05-27T09:24:37.712 INFO:teuthology.orchestra.run.smithi114.stdout:(13/159): cephfs-shell-19.0.0-3868.ga6507388.el 125 kB/s | 27 kB 00:00 2024-05-27T09:24:37.763 INFO:teuthology.orchestra.run.smithi114.stdout:(14/159): libcephfs-devel-19.0.0-3868.ga6507388 428 kB/s | 32 kB 00:00 2024-05-27T09:24:37.768 INFO:teuthology.orchestra.run.smithi163.stdout:(112/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-05-27T09:24:37.797 INFO:teuthology.orchestra.run.smithi163.stdout:(113/159): libunwind-1.6.2-1.el9.x86_64.rpm 426 kB/s | 67 kB 00:00 2024-05-27T09:24:37.844 INFO:teuthology.orchestra.run.smithi163.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-05-27T09:24:37.846 INFO:teuthology.orchestra.run.smithi114.stdout:(15/159): libcephfs2-19.0.0-3868.ga6507388.el9. 5.6 MB/s | 756 kB 00:00 2024-05-27T09:24:37.886 INFO:teuthology.orchestra.run.smithi163.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 477 kB/s | 43 kB 00:00 2024-05-27T09:24:37.897 INFO:teuthology.orchestra.run.smithi114.stdout:(16/159): libcephsqlite-19.0.0-3868.ga6507388.e 1.2 MB/s | 157 kB 00:00 2024-05-27T09:24:37.911 INFO:teuthology.orchestra.run.smithi163.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 479 kB/s | 32 kB 00:00 2024-05-27T09:24:37.922 INFO:teuthology.orchestra.run.smithi114.stdout:(17/159): librados-devel-19.0.0-3868.ga6507388. 1.7 MB/s | 127 kB 00:00 2024-05-27T09:24:37.961 INFO:teuthology.orchestra.run.smithi163.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-05-27T09:24:37.987 INFO:teuthology.orchestra.run.smithi163.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-05-27T09:24:37.997 INFO:teuthology.orchestra.run.smithi114.stdout:(18/159): libradosstriper1-19.0.0-3868.ga650738 4.9 MB/s | 497 kB 00:00 2024-05-27T09:24:38.054 INFO:teuthology.orchestra.run.smithi163.stdout:(119/159): luarocks-3.9.2-5.el9.noarch.rpm 384 kB/s | 151 kB 00:00 2024-05-27T09:24:38.079 INFO:teuthology.orchestra.run.smithi163.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-05-27T09:24:38.164 INFO:teuthology.orchestra.run.smithi114.stdout:(19/159): python3-ceph-argparse-19.0.0-3868.ga6 272 kB/s | 45 kB 00:00 2024-05-27T09:24:38.196 INFO:teuthology.orchestra.run.smithi163.stdout:(121/159): python3-grpcio-1.46.7-10.el9.x86_64. 17 MB/s | 2.0 MB 00:00 2024-05-27T09:24:38.221 INFO:teuthology.orchestra.run.smithi163.stdout:(122/159): python3-cherrypy-18.6.1-2.el9.noarch 1.3 MB/s | 358 kB 00:00 2024-05-27T09:24:38.247 INFO:teuthology.orchestra.run.smithi163.stdout:(123/159): python3-google-auth-2.29.0-1.el9.noa 1.0 MB/s | 206 kB 00:00 2024-05-27T09:24:38.256 INFO:teuthology.orchestra.run.smithi114.stdout:(20/159): python3-ceph-common-19.0.0-3868.ga650 1.5 MB/s | 136 kB 00:00 2024-05-27T09:24:38.272 INFO:teuthology.orchestra.run.smithi163.stdout:(124/159): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-05-27T09:24:38.297 INFO:teuthology.orchestra.run.smithi163.stdout:(125/159): python3-jaraco-8.2.1-3.el9.noarch.rp 141 kB/s | 11 kB 00:00 2024-05-27T09:24:38.322 INFO:teuthology.orchestra.run.smithi163.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-05-27T09:24:38.348 INFO:teuthology.orchestra.run.smithi163.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-05-27T09:24:38.373 INFO:teuthology.orchestra.run.smithi163.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-05-27T09:24:38.398 INFO:teuthology.orchestra.run.smithi163.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-05-27T09:24:38.406 INFO:teuthology.orchestra.run.smithi114.stdout:(21/159): librgw2-19.0.0-3868.ga6507388.el9.x86 11 MB/s | 5.2 MB 00:00 2024-05-27T09:24:38.432 INFO:teuthology.orchestra.run.smithi163.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-05-27T09:24:38.440 INFO:teuthology.orchestra.run.smithi114.stdout:(22/159): python3-cephfs-19.0.0-3868.ga6507388. 885 kB/s | 161 kB 00:00 2024-05-27T09:24:38.457 INFO:teuthology.orchestra.run.smithi163.stdout:(131/159): python3-logutils-0.3.5-21.el9.noarch 552 kB/s | 46 kB 00:00 2024-05-27T09:24:38.483 INFO:teuthology.orchestra.run.smithi163.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 940 kB/s | 79 kB 00:00 2024-05-27T09:24:38.499 INFO:teuthology.orchestra.run.smithi114.stdout:(23/159): python3-rados-19.0.0-3868.ga6507388.e 3.4 MB/s | 321 kB 00:00 2024-05-27T09:24:38.508 INFO:teuthology.orchestra.run.smithi163.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 766 kB/s | 58 kB 00:00 2024-05-27T09:24:38.532 INFO:teuthology.orchestra.run.smithi114.stdout:(24/159): python3-rbd-19.0.0-3868.ga6507388.el9 3.2 MB/s | 301 kB 00:00 2024-05-27T09:24:38.550 INFO:teuthology.orchestra.run.smithi163.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-05-27T09:24:38.575 INFO:teuthology.orchestra.run.smithi163.stdout:(135/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-05-27T09:24:38.587 INFO:teuthology.orchestra.run.smithi114.stdout:(25/159): python3-rgw-19.0.0-3868.ga6507388.el9 1.1 MB/s | 100 kB 00:00 2024-05-27T09:24:38.600 INFO:teuthology.orchestra.run.smithi163.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 977 kB/s | 90 kB 00:00 2024-05-27T09:24:38.608 INFO:teuthology.orchestra.run.smithi114.stdout:(26/159): rbd-fuse-19.0.0-3868.ga6507388.el9.x8 1.1 MB/s | 86 kB 00:00 2024-05-27T09:24:38.626 INFO:teuthology.orchestra.run.smithi163.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-05-27T09:24:38.651 INFO:teuthology.orchestra.run.smithi163.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-05-27T09:24:38.676 INFO:teuthology.orchestra.run.smithi163.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-27T09:24:38.700 INFO:teuthology.orchestra.run.smithi114.stdout:(27/159): rbd-nbd-19.0.0-3868.ga6507388.el9.x86 1.8 MB/s | 170 kB 00:00 2024-05-27T09:24:38.701 INFO:teuthology.orchestra.run.smithi163.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 783 kB/s | 59 kB 00:00 2024-05-27T09:24:38.727 INFO:teuthology.orchestra.run.smithi163.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-05-27T09:24:38.752 INFO:teuthology.orchestra.run.smithi163.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-05-27T09:24:38.794 INFO:teuthology.orchestra.run.smithi163.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-05-27T09:24:38.800 INFO:teuthology.orchestra.run.smithi114.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3868.g 345 kB/s | 34 kB 00:00 2024-05-27T09:24:38.819 INFO:teuthology.orchestra.run.smithi163.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 972 kB/s | 90 kB 00:00 2024-05-27T09:24:38.853 INFO:teuthology.orchestra.run.smithi163.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-05-27T09:24:38.878 INFO:teuthology.orchestra.run.smithi163.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-05-27T09:24:38.912 INFO:teuthology.orchestra.run.smithi163.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-05-27T09:24:38.926 INFO:teuthology.orchestra.run.smithi114.stdout:(29/159): rbd-mirror-19.0.0-3868.ga6507388.el9. 9.0 MB/s | 3.0 MB 00:00 2024-05-27T09:24:38.959 INFO:teuthology.orchestra.run.smithi114.stdout:(30/159): ceph-mgr-cephadm-19.0.0-3868.ga650738 907 kB/s | 143 kB 00:00 2024-05-27T09:24:38.987 INFO:teuthology.orchestra.run.smithi163.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-05-27T09:24:39.146 INFO:teuthology.orchestra.run.smithi163.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 15 MB/s | 2.3 MB 00:00 2024-05-27T09:24:39.213 INFO:teuthology.orchestra.run.smithi163.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-05-27T09:24:39.246 INFO:teuthology.orchestra.run.smithi163.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 1.2 MB/s | 41 kB 00:00 2024-05-27T09:24:39.288 INFO:teuthology.orchestra.run.smithi163.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 5.8 MB/s | 248 kB 00:00 2024-05-27T09:24:39.310 INFO:teuthology.orchestra.run.smithi114.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3868.ga6507 9.3 MB/s | 3.5 MB 00:00 2024-05-27T09:24:39.322 INFO:teuthology.orchestra.run.smithi163.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 1.1 MB/s | 37 kB 00:00 2024-05-27T09:24:39.364 INFO:teuthology.orchestra.run.smithi163.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.9 MB/s | 207 kB 00:00 2024-05-27T09:24:39.406 INFO:teuthology.orchestra.run.smithi163.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 6.1 MB/s | 260 kB 00:00 2024-05-27T09:24:39.465 INFO:teuthology.orchestra.run.smithi163.stdout:(156/159): openssl-3.2.1-1.el9.x86_64.rpm 23 MB/s | 1.3 MB 00:00 2024-05-27T09:24:39.540 INFO:teuthology.orchestra.run.smithi163.stdout:(157/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 32 MB/s | 2.4 MB 00:00 2024-05-27T09:24:39.657 INFO:teuthology.orchestra.run.smithi163.stdout:(158/159): librbd1-19.0.0-3868.ga6507388.el9.x8 4.2 MB/s | 3.1 MB 00:00 2024-05-27T09:24:39.751 INFO:teuthology.orchestra.run.smithi114.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3868.ga6 623 kB/s | 274 kB 00:00 2024-05-27T09:24:39.843 INFO:teuthology.orchestra.run.smithi114.stdout:(33/159): ceph-mgr-rook-19.0.0-3868.ga6507388.e 544 kB/s | 49 kB 00:00 2024-05-27T09:24:39.935 INFO:teuthology.orchestra.run.smithi114.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3868.ga 181 kB/s | 16 kB 00:00 2024-05-27T09:24:40.007 INFO:teuthology.orchestra.run.smithi163.stdout:(159/159): librados2-19.0.0-3868.ga6507388.el9. 2.9 MB/s | 3.3 MB 00:01 2024-05-27T09:24:40.013 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:24:40.013 INFO:teuthology.orchestra.run.smithi163.stdout:Total 19 MB/s | 219 MB 00:11 2024-05-27T09:24:40.077 INFO:teuthology.orchestra.run.smithi114.stdout:(35/159): ceph-mgr-diskprediction-local-19.0.0- 6.6 MB/s | 7.4 MB 00:01 2024-05-27T09:24:40.111 INFO:teuthology.orchestra.run.smithi114.stdout:(36/159): ceph-volume-19.0.0-3868.ga6507388.el9 1.5 MB/s | 273 kB 00:00 2024-05-27T09:24:40.561 INFO:teuthology.orchestra.run.smithi114.stdout:(37/159): ceph-test-19.0.0-3868.ga6507388.el9.x 11 MB/s | 43 MB 00:04 2024-05-27T09:24:40.578 INFO:teuthology.orchestra.run.smithi114.stdout:(38/159): cephfs-top-19.0.0-3868.ga6507388.el9. 39 kB/s | 18 kB 00:00 2024-05-27T09:24:40.620 INFO:teuthology.orchestra.run.smithi114.stdout:(39/159): cephadm-19.0.0-3868.ga6507388.el9.noa 1.4 MB/s | 764 kB 00:00 2024-05-27T09:24:40.737 INFO:teuthology.orchestra.run.smithi114.stdout:(40/159): fuse-2.9.9-15.el9.x86_64.rpm 462 kB/s | 80 kB 00:00 2024-05-27T09:24:40.829 INFO:teuthology.orchestra.run.smithi114.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 941 kB/s | 195 kB 00:00 2024-05-27T09:24:40.862 INFO:teuthology.orchestra.run.smithi114.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-05-27T09:24:40.896 INFO:teuthology.orchestra.run.smithi114.stdout:(43/159): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-05-27T09:24:40.930 INFO:teuthology.orchestra.run.smithi114.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-05-27T09:24:40.963 INFO:teuthology.orchestra.run.smithi114.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-05-27T09:24:40.997 INFO:teuthology.orchestra.run.smithi114.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-05-27T09:24:41.072 INFO:teuthology.orchestra.run.smithi114.stdout:(47/159): python3-cryptography-36.0.1-4.el9.x86 5.1 MB/s | 1.2 MB 00:00 2024-05-27T09:24:41.198 INFO:teuthology.orchestra.run.smithi114.stdout:(48/159): python3-urllib3-1.26.5-5.el9.noarch.r 920 kB/s | 215 kB 00:00 2024-05-27T09:24:41.243 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-05-27T09:24:41.406 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-05-27T09:24:41.406 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-05-27T09:24:42.065 INFO:teuthology.orchestra.run.smithi114.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 123 kB/s | 107 kB 00:00 2024-05-27T09:24:42.115 INFO:teuthology.orchestra.run.smithi114.stdout:(50/159): automake-1.16.2-8.el9.noarch.rpm 617 kB/s | 689 kB 00:01 2024-05-27T09:24:42.265 INFO:teuthology.orchestra.run.smithi114.stdout:(51/159): bison-3.7.4-5.el9.x86_64.rpm 791 kB/s | 943 kB 00:01 2024-05-27T09:24:42.707 INFO:teuthology.orchestra.run.smithi114.stdout:(52/159): elfutils-libelf-devel-0.191-4.el9.x86 34 kB/s | 22 kB 00:00 2024-05-27T09:24:43.032 INFO:teuthology.orchestra.run.smithi114.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 43 kB/s | 33 kB 00:00 2024-05-27T09:24:43.091 INFO:teuthology.orchestra.run.smithi114.stdout:(54/159): flex-2.6.4-9.el9.x86_64.rpm 322 kB/s | 313 kB 00:00 2024-05-27T09:24:43.180 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-05-27T09:24:43.180 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-05-27T09:24:43.616 INFO:teuthology.orchestra.run.smithi114.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 31 kB/s | 18 kB 00:00 2024-05-27T09:24:43.675 INFO:teuthology.orchestra.run.smithi114.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 19 kB/s | 11 kB 00:00 2024-05-27T09:24:44.025 INFO:teuthology.orchestra.run.smithi114.stdout:(57/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.3 MB/s | 3.0 MB 00:01 2024-05-27T09:24:44.359 INFO:teuthology.orchestra.run.smithi114.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 66 kB/s | 45 kB 00:00 2024-05-27T09:24:44.467 INFO:teuthology.orchestra.run.smithi114.stdout:(59/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 189 kB/s | 160 kB 00:00 2024-05-27T09:24:45.142 INFO:teuthology.orchestra.run.smithi114.stdout:(60/159): librdkafka-1.6.1-102.el9.x86_64.rpm 595 kB/s | 662 kB 00:01 2024-05-27T09:24:45.168 INFO:teuthology.orchestra.run.smithi114.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 36 kB/s | 25 kB 00:00 2024-05-27T09:24:45.510 INFO:teuthology.orchestra.run.smithi114.stdout:(62/159): libtool-2.4.6-45.el9.x86_64.rpm 506 kB/s | 582 kB 00:01 2024-05-27T09:24:45.835 INFO:teuthology.orchestra.run.smithi114.stdout:(63/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 69 kB/s | 47 kB 00:00 2024-05-27T09:24:46.160 INFO:teuthology.orchestra.run.smithi114.stdout:(64/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 295 kB/s | 292 kB 00:00 2024-05-27T09:24:46.394 INFO:teuthology.orchestra.run.smithi114.stdout:(65/159): lua-5.4.4-4.el9.x86_64.rpm 214 kB/s | 188 kB 00:00 2024-05-27T09:24:46.529 INFO:teuthology.orchestra.run.smithi114.stdout:(66/159): openblas-0.3.21-2.el9.x86_64.rpm 51 kB/s | 35 kB 00:00 2024-05-27T09:24:47.511 INFO:teuthology.orchestra.run.smithi114.stdout:(67/159): openblas-openmp-0.3.21-2.el9.x86_64.r 3.5 MB/s | 4.7 MB 00:01 2024-05-27T09:24:47.653 INFO:teuthology.orchestra.run.smithi114.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 1.4 MB/s | 1.6 MB 00:01 2024-05-27T09:24:47.761 INFO:teuthology.orchestra.run.smithi114.stdout:(69/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.4 MB/s | 4.4 MB 00:01 2024-05-27T09:24:48.620 INFO:teuthology.orchestra.run.smithi114.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 138 kB/s | 133 kB 00:00 2024-05-27T09:24:48.679 INFO:teuthology.orchestra.run.smithi114.stdout:(71/159): postgresql-contrib-13.14-1.el9.x86_64 781 kB/s | 911 kB 00:01 2024-05-27T09:24:49.237 INFO:teuthology.orchestra.run.smithi114.stdout:(72/159): postgresql-server-13.14-1.el9.x86_64. 4.1 MB/s | 5.8 MB 00:01 2024-05-27T09:24:49.406 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-05-27T09:24:49.446 INFO:teuthology.orchestra.run.smithi114.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 114 kB/s | 87 kB 00:00 2024-05-27T09:24:49.561 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-27T09:24:49.599 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-27T09:24:49.662 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-27T09:24:49.694 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-27T09:24:49.720 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-27T09:24:49.721 INFO:teuthology.orchestra.run.smithi114.stdout:(74/159): protobuf-3.14.0-13.el9.x86_64.rpm 940 kB/s | 1.0 MB 00:01 2024-05-27T09:24:49.762 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-27T09:24:50.091 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-27T09:24:50.198 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librados2-2:19.0.0-3868.ga6507388.el9.x86_64 8/170 2024-05-27T09:24:50.246 INFO:teuthology.orchestra.run.smithi114.stdout:(75/159): python3-colorama-0.4.6-3.el9.noarch.r 77 kB/s | 61 kB 00:00 2024-05-27T09:24:50.322 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-2:19.0.0-3868.ga6507388.el9.x86_64 8/170 2024-05-27T09:24:50.349 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs2-2:19.0.0-3868.ga6507388.el9.x86_64 9/170 2024-05-27T09:24:50.440 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephfs2-2:19.0.0-3868.ga6507388.el9.x86_64 9/170 2024-05-27T09:24:50.470 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rados-2:19.0.0-3868.ga6507388.el9.x86_64 10/170 2024-05-27T09:24:50.503 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-27T09:24:50.622 INFO:teuthology.orchestra.run.smithi114.stdout:(76/159): python3-babel-2.9.1-2.el9.noarch.rpm 4.3 MB/s | 6.0 MB 00:01 2024-05-27T09:24:50.664 INFO:teuthology.orchestra.run.smithi114.stdout:(77/159): python3-devel-3.9.18-3.el9.x86_64.rpm 280 kB/s | 244 kB 00:00 2024-05-27T09:24:50.695 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-27T09:24:50.751 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-27T09:24:50.779 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephsqlite-2:19.0.0-3868.ga6507388.el9.x86_64 14/170 2024-05-27T09:24:50.884 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3868.ga6507388.el9.x86_64 14/170 2024-05-27T09:24:50.914 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libradosstriper1-2:19.0.0-3868.ga6507388.el9.x86 15/170 2024-05-27T09:24:50.976 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3868.ga6507388.el9.x86 15/170 2024-05-27T09:24:50.998 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T09:24:51.016 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T09:24:51.206 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T09:24:51.222 INFO:teuthology.orchestra.run.smithi114.stdout:(78/159): python3-jinja2-2.11.3-6.el9.noarch.rp 255 kB/s | 248 kB 00:00 2024-05-27T09:24:51.264 INFO:teuthology.orchestra.run.smithi114.stdout:(79/159): python3-jsonpatch-1.21-16.el9.noarch. 40 kB/s | 26 kB 00:00 2024-05-27T09:24:51.289 INFO:teuthology.orchestra.run.smithi114.stdout:(80/159): python3-jsonpointer-2.0-4.el9.noarch. 31 kB/s | 19 kB 00:00 2024-05-27T09:24:51.335 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T09:24:51.522 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T09:24:51.572 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T09:24:51.669 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T09:24:51.782 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T09:24:51.940 INFO:teuthology.orchestra.run.smithi114.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 52 kB/s | 35 kB 00:00 2024-05-27T09:24:51.998 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T09:24:52.065 INFO:teuthology.orchestra.run.smithi114.stdout:(82/159): python3-mako-1.1.4-6.el9.noarch.rpm 205 kB/s | 172 kB 00:00 2024-05-27T09:24:52.121 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-27T09:24:52.155 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-27T09:24:52.188 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-27T09:24:52.261 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-27T09:24:52.295 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-27T09:24:52.339 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-27T09:24:52.380 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-27T09:24:52.408 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-27T09:24:52.478 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-27T09:24:52.530 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-27T09:24:52.561 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-27T09:24:52.596 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-27T09:24:52.672 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-27T09:24:52.707 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-common-2:19.0.0-3868.ga6507388.el9. 33/170 2024-05-27T09:24:52.740 INFO:teuthology.orchestra.run.smithi114.stdout:(83/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.5 MB/s | 5.1 MB 00:01 2024-05-27T09:24:52.750 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-argparse-2:19.0.0-3868.ga6507388.el 34/170 2024-05-27T09:24:52.779 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cephfs-2:19.0.0-3868.ga6507388.el9.x86_6 35/170 2024-05-27T09:24:52.848 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-27T09:24:52.882 INFO:teuthology.orchestra.run.smithi114.stdout:(84/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 266 kB/s | 250 kB 00:00 2024-05-27T09:24:52.893 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-27T09:24:52.967 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-27T09:24:53.066 INFO:teuthology.orchestra.run.smithi114.stdout:(85/159): python3-oauthlib-3.1.1-5.el9.noarch.r 222 kB/s | 222 kB 00:00 2024-05-27T09:24:53.101 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-27T09:24:53.162 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-27T09:24:53.566 INFO:teuthology.orchestra.run.smithi114.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 62 kB/s | 42 kB 00:00 2024-05-27T09:24:53.599 INFO:teuthology.orchestra.run.smithi114.stdout:(87/159): python3-packaging-20.9-5.el9.noarch.r 90 kB/s | 77 kB 00:00 2024-05-27T09:24:53.685 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-27T09:24:53.766 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-27T09:24:53.795 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-27T09:24:53.840 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-27T09:24:53.889 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-27T09:24:53.927 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librados-devel-2:19.0.0-3868.ga6507388.el9.x86_6 46/170 2024-05-27T09:24:53.968 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-27T09:24:54.125 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-27T09:24:54.158 INFO:teuthology.orchestra.run.smithi114.stdout:(88/159): python3-protobuf-3.14.0-13.el9.noarch 247 kB/s | 269 kB 00:01 2024-05-27T09:24:54.196 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-27T09:24:54.287 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-27T09:24:54.352 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-27T09:24:54.425 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-27T09:24:54.456 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-27T09:24:54.492 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-27T09:24:54.539 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-27T09:24:54.550 INFO:teuthology.orchestra.run.smithi114.stdout:(89/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 162 kB/s | 159 kB 00:00 2024-05-27T09:24:54.575 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-27T09:24:54.600 INFO:teuthology.orchestra.run.smithi114.stdout:(90/159): python3-pyasn1-modules-0.4.8-6.el9.no 279 kB/s | 279 kB 00:00 2024-05-27T09:24:54.618 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-27T09:24:54.659 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-27T09:24:54.796 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-27T09:24:54.837 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-27T09:24:54.879 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-27T09:24:54.934 INFO:teuthology.orchestra.run.smithi114.stdout:(91/159): python3-pytz-2021.1-5.el9.noarch.rpm 66 kB/s | 51 kB 00:00 2024-05-27T09:24:55.132 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-27T09:24:55.162 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-27T09:24:55.202 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-27T09:24:55.227 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-27T09:24:55.252 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-27T09:24:55.267 INFO:teuthology.orchestra.run.smithi114.stdout:(92/159): python3-requests-oauthlib-1.3.0-12.el 75 kB/s | 54 kB 00:00 2024-05-27T09:24:55.327 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-27T09:24:55.454 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-27T09:24:55.492 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-27T09:24:55.520 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-27T09:24:55.554 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-27T09:24:55.602 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-27T09:24:55.634 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-27T09:24:55.634 INFO:teuthology.orchestra.run.smithi114.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 60 kB/s | 42 kB 00:00 2024-05-27T09:24:55.663 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-27T09:24:55.689 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-27T09:24:55.722 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-27T09:24:55.764 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-27T09:24:55.799 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-27T09:24:55.885 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-27T09:24:55.929 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-27T09:24:55.951 INFO:teuthology.orchestra.run.smithi114.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 65 kB/s | 44 kB 00:00 2024-05-27T09:24:55.960 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-27T09:24:56.007 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-27T09:24:56.106 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-27T09:24:56.161 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-27T09:24:56.335 INFO:teuthology.orchestra.run.smithi114.stdout:(95/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.4 MB/s | 16 MB 00:01 2024-05-27T09:24:56.727 INFO:teuthology.orchestra.run.smithi114.stdout:(96/159): qatlib-24.02.0-1.el9.x86_64.rpm 203 kB/s | 221 kB 00:01 2024-05-27T09:24:56.802 INFO:teuthology.orchestra.run.smithi114.stdout:(97/159): qatlib-service-24.02.0-1.el9.x86_64.r 42 kB/s | 36 kB 00:00 2024-05-27T09:24:56.889 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-27T09:24:56.933 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-27T09:24:56.976 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-27T09:24:57.073 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-27T09:24:57.102 INFO:teuthology.orchestra.run.smithi114.stdout:(98/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 68 kB/s | 47 kB 00:00 2024-05-27T09:24:57.122 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-27T09:24:57.167 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-27T09:24:57.255 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-27T09:24:57.303 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-27T09:24:57.486 INFO:teuthology.orchestra.run.smithi114.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 85 kB/s | 58 kB 00:00 2024-05-27T09:24:57.489 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-27T09:24:57.526 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-27T09:24:57.603 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-27T09:24:57.678 INFO:teuthology.orchestra.run.smithi114.stdout:(100/159): socat-1.7.4.1-5.el9.x86_64.rpm 321 kB/s | 305 kB 00:00 2024-05-27T09:24:57.937 INFO:teuthology.orchestra.run.smithi114.stdout:(101/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 113 kB/s | 94 kB 00:00 2024-05-27T09:24:58.112 INFO:teuthology.orchestra.run.smithi114.stdout:(102/159): lua-devel-5.4.4-4.el9.x86_64.rpm 130 kB/s | 22 kB 00:00 2024-05-27T09:24:58.255 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-27T09:24:58.279 INFO:teuthology.orchestra.run.smithi114.stdout:(103/159): protobuf-compiler-3.14.0-13.el9.x86_ 5.1 MB/s | 863 kB 00:00 2024-05-27T09:24:58.329 INFO:teuthology.orchestra.run.smithi114.stdout:(104/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 76 kB/s | 64 kB 00:00 2024-05-27T09:24:58.365 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-27T09:24:58.471 INFO:teuthology.orchestra.run.smithi114.stdout:(105/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 57 kB/s | 45 kB 00:00 2024-05-27T09:24:58.723 INFO:teuthology.orchestra.run.smithi114.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 78 kB/s | 19 kB 00:00 2024-05-27T09:24:58.863 INFO:teuthology.orchestra.run.smithi114.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 580 kB/s | 309 kB 00:00 2024-05-27T09:24:58.905 INFO:teuthology.orchestra.run.smithi114.stdout:(108/159): abseil-cpp-20211102.0-3.el9.x86_64.r 877 kB/s | 548 kB 00:00 2024-05-27T09:24:58.964 INFO:teuthology.orchestra.run.smithi114.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-05-27T09:24:59.031 INFO:teuthology.orchestra.run.smithi114.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 390 kB/s | 49 kB 00:00 2024-05-27T09:24:59.231 INFO:teuthology.orchestra.run.smithi114.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 253 kB/s | 67 kB 00:00 2024-05-27T09:24:59.265 INFO:teuthology.orchestra.run.smithi114.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 650 kB/s | 151 kB 00:00 2024-05-27T09:24:59.498 INFO:teuthology.orchestra.run.smithi114.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 2.3 MB/s | 548 kB 00:00 2024-05-27T09:24:59.606 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-27T09:24:59.624 INFO:teuthology.orchestra.run.smithi114.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 349 kB/s | 43 kB 00:00 2024-05-27T09:24:59.674 INFO:teuthology.orchestra.run.smithi114.stdout:(115/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.9 MB/s | 839 kB 00:00 2024-05-27T09:24:59.687 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-27T09:24:59.715 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-27T09:24:59.732 INFO:teuthology.orchestra.run.smithi114.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 299 kB/s | 32 kB 00:00 2024-05-27T09:24:59.748 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-27T09:24:59.858 INFO:teuthology.orchestra.run.smithi114.stdout:(117/159): libarrow-9.0.0-11.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-05-27T09:24:59.883 INFO:teuthology.orchestra.run.smithi114.stdout:(118/159): python3-certifi-2021.10.8-2.el9.noar 72 kB/s | 15 kB 00:00 2024-05-27T09:24:59.917 INFO:teuthology.orchestra.run.smithi114.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 937 kB/s | 172 kB 00:00 2024-05-27T09:25:00.025 INFO:teuthology.orchestra.run.smithi114.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-05-27T09:25:00.059 INFO:teuthology.orchestra.run.smithi114.stdout:(121/159): python3-cherrypy-18.6.1-2.el9.noarch 1.8 MB/s | 358 kB 00:00 2024-05-27T09:25:00.069 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-27T09:25:00.092 INFO:teuthology.orchestra.run.smithi114.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 1.1 MB/s | 206 kB 00:00 2024-05-27T09:25:00.145 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librbd1-2:19.0.0-3868.ga6507388.el9.x86_64 103/170 2024-05-27T09:25:00.176 INFO:teuthology.orchestra.run.smithi114.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-05-27T09:25:00.201 INFO:teuthology.orchestra.run.smithi114.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-05-27T09:25:00.218 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-2:19.0.0-3868.ga6507388.el9.x86_64 103/170 2024-05-27T09:25:00.249 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rbd-2:19.0.0-3868.ga6507388.el9.x86_64 104/170 2024-05-27T09:25:00.285 INFO:teuthology.orchestra.run.smithi114.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 164 kB/s | 18 kB 00:00 2024-05-27T09:25:00.305 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-27T09:25:00.310 INFO:teuthology.orchestra.run.smithi114.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 215 kB/s | 23 kB 00:00 2024-05-27T09:25:00.369 INFO:teuthology.orchestra.run.smithi114.stdout:(127/159): python3-grpcio-1.46.7-10.el9.x86_64. 5.9 MB/s | 2.0 MB 00:00 2024-05-27T09:25:00.394 INFO:teuthology.orchestra.run.smithi114.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 180 kB/s | 19 kB 00:00 2024-05-27T09:25:00.420 INFO:teuthology.orchestra.run.smithi114.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-05-27T09:25:00.503 INFO:teuthology.orchestra.run.smithi114.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 428 kB/s | 46 kB 00:00 2024-05-27T09:25:00.537 INFO:teuthology.orchestra.run.smithi114.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 677 kB/s | 79 kB 00:00 2024-05-27T09:25:00.579 INFO:teuthology.orchestra.run.smithi114.stdout:(132/159): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-05-27T09:25:00.604 INFO:teuthology.orchestra.run.smithi114.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 579 kB/s | 58 kB 00:00 2024-05-27T09:25:00.671 INFO:teuthology.orchestra.run.smithi114.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-05-27T09:25:00.697 INFO:teuthology.orchestra.run.smithi114.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 141 kB/s | 16 kB 00:00 2024-05-27T09:25:00.738 INFO:teuthology.orchestra.run.smithi114.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 674 kB/s | 90 kB 00:00 2024-05-27T09:25:00.772 INFO:teuthology.orchestra.run.smithi114.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 282 kB/s | 28 kB 00:00 2024-05-27T09:25:00.789 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-27T09:25:00.797 INFO:teuthology.orchestra.run.smithi114.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 308 kB/s | 31 kB 00:00 2024-05-27T09:25:00.830 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librgw2-2:19.0.0-3868.ga6507388.el9.x86_64 107/170 2024-05-27T09:25:00.856 INFO:teuthology.orchestra.run.smithi114.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-05-27T09:25:00.881 INFO:teuthology.orchestra.run.smithi114.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 546 kB/s | 59 kB 00:00 2024-05-27T09:25:00.885 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librgw2-2:19.0.0-3868.ga6507388.el9.x86_64 107/170 2024-05-27T09:25:00.915 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rgw-2:19.0.0-3868.ga6507388.el9.x86_64 108/170 2024-05-27T09:25:00.922 INFO:teuthology.orchestra.run.smithi114.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 287 kB/s | 36 kB 00:00 2024-05-27T09:25:00.965 INFO:teuthology.orchestra.run.smithi114.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 470 kB/s | 51 kB 00:00 2024-05-27T09:25:00.998 INFO:teuthology.orchestra.run.smithi114.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-05-27T09:25:01.032 INFO:teuthology.orchestra.run.smithi114.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 823 kB/s | 90 kB 00:00 2024-05-27T09:25:01.099 INFO:teuthology.orchestra.run.smithi114.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 201 kB/s | 20 kB 00:00 2024-05-27T09:25:01.133 INFO:teuthology.orchestra.run.smithi114.stdout:(146/159): python3-werkzeug-2.0.3-3.el9.1.noarc 2.5 MB/s | 427 kB 00:00 2024-05-27T09:25:01.166 INFO:teuthology.orchestra.run.smithi114.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-05-27T09:25:01.391 INFO:teuthology.orchestra.run.smithi114.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 5.4 MB/s | 1.6 MB 00:00 2024-05-27T09:25:01.658 INFO:teuthology.orchestra.run.smithi114.stdout:(149/159): librbd1-19.0.0-3868.ga6507388.el9.x8 6.3 MB/s | 3.1 MB 00:00 2024-05-27T09:25:01.734 INFO:teuthology.orchestra.run.smithi114.stdout:(150/159): librados2-19.0.0-3868.ga6507388.el9. 5.5 MB/s | 3.3 MB 00:00 2024-05-27T09:25:01.817 INFO:teuthology.orchestra.run.smithi114.stdout:(151/159): NetworkManager-1.47.90-1.el9.x86_64. 5.4 MB/s | 2.3 MB 00:00 2024-05-27T09:25:01.834 INFO:teuthology.orchestra.run.smithi114.stdout:(152/159): NetworkManager-team-1.47.90-1.el9.x8 417 kB/s | 41 kB 00:00 2024-05-27T09:25:01.901 INFO:teuthology.orchestra.run.smithi114.stdout:(153/159): NetworkManager-tui-1.47.90-1.el9.x86 2.9 MB/s | 248 kB 00:00 2024-05-27T09:25:01.918 INFO:teuthology.orchestra.run.smithi114.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 442 kB/s | 37 kB 00:00 2024-05-27T09:25:01.977 INFO:teuthology.orchestra.run.smithi114.stdout:(155/159): NetworkManager-libnm-1.47.90-1.el9.x 5.7 MB/s | 1.8 MB 00:00 2024-05-27T09:25:02.010 INFO:teuthology.orchestra.run.smithi114.stdout:(156/159): elfutils-libelf-0.191-4.el9.x86_64.r 1.9 MB/s | 207 kB 00:00 2024-05-27T09:25:02.044 INFO:teuthology.orchestra.run.smithi114.stdout:(157/159): elfutils-libs-0.191-4.el9.x86_64.rpm 2.0 MB/s | 260 kB 00:00 2024-05-27T09:25:02.127 INFO:teuthology.orchestra.run.smithi114.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 1.3 MB 00:00 2024-05-27T09:25:02.228 INFO:teuthology.orchestra.run.smithi114.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-05-27T09:25:02.234 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:25:02.235 INFO:teuthology.orchestra.run.smithi114.stdout:Total 7.4 MB/s | 219 MB 00:29 2024-05-27T09:25:03.190 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 109/170 2024-05-27T09:25:03.236 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 109/170 2024-05-27T09:25:03.791 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 109/170 2024-05-27T09:25:03.831 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-base-2:19.0.0-3868.ga6507388.el9.x86_64 110/170 2024-05-27T09:25:03.921 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-base-2:19.0.0-3868.ga6507388.el9.x86_64 110/170 2024-05-27T09:25:03.921 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T09:25:03.921 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T09:25:03.921 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:25:03.949 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-selinux-2:19.0.0-3868.ga6507388.el9.x86_64 111/170 2024-05-27T09:25:05.306 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-05-27T09:25:05.556 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-05-27T09:25:05.556 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-05-27T09:25:08.233 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-05-27T09:25:08.233 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-05-27T09:25:14.703 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-05-27T09:25:14.798 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-27T09:25:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-27T09:25:14.965 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-27T09:25:15.024 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-27T09:25:15.056 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-27T09:25:15.159 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-27T09:25:15.591 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-27T09:25:15.592 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3868.ga6507388.el9.x86_64 111/170 2024-05-27T09:25:15.592 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /sys 2024-05-27T09:25:15.592 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /proc 2024-05-27T09:25:15.593 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /mnt 2024-05-27T09:25:15.593 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /var/tmp 2024-05-27T09:25:15.593 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /home 2024-05-27T09:25:15.593 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /root 2024-05-27T09:25:15.593 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /tmp 2024-05-27T09:25:15.593 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:25:15.697 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librados2-2:19.0.0-3868.ga6507388.el9.x86_64 8/170 2024-05-27T09:25:15.783 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-volume-2:19.0.0-3868.ga6507388.el9.noarch 112/170 2024-05-27T09:25:15.799 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-volume-2:19.0.0-3868.ga6507388.el9.noarch 112/170 2024-05-27T09:25:15.799 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:25:15.799 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-27T09:25:15.799 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:25:15.874 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:19.0.0-3868.ga6507388.el9.x86_64 8/170 2024-05-27T09:25:15.913 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs2-2:19.0.0-3868.ga6507388.el9.x86_64 9/170 2024-05-27T09:25:16.049 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephfs2-2:19.0.0-3868.ga6507388.el9.x86_64 9/170 2024-05-27T09:25:16.088 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rados-2:19.0.0-3868.ga6507388.el9.x86_64 10/170 2024-05-27T09:25:16.177 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-27T09:25:16.519 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-27T09:25:16.575 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-27T09:25:16.602 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephsqlite-2:19.0.0-3868.ga6507388.el9.x86_64 14/170 2024-05-27T09:25:16.751 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3868.ga6507388.el9.x86_64 14/170 2024-05-27T09:25:16.778 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libradosstriper1-2:19.0.0-3868.ga6507388.el9.x86 15/170 2024-05-27T09:25:16.863 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3868.ga6507388.el9.x86 15/170 2024-05-27T09:25:16.887 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T09:25:16.954 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T09:25:17.236 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T09:25:17.386 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-osd-2:19.0.0-3868.ga6507388.el9.x86_64 113/170 2024-05-27T09:25:17.412 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-osd-2:19.0.0-3868.ga6507388.el9.x86_64 113/170 2024-05-27T09:25:17.412 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:25:17.412 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T09:25:17.412 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T09:25:17.412 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T09:25:17.413 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:25:17.551 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T09:25:17.645 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mds-2:19.0.0-3868.ga6507388.el9.x86_64 114/170 2024-05-27T09:25:17.668 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mds-2:19.0.0-3868.ga6507388.el9.x86_64 114/170 2024-05-27T09:25:17.668 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:25:17.668 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T09:25:17.668 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T09:25:17.668 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T09:25:17.668 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:25:17.752 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T09:25:17.840 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T09:25:18.025 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T09:25:18.114 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T09:25:18.129 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mon-2:19.0.0-3868.ga6507388.el9.x86_64 115/170 2024-05-27T09:25:18.152 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mon-2:19.0.0-3868.ga6507388.el9.x86_64 115/170 2024-05-27T09:25:18.152 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:25:18.152 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T09:25:18.152 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T09:25:18.152 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T09:25:18.152 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:25:18.251 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-27T09:25:18.308 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-27T09:25:18.363 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-27T09:25:18.441 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T09:25:18.561 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-27T09:25:18.605 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-27T09:25:18.632 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-27T09:25:18.660 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-27T09:25:18.684 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-27T09:25:18.698 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-27T09:25:18.772 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-27T09:25:18.794 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-27T09:25:18.840 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-27T09:25:18.900 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-27T09:25:18.904 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-27T09:25:18.943 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-27T09:25:18.965 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-27T09:25:18.975 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-27T09:25:18.996 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-27T09:25:19.024 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-27T09:25:19.077 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-27T09:25:19.155 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-27T09:25:19.195 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-27T09:25:19.250 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-27T09:25:19.358 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-27T09:25:19.397 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-common-2:19.0.0-3868.ga6507388.el9. 33/170 2024-05-27T09:25:19.446 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-argparse-2:19.0.0-3868.ga6507388.el 34/170 2024-05-27T09:25:19.460 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephadm-2:19.0.0-3868.ga6507388.el9.noarch 127/170 2024-05-27T09:25:19.479 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cephfs-2:19.0.0-3868.ga6507388.el9.x86_6 35/170 2024-05-27T09:25:19.490 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephadm-2:19.0.0-3868.ga6507388.el9.noarch 127/170 2024-05-27T09:25:19.530 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3868.ga6507388.e 128/170 2024-05-27T09:25:19.590 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-27T09:25:19.600 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3868.ga6507388. 129/170 2024-05-27T09:25:19.630 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3868.ga6507388.el9.noa 130/170 2024-05-27T09:25:19.657 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-27T09:25:19.757 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-27T09:25:19.944 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-27T09:25:20.011 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-27T09:25:20.666 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3868.ga6507388.el9.noa 130/170 2024-05-27T09:25:20.684 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-27T09:25:20.698 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3868.ga6507388.el9.n 131/170 2024-05-27T09:25:20.783 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-27T09:25:20.820 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-27T09:25:20.863 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-27T09:25:20.921 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-27T09:25:20.968 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librados-devel-2:19.0.0-3868.ga6507388.el9.x86_6 46/170 2024-05-27T09:25:21.033 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-27T09:25:21.264 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-27T09:25:21.323 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-27T09:25:21.442 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-27T09:25:21.515 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-27T09:25:21.600 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-27T09:25:21.639 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-27T09:25:21.657 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3868.ga6507388.el9.n 131/170 2024-05-27T09:25:21.670 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-27T09:25:21.682 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3868.ga65 132/170 2024-05-27T09:25:21.733 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-27T09:25:21.778 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-27T09:25:21.795 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3868.ga65 132/170 2024-05-27T09:25:21.838 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-27T09:25:21.897 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-27T09:25:21.968 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3868.ga6507388.el 133/170 2024-05-27T09:25:22.001 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-2:19.0.0-3868.ga6507388.el9.x86_64 134/170 2024-05-27T09:25:22.025 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3868.ga6507388.el9.x86_64 134/170 2024-05-27T09:25:22.025 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:25:22.025 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T09:25:22.025 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T09:25:22.025 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T09:25:22.025 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:25:22.085 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-rook-2:19.0.0-3868.ga6507388.el9.noarch 135/170 2024-05-27T09:25:22.093 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-27T09:25:22.097 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3868.ga6507388.el9.noarch 135/170 2024-05-27T09:25:22.147 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-27T09:25:22.194 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-2:19.0.0-3868.ga6507388.el9.x86_64 136/170 2024-05-27T09:25:22.201 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-27T09:25:22.242 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 137/170 2024-05-27T09:25:22.328 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-27T09:25:22.372 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-27T09:25:22.417 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-27T09:25:22.453 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-27T09:25:22.486 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-27T09:25:22.679 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-27T09:25:22.857 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-27T09:25:22.901 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-27T09:25:22.940 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-27T09:25:22.989 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-27T09:25:23.039 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-27T09:25:23.085 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-27T09:25:23.127 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-27T09:25:23.172 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-27T09:25:23.173 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-27T09:25:23.205 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-radosgw-2:19.0.0-3868.ga6507388.el9.x86_64 139/170 2024-05-27T09:25:23.215 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-27T09:25:23.228 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3868.ga6507388.el9.x86_64 139/170 2024-05-27T09:25:23.228 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:25:23.228 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T09:25:23.228 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T09:25:23.228 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T09:25:23.228 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:25:23.273 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3868.ga6507 140/170 2024-05-27T09:25:23.280 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-27T09:25:23.295 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3868.ga6507 140/170 2024-05-27T09:25:23.295 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:25:23.295 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T09:25:23.295 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:25:23.323 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-27T09:25:23.339 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 141/170 2024-05-27T09:25:23.362 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 141/170 2024-05-27T09:25:23.362 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:25:23.363 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-27T09:25:23.363 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T09:25:23.363 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T09:25:23.363 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:25:23.462 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-27T09:25:23.519 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-27T09:25:23.572 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-27T09:25:23.642 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-27T09:25:23.682 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 142/170 2024-05-27T09:25:23.704 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 142/170 2024-05-27T09:25:23.705 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:25:23.705 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T09:25:23.705 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T09:25:23.705 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T09:25:23.705 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:25:23.784 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-27T09:25:23.852 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-27T09:25:24.798 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-27T09:25:24.859 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-27T09:25:24.893 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-27T09:25:25.051 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-27T09:25:25.126 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-27T09:25:25.182 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-27T09:25:25.292 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-27T09:25:25.346 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-27T09:25:25.574 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-27T09:25:25.609 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-27T09:25:25.713 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-27T09:25:26.528 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-27T09:25:26.672 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-27T09:25:27.961 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-test-2:19.0.0-3868.ga6507388.el9.x86_64 143/170 2024-05-27T09:25:28.002 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 144/170 2024-05-27T09:25:28.043 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-nbd-2:19.0.0-3868.ga6507388.el9.x86_64 145/170 2024-05-27T09:25:28.076 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-27T09:25:28.103 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-27T09:25:28.257 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-27T09:25:28.288 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-shell-2:19.0.0-3868.ga6507388.el9.x86_64 148/170 2024-05-27T09:25:28.333 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-27T09:25:28.358 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-27T09:25:28.365 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs-devel-2:19.0.0-3868.ga6507388.el9.x86_ 150/170 2024-05-27T09:25:28.395 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-27T09:25:28.406 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-27T09:25:28.431 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-27T09:25:28.444 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-27T09:25:28.499 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T09:25:28.822 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-27T09:25:28.871 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librbd1-2:19.0.0-3868.ga6507388.el9.x86_64 103/170 2024-05-27T09:25:28.992 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:19.0.0-3868.ga6507388.el9.x86_64 103/170 2024-05-27T09:25:29.029 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rbd-2:19.0.0-3868.ga6507388.el9.x86_64 104/170 2024-05-27T09:25:29.057 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T09:25:29.093 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-27T09:25:29.097 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T09:25:29.118 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T09:25:29.653 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-27T09:25:29.691 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librgw2-2:19.0.0-3868.ga6507388.el9.x86_64 107/170 2024-05-27T09:25:29.784 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librgw2-2:19.0.0-3868.ga6507388.el9.x86_64 107/170 2024-05-27T09:25:29.818 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rgw-2:19.0.0-3868.ga6507388.el9.x86_64 108/170 2024-05-27T09:25:29.976 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-top-2:19.0.0-3868.ga6507388.el9.noarch 155/170 2024-05-27T09:25:30.062 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-27T09:25:30.111 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-27T09:25:30.200 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-27T09:25:30.257 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-27T09:25:30.282 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T09:25:30.308 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-27T09:25:30.334 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-27T09:25:30.334 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T09:25:30.419 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T09:25:30.452 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-27T09:25:30.452 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T09:25:30.545 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T09:25:30.570 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-27T09:25:30.575 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T09:25:30.575 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T09:25:30.755 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T09:25:30.755 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T09:25:30.831 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T09:25:30.856 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-27T09:25:30.919 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T09:25:32.463 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 109/170 2024-05-27T09:25:32.506 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 109/170 2024-05-27T09:25:33.171 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 109/170 2024-05-27T09:25:33.218 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-base-2:19.0.0-3868.ga6507388.el9.x86_64 110/170 2024-05-27T09:25:33.364 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-base-2:19.0.0-3868.ga6507388.el9.x86_64 110/170 2024-05-27T09:25:33.364 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T09:25:33.364 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T09:25:33.364 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:25:33.411 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-selinux-2:19.0.0-3868.ga6507388.el9.x86_64 111/170 2024-05-27T09:25:36.583 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T09:25:36.583 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-2:19.0.0-3868.ga6507388.el9.x86_64 1/170 2024-05-27T09:25:36.583 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-base-2:19.0.0-3868.ga6507388.el9.x86_64 2/170 2024-05-27T09:25:36.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 3/170 2024-05-27T09:25:36.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 4/170 2024-05-27T09:25:36.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3868.ga6507 5/170 2024-05-27T09:25:36.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mds-2:19.0.0-3868.ga6507388.el9.x86_64 6/170 2024-05-27T09:25:36.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-2:19.0.0-3868.ga6507388.el9.x86_64 7/170 2024-05-27T09:25:36.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mon-2:19.0.0-3868.ga6507388.el9.x86_64 8/170 2024-05-27T09:25:36.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-osd-2:19.0.0-3868.ga6507388.el9.x86_64 9/170 2024-05-27T09:25:36.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-radosgw-2:19.0.0-3868.ga6507388.el9.x86_64 10/170 2024-05-27T09:25:36.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-selinux-2:19.0.0-3868.ga6507388.el9.x86_64 11/170 2024-05-27T09:25:36.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-test-2:19.0.0-3868.ga6507388.el9.x86_64 12/170 2024-05-27T09:25:36.584 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 13/170 2024-05-27T09:25:36.585 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-shell-2:19.0.0-3868.ga6507388.el9.x86_64 14/170 2024-05-27T09:25:36.585 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs-devel-2:19.0.0-3868.ga6507388.el9.x86_ 15/170 2024-05-27T09:25:36.585 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs2-2:19.0.0-3868.ga6507388.el9.x86_64 16/170 2024-05-27T09:25:36.585 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephsqlite-2:19.0.0-3868.ga6507388.el9.x86_64 17/170 2024-05-27T09:25:36.585 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados-devel-2:19.0.0-3868.ga6507388.el9.x86_6 18/170 2024-05-27T09:25:36.586 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libradosstriper1-2:19.0.0-3868.ga6507388.el9.x86 19/170 2024-05-27T09:25:36.586 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librgw2-2:19.0.0-3868.ga6507388.el9.x86_64 20/170 2024-05-27T09:25:36.586 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3868.ga6507388.el 21/170 2024-05-27T09:25:36.586 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-common-2:19.0.0-3868.ga6507388.el9. 22/170 2024-05-27T09:25:36.586 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cephfs-2:19.0.0-3868.ga6507388.el9.x86_6 23/170 2024-05-27T09:25:36.587 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rados-2:19.0.0-3868.ga6507388.el9.x86_64 24/170 2024-05-27T09:25:36.587 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rbd-2:19.0.0-3868.ga6507388.el9.x86_64 25/170 2024-05-27T09:25:36.587 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rgw-2:19.0.0-3868.ga6507388.el9.x86_64 26/170 2024-05-27T09:25:36.587 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 27/170 2024-05-27T09:25:36.587 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 28/170 2024-05-27T09:25:36.587 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-nbd-2:19.0.0-3868.ga6507388.el9.x86_64 29/170 2024-05-27T09:25:36.587 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3868.ga6507388. 30/170 2024-05-27T09:25:36.602 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3868.ga6507388.el9.noa 31/170 2024-05-27T09:25:36.602 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3868.ga6507388.el9.n 32/170 2024-05-27T09:25:36.602 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3868.ga65 33/170 2024-05-27T09:25:36.602 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3868.ga6507388.el 34/170 2024-05-27T09:25:36.602 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3868.ga6507388.el9.noarch 35/170 2024-05-27T09:25:36.602 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3868.ga6507388.e 36/170 2024-05-27T09:25:36.602 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-volume-2:19.0.0-3868.ga6507388.el9.noarch 37/170 2024-05-27T09:25:36.603 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephadm-2:19.0.0-3868.ga6507388.el9.noarch 38/170 2024-05-27T09:25:36.603 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-top-2:19.0.0-3868.ga6507388.el9.noarch 39/170 2024-05-27T09:25:36.603 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-27T09:25:36.603 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-27T09:25:36.603 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-27T09:25:36.603 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-27T09:25:36.603 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-27T09:25:36.603 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-27T09:25:36.604 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-27T09:25:36.604 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-27T09:25:36.604 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-27T09:25:36.604 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-27T09:25:36.604 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-27T09:25:36.604 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-27T09:25:36.604 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-27T09:25:36.604 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-27T09:25:36.604 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-27T09:25:36.605 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-27T09:25:36.605 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-27T09:25:36.605 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-27T09:25:36.605 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-27T09:25:36.605 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-27T09:25:36.605 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-27T09:25:36.605 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-27T09:25:36.605 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-27T09:25:36.606 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-27T09:25:36.606 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-27T09:25:36.606 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-27T09:25:36.606 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-27T09:25:36.606 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-27T09:25:36.606 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-27T09:25:36.606 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-27T09:25:36.607 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-27T09:25:36.607 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-27T09:25:36.607 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-27T09:25:36.607 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-27T09:25:36.607 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-27T09:25:36.607 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-27T09:25:36.607 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-27T09:25:36.607 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-27T09:25:36.608 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-27T09:25:36.608 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-27T09:25:36.608 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-27T09:25:36.608 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-27T09:25:36.608 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-27T09:25:36.608 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-27T09:25:36.608 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-27T09:25:36.608 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-27T09:25:36.608 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-27T09:25:36.609 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-27T09:25:36.609 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-27T09:25:36.609 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-27T09:25:36.609 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-27T09:25:36.609 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-27T09:25:36.609 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-27T09:25:36.609 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-27T09:25:36.609 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-27T09:25:36.610 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-27T09:25:36.610 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-27T09:25:36.610 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-27T09:25:36.610 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-27T09:25:36.610 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-27T09:25:36.610 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-27T09:25:36.610 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-27T09:25:36.611 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-27T09:25:36.611 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-27T09:25:36.611 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-27T09:25:36.611 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-27T09:25:36.611 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-27T09:25:36.611 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-27T09:25:36.611 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-27T09:25:36.611 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-27T09:25:36.611 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-27T09:25:36.612 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-27T09:25:36.612 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-27T09:25:36.612 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-27T09:25:36.612 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-27T09:25:36.612 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-27T09:25:36.612 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-27T09:25:36.612 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-27T09:25:36.613 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-27T09:25:36.613 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-27T09:25:36.613 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-27T09:25:36.613 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-27T09:25:36.613 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-27T09:25:36.613 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-27T09:25:36.613 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-27T09:25:36.613 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-27T09:25:36.614 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-27T09:25:36.614 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-27T09:25:36.614 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-27T09:25:36.614 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-27T09:25:36.614 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-27T09:25:36.614 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-27T09:25:36.614 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-27T09:25:36.614 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-27T09:25:36.615 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-27T09:25:36.615 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-27T09:25:36.615 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-27T09:25:36.615 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-27T09:25:36.615 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-27T09:25:36.615 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-27T09:25:36.615 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-27T09:25:36.615 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-27T09:25:36.615 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-27T09:25:36.616 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-27T09:25:36.616 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-27T09:25:36.616 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-27T09:25:36.616 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-27T09:25:36.616 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-27T09:25:36.616 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-27T09:25:36.616 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-2:19.0.0-3868.ga6507388.el9.x86_64 149/170 2024-05-27T09:25:36.616 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-27T09:25:36.617 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-2:19.0.0-3868.ga6507388.el9.x86_64 151/170 2024-05-27T09:25:36.617 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-27T09:25:36.617 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T09:25:36.617 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-27T09:25:36.617 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-27T09:25:36.617 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-27T09:25:36.617 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-27T09:25:36.617 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-27T09:25:36.618 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-27T09:25:36.618 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T09:25:36.618 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-27T09:25:36.618 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-27T09:25:36.618 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-27T09:25:36.618 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-27T09:25:36.619 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-27T09:25:36.619 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-27T09:25:36.619 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-27T09:25:36.619 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-27T09:25:36.619 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-27T09:25:39.210 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-27T09:25:39.210 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:25:39.210 INFO:teuthology.orchestra.run.smithi163.stdout:Upgraded: 2024-05-27T09:25:39.210 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-27T09:25:39.210 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-27T09:25:39.210 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-27T09:25:39.210 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-27T09:25:39.210 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-27T09:25:39.211 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-27T09:25:39.211 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-27T09:25:39.211 INFO:teuthology.orchestra.run.smithi163.stdout: librados2-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.211 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.211 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-27T09:25:39.211 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-27T09:25:39.211 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-05-27T09:25:39.211 INFO:teuthology.orchestra.run.smithi163.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-27T09:25:39.211 INFO:teuthology.orchestra.run.smithi163.stdout: automake-1.16.2-8.el9.noarch 2024-05-27T09:25:39.211 INFO:teuthology.orchestra.run.smithi163.stdout: bison-3.7.4-5.el9.x86_64 2024-05-27T09:25:39.211 INFO:teuthology.orchestra.run.smithi163.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T09:25:39.211 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.211 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.211 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.211 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.212 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T09:25:39.212 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.212 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.212 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.212 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T09:25:39.212 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T09:25:39.212 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T09:25:39.212 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T09:25:39.212 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T09:25:39.212 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.212 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.212 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T09:25:39.213 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.213 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.213 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.213 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-volume-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T09:25:39.213 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T09:25:39.213 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.213 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-shell-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.213 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-top-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T09:25:39.213 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-27T09:25:39.213 INFO:teuthology.orchestra.run.smithi163.stdout: flex-2.6.4-9.el9.x86_64 2024-05-27T09:25:39.213 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T09:25:39.213 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T09:25:39.213 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T09:25:39.214 INFO:teuthology.orchestra.run.smithi163.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-27T09:25:39.214 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T09:25:39.214 INFO:teuthology.orchestra.run.smithi163.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-27T09:25:39.214 INFO:teuthology.orchestra.run.smithi163.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-27T09:25:39.214 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T09:25:39.214 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T09:25:39.214 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.214 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.214 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.214 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T09:25:39.214 INFO:teuthology.orchestra.run.smithi163.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T09:25:39.214 INFO:teuthology.orchestra.run.smithi163.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T09:25:39.214 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T09:25:39.215 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T09:25:39.215 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.215 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.215 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T09:25:39.215 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.217 INFO:teuthology.orchestra.run.smithi163.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-27T09:25:39.217 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T09:25:39.217 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-27T09:25:39.218 INFO:teuthology.orchestra.run.smithi163.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-27T09:25:39.218 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T09:25:39.218 INFO:teuthology.orchestra.run.smithi163.stdout: lua-5.4.4-4.el9.x86_64 2024-05-27T09:25:39.218 INFO:teuthology.orchestra.run.smithi163.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-27T09:25:39.218 INFO:teuthology.orchestra.run.smithi163.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-27T09:25:39.218 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T09:25:39.218 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T09:25:39.218 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-27T09:25:39.218 INFO:teuthology.orchestra.run.smithi163.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T09:25:39.219 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-13.14-1.el9.x86_64 2024-05-27T09:25:39.219 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-27T09:25:39.219 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-27T09:25:39.219 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-27T09:25:39.219 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-27T09:25:39.219 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-27T09:25:39.219 INFO:teuthology.orchestra.run.smithi163.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T09:25:39.219 INFO:teuthology.orchestra.run.smithi163.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-27T09:25:39.219 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T09:25:39.219 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T09:25:39.219 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T09:25:39.219 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.220 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.220 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.220 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T09:25:39.220 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T09:25:39.220 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T09:25:39.220 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T09:25:39.220 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-27T09:25:39.220 INFO:teuthology.orchestra.run.smithi163.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-27T09:25:39.220 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T09:25:39.220 INFO:teuthology.orchestra.run.smithi163.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T09:25:39.220 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T09:25:39.220 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-27T09:25:39.220 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-27T09:25:39.220 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T09:25:39.221 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T09:25:39.221 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T09:25:39.221 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T09:25:39.221 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T09:25:39.221 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T09:25:39.221 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T09:25:39.221 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T09:25:39.221 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T09:25:39.221 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T09:25:39.221 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T09:25:39.221 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T09:25:39.221 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T09:25:39.222 INFO:teuthology.orchestra.run.smithi163.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T09:25:39.222 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T09:25:39.222 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T09:25:39.222 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T09:25:39.222 INFO:teuthology.orchestra.run.smithi163.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-27T09:25:39.222 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T09:25:39.222 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T09:25:39.222 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T09:25:39.222 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T09:25:39.222 INFO:teuthology.orchestra.run.smithi163.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-27T09:25:39.222 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T09:25:39.223 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T09:25:39.223 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T09:25:39.223 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T09:25:39.223 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-27T09:25:39.223 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T09:25:39.223 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.223 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.223 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T09:25:39.223 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T09:25:39.223 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T09:25:39.223 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.223 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T09:25:39.223 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T09:25:39.224 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T09:25:39.224 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T09:25:39.224 INFO:teuthology.orchestra.run.smithi163.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T09:25:39.224 INFO:teuthology.orchestra.run.smithi163.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T09:25:39.224 INFO:teuthology.orchestra.run.smithi163.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T09:25:39.224 INFO:teuthology.orchestra.run.smithi163.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-27T09:25:39.224 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T09:25:39.224 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T09:25:39.224 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T09:25:39.225 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T09:25:39.225 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-27T09:25:39.225 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-27T09:25:39.225 INFO:teuthology.orchestra.run.smithi163.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-27T09:25:39.225 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.225 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.225 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:25:39.225 INFO:teuthology.orchestra.run.smithi163.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T09:25:39.225 INFO:teuthology.orchestra.run.smithi163.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-27T09:25:39.225 INFO:teuthology.orchestra.run.smithi163.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T09:25:39.225 INFO:teuthology.orchestra.run.smithi163.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-27T09:25:39.225 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-27T09:25:39.225 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T09:25:39.226 INFO:teuthology.orchestra.run.smithi163.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-27T09:25:39.226 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:25:39.226 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-05-27T09:25:39.649 DEBUG:teuthology.parallel:result is None 2024-05-27T09:25:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3868.ga6507388.el9.x86_64 111/170 2024-05-27T09:25:46.653 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /sys 2024-05-27T09:25:46.653 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /proc 2024-05-27T09:25:46.653 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /mnt 2024-05-27T09:25:46.653 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /var/tmp 2024-05-27T09:25:46.653 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /home 2024-05-27T09:25:46.653 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /root 2024-05-27T09:25:46.653 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /tmp 2024-05-27T09:25:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:25:46.893 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-volume-2:19.0.0-3868.ga6507388.el9.noarch 112/170 2024-05-27T09:25:46.942 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-volume-2:19.0.0-3868.ga6507388.el9.noarch 112/170 2024-05-27T09:25:46.942 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:25:46.942 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-27T09:25:46.943 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:25:48.723 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-osd-2:19.0.0-3868.ga6507388.el9.x86_64 113/170 2024-05-27T09:25:48.802 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-osd-2:19.0.0-3868.ga6507388.el9.x86_64 113/170 2024-05-27T09:25:48.803 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:25:48.803 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T09:25:48.803 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T09:25:48.803 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T09:25:48.803 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:25:49.058 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mds-2:19.0.0-3868.ga6507388.el9.x86_64 114/170 2024-05-27T09:25:49.142 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mds-2:19.0.0-3868.ga6507388.el9.x86_64 114/170 2024-05-27T09:25:49.142 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:25:49.142 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T09:25:49.142 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T09:25:49.142 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T09:25:49.142 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:25:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mon-2:19.0.0-3868.ga6507388.el9.x86_64 115/170 2024-05-27T09:25:49.731 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mon-2:19.0.0-3868.ga6507388.el9.x86_64 115/170 2024-05-27T09:25:49.731 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:25:49.731 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T09:25:49.731 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T09:25:49.731 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T09:25:49.731 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:25:49.865 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-27T09:25:49.926 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-27T09:25:50.007 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-27T09:25:50.244 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-27T09:25:50.311 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-27T09:25:50.408 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-27T09:25:50.513 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-27T09:25:50.686 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-27T09:25:50.744 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-27T09:25:50.796 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-27T09:25:50.826 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-27T09:25:51.483 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephadm-2:19.0.0-3868.ga6507388.el9.noarch 127/170 2024-05-27T09:25:51.529 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephadm-2:19.0.0-3868.ga6507388.el9.noarch 127/170 2024-05-27T09:25:51.603 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3868.ga6507388.e 128/170 2024-05-27T09:25:51.715 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3868.ga6507388. 129/170 2024-05-27T09:25:51.747 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3868.ga6507388.el9.noa 130/170 2024-05-27T09:25:53.059 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3868.ga6507388.el9.noa 130/170 2024-05-27T09:25:53.102 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3868.ga6507388.el9.n 131/170 2024-05-27T09:25:54.205 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3868.ga6507388.el9.n 131/170 2024-05-27T09:25:54.243 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3868.ga65 132/170 2024-05-27T09:25:54.400 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3868.ga65 132/170 2024-05-27T09:25:54.598 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3868.ga6507388.el 133/170 2024-05-27T09:25:54.631 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-2:19.0.0-3868.ga6507388.el9.x86_64 134/170 2024-05-27T09:25:54.700 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3868.ga6507388.el9.x86_64 134/170 2024-05-27T09:25:54.700 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:25:54.700 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T09:25:54.700 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T09:25:54.700 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T09:25:54.700 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:25:54.757 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-rook-2:19.0.0-3868.ga6507388.el9.noarch 135/170 2024-05-27T09:25:54.794 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3868.ga6507388.el9.noarch 135/170 2024-05-27T09:25:54.908 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-2:19.0.0-3868.ga6507388.el9.x86_64 136/170 2024-05-27T09:25:54.974 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 137/170 2024-05-27T09:25:56.002 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-27T09:25:56.034 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-radosgw-2:19.0.0-3868.ga6507388.el9.x86_64 139/170 2024-05-27T09:25:56.104 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3868.ga6507388.el9.x86_64 139/170 2024-05-27T09:25:56.104 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:25:56.104 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T09:25:56.105 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T09:25:56.105 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T09:25:56.105 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:25:56.151 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3868.ga6507 140/170 2024-05-27T09:25:56.222 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3868.ga6507 140/170 2024-05-27T09:25:56.222 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:25:56.222 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T09:25:56.223 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:25:56.302 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephfs-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 141/170 2024-05-27T09:25:56.375 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 141/170 2024-05-27T09:25:56.376 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:25:56.376 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-27T09:25:56.376 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T09:25:56.376 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T09:25:56.376 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:25:56.744 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 142/170 2024-05-27T09:25:56.811 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 142/170 2024-05-27T09:25:56.811 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:25:56.811 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T09:25:56.811 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T09:25:56.811 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T09:25:56.811 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:01.476 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-test-2:19.0.0-3868.ga6507388.el9.x86_64 143/170 2024-05-27T09:26:01.539 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 144/170 2024-05-27T09:26:01.594 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-nbd-2:19.0.0-3868.ga6507388.el9.x86_64 145/170 2024-05-27T09:26:01.632 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-27T09:26:01.672 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-27T09:26:01.969 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephfs-shell-2:19.0.0-3868.ga6507388.el9.x86_64 148/170 2024-05-27T09:26:02.013 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-27T09:26:02.046 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs-devel-2:19.0.0-3868.ga6507388.el9.x86_ 150/170 2024-05-27T09:26:02.088 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-27T09:26:02.150 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-27T09:26:02.204 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T09:26:03.105 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T09:26:03.149 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T09:26:03.206 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T09:26:04.538 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephfs-top-2:19.0.0-3868.ga6507388.el9.noarch 155/170 2024-05-27T09:26:04.647 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-27T09:26:04.707 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-27T09:26:04.839 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-27T09:26:04.874 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-27T09:26:04.907 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T09:26:04.942 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-27T09:26:05.013 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-27T09:26:05.014 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T09:26:05.166 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T09:26:05.212 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-27T09:26:05.213 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T09:26:05.349 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T09:26:05.388 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-27T09:26:05.421 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T09:26:05.421 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T09:26:05.759 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T09:26:05.759 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T09:26:05.896 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T09:26:05.966 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-27T09:26:06.086 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T09:26:06.335 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T09:26:06.335 INFO:teuthology.orchestra.run.smithi114.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-27T09:26:06.335 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:12.053 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-2:19.0.0-3868.ga6507388.el9.x86_64 1/170 2024-05-27T09:26:12.053 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-base-2:19.0.0-3868.ga6507388.el9.x86_64 2/170 2024-05-27T09:26:12.053 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 3/170 2024-05-27T09:26:12.053 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 4/170 2024-05-27T09:26:12.053 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3868.ga6507 5/170 2024-05-27T09:26:12.053 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mds-2:19.0.0-3868.ga6507388.el9.x86_64 6/170 2024-05-27T09:26:12.054 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-2:19.0.0-3868.ga6507388.el9.x86_64 7/170 2024-05-27T09:26:12.054 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mon-2:19.0.0-3868.ga6507388.el9.x86_64 8/170 2024-05-27T09:26:12.054 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-osd-2:19.0.0-3868.ga6507388.el9.x86_64 9/170 2024-05-27T09:26:12.054 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-radosgw-2:19.0.0-3868.ga6507388.el9.x86_64 10/170 2024-05-27T09:26:12.054 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-selinux-2:19.0.0-3868.ga6507388.el9.x86_64 11/170 2024-05-27T09:26:12.054 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-test-2:19.0.0-3868.ga6507388.el9.x86_64 12/170 2024-05-27T09:26:12.054 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephfs-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 13/170 2024-05-27T09:26:12.054 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephfs-shell-2:19.0.0-3868.ga6507388.el9.x86_64 14/170 2024-05-27T09:26:12.054 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs-devel-2:19.0.0-3868.ga6507388.el9.x86_ 15/170 2024-05-27T09:26:12.054 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs2-2:19.0.0-3868.ga6507388.el9.x86_64 16/170 2024-05-27T09:26:12.054 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephsqlite-2:19.0.0-3868.ga6507388.el9.x86_64 17/170 2024-05-27T09:26:12.055 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados-devel-2:19.0.0-3868.ga6507388.el9.x86_6 18/170 2024-05-27T09:26:12.055 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libradosstriper1-2:19.0.0-3868.ga6507388.el9.x86 19/170 2024-05-27T09:26:12.055 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librgw2-2:19.0.0-3868.ga6507388.el9.x86_64 20/170 2024-05-27T09:26:12.055 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3868.ga6507388.el 21/170 2024-05-27T09:26:12.055 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-common-2:19.0.0-3868.ga6507388.el9. 22/170 2024-05-27T09:26:12.055 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cephfs-2:19.0.0-3868.ga6507388.el9.x86_6 23/170 2024-05-27T09:26:12.055 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rados-2:19.0.0-3868.ga6507388.el9.x86_64 24/170 2024-05-27T09:26:12.055 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rbd-2:19.0.0-3868.ga6507388.el9.x86_64 25/170 2024-05-27T09:26:12.055 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rgw-2:19.0.0-3868.ga6507388.el9.x86_64 26/170 2024-05-27T09:26:12.055 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 27/170 2024-05-27T09:26:12.055 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 28/170 2024-05-27T09:26:12.055 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-nbd-2:19.0.0-3868.ga6507388.el9.x86_64 29/170 2024-05-27T09:26:12.055 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3868.ga6507388. 30/170 2024-05-27T09:26:12.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3868.ga6507388.el9.noa 31/170 2024-05-27T09:26:12.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3868.ga6507388.el9.n 32/170 2024-05-27T09:26:12.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3868.ga65 33/170 2024-05-27T09:26:12.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3868.ga6507388.el 34/170 2024-05-27T09:26:12.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3868.ga6507388.el9.noarch 35/170 2024-05-27T09:26:12.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3868.ga6507388.e 36/170 2024-05-27T09:26:12.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-volume-2:19.0.0-3868.ga6507388.el9.noarch 37/170 2024-05-27T09:26:12.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephadm-2:19.0.0-3868.ga6507388.el9.noarch 38/170 2024-05-27T09:26:12.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephfs-top-2:19.0.0-3868.ga6507388.el9.noarch 39/170 2024-05-27T09:26:12.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-27T09:26:12.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-27T09:26:12.056 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-27T09:26:12.057 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-27T09:26:12.057 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-27T09:26:12.057 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-27T09:26:12.057 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-27T09:26:12.057 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-27T09:26:12.057 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-27T09:26:12.057 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-27T09:26:12.057 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-27T09:26:12.057 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-27T09:26:12.057 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-27T09:26:12.057 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-27T09:26:12.057 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-27T09:26:12.058 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-27T09:26:12.058 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-27T09:26:12.058 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-27T09:26:12.058 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-27T09:26:12.058 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-27T09:26:12.058 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-27T09:26:12.058 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-27T09:26:12.058 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-27T09:26:12.058 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-27T09:26:12.058 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-27T09:26:12.058 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-27T09:26:12.059 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-27T09:26:12.059 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-27T09:26:12.059 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-27T09:26:12.059 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-27T09:26:12.059 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-27T09:26:12.059 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-27T09:26:12.059 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-27T09:26:12.059 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-27T09:26:12.059 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-27T09:26:12.060 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-27T09:26:12.060 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-27T09:26:12.060 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-27T09:26:12.060 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-27T09:26:12.060 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-27T09:26:12.060 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-27T09:26:12.060 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-27T09:26:12.060 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-27T09:26:12.060 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-27T09:26:12.061 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-27T09:26:12.061 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-27T09:26:12.061 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-27T09:26:12.061 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-27T09:26:12.061 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-27T09:26:12.061 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-27T09:26:12.061 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-27T09:26:12.061 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-27T09:26:12.061 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-27T09:26:12.061 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-27T09:26:12.061 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-27T09:26:12.061 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-27T09:26:12.061 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-27T09:26:12.062 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-27T09:26:12.062 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-27T09:26:12.062 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-27T09:26:12.062 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-27T09:26:12.062 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-27T09:26:12.062 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-27T09:26:12.062 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-27T09:26:12.062 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-27T09:26:12.062 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-27T09:26:12.062 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-27T09:26:12.062 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-27T09:26:12.062 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-27T09:26:12.063 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-27T09:26:12.063 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-27T09:26:12.063 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-27T09:26:12.063 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-27T09:26:12.063 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-27T09:26:12.063 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-27T09:26:12.063 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-27T09:26:12.063 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-27T09:26:12.063 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-27T09:26:12.063 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-27T09:26:12.063 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-27T09:26:12.064 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-27T09:26:12.064 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-27T09:26:12.064 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-27T09:26:12.064 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-27T09:26:12.064 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-27T09:26:12.064 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-27T09:26:12.064 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-27T09:26:12.064 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-27T09:26:12.064 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-27T09:26:12.064 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-27T09:26:12.064 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-27T09:26:12.064 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-27T09:26:12.064 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-27T09:26:12.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-27T09:26:12.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-27T09:26:12.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-27T09:26:12.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-27T09:26:12.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-27T09:26:12.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-27T09:26:12.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-27T09:26:12.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-27T09:26:12.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-27T09:26:12.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-27T09:26:12.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-27T09:26:12.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-27T09:26:12.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-27T09:26:12.065 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-27T09:26:12.066 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-27T09:26:12.066 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-27T09:26:12.066 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:19.0.0-3868.ga6507388.el9.x86_64 149/170 2024-05-27T09:26:12.066 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-27T09:26:12.066 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:19.0.0-3868.ga6507388.el9.x86_64 151/170 2024-05-27T09:26:12.066 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-27T09:26:12.066 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T09:26:12.066 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-27T09:26:12.066 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-27T09:26:12.066 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-27T09:26:12.066 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-27T09:26:12.066 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-27T09:26:12.066 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-27T09:26:12.067 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T09:26:12.067 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-27T09:26:12.067 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-27T09:26:12.067 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-27T09:26:12.067 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-27T09:26:12.067 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-27T09:26:12.067 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-27T09:26:12.067 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-27T09:26:12.067 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-27T09:26:12.067 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-27T09:26:14.272 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-27T09:26:14.272 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:14.273 INFO:teuthology.orchestra.run.smithi114.stdout:Upgraded: 2024-05-27T09:26:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-27T09:26:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-27T09:26:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-27T09:26:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-27T09:26:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-27T09:26:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-27T09:26:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-27T09:26:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: librados2-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-27T09:26:14.273 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-27T09:26:14.274 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-05-27T09:26:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-27T09:26:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: automake-1.16.2-8.el9.noarch 2024-05-27T09:26:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: bison-3.7.4-5.el9.x86_64 2024-05-27T09:26:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T09:26:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T09:26:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.274 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T09:26:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T09:26:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T09:26:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T09:26:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T09:26:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T09:26:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.275 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-volume-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T09:26:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T09:26:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-shell-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-top-2:19.0.0-3868.ga6507388.el9.noarch 2024-05-27T09:26:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-27T09:26:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: flex-2.6.4-9.el9.x86_64 2024-05-27T09:26:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T09:26:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T09:26:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T09:26:14.276 INFO:teuthology.orchestra.run.smithi114.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-27T09:26:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T09:26:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-27T09:26:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-27T09:26:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T09:26:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T09:26:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T09:26:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T09:26:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T09:26:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T09:26:14.277 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T09:26:14.278 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.278 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.278 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T09:26:14.278 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.278 INFO:teuthology.orchestra.run.smithi114.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-27T09:26:14.278 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T09:26:14.278 INFO:teuthology.orchestra.run.smithi114.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-27T09:26:14.279 INFO:teuthology.orchestra.run.smithi114.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-27T09:26:14.279 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T09:26:14.279 INFO:teuthology.orchestra.run.smithi114.stdout: lua-5.4.4-4.el9.x86_64 2024-05-27T09:26:14.279 INFO:teuthology.orchestra.run.smithi114.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-27T09:26:14.279 INFO:teuthology.orchestra.run.smithi114.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-27T09:26:14.279 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T09:26:14.279 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T09:26:14.279 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-27T09:26:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T09:26:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-13.14-1.el9.x86_64 2024-05-27T09:26:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-27T09:26:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-27T09:26:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-27T09:26:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-27T09:26:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-27T09:26:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T09:26:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-27T09:26:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T09:26:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T09:26:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T09:26:14.280 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T09:26:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T09:26:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T09:26:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T09:26:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-27T09:26:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-27T09:26:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T09:26:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T09:26:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T09:26:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-27T09:26:14.281 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-27T09:26:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T09:26:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T09:26:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T09:26:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T09:26:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T09:26:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T09:26:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T09:26:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T09:26:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T09:26:14.282 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T09:26:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T09:26:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T09:26:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T09:26:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T09:26:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T09:26:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T09:26:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T09:26:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-27T09:26:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T09:26:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T09:26:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T09:26:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T09:26:14.283 INFO:teuthology.orchestra.run.smithi114.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-27T09:26:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T09:26:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T09:26:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T09:26:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T09:26:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-27T09:26:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T09:26:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T09:26:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T09:26:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T09:26:14.284 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T09:26:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T09:26:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T09:26:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T09:26:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T09:26:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T09:26:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T09:26:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-27T09:26:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T09:26:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T09:26:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T09:26:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T09:26:14.285 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-27T09:26:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-27T09:26:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-27T09:26:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd-2:19.0.0-3868.ga6507388.el9.x86_64 2024-05-27T09:26:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T09:26:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-27T09:26:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T09:26:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-27T09:26:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-27T09:26:14.286 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T09:26:14.287 INFO:teuthology.orchestra.run.smithi114.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-27T09:26:14.287 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:14.287 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-05-27T09:26:14.934 DEBUG:teuthology.parallel:result is None 2024-05-27T09:26:14.934 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:26:15.162 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T09:26:15.221 INFO:teuthology.orchestra.run.smithi114.stdout:19.0.0-3868.ga6507388.el9 2024-05-27T09:26:15.221 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3868.ga6507388.el9 2024-05-27T09:26:15.221 INFO:teuthology.task.install:The correct ceph version 19.0.0-3868.ga6507388 is installed. 2024-05-27T09:26:15.223 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:26:15.448 DEBUG:teuthology.orchestra.run.smithi163:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T09:26:15.474 INFO:teuthology.orchestra.run.smithi163.stdout:19.0.0-3868.ga6507388.el9 2024-05-27T09:26:15.474 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3868.ga6507388.el9 2024-05-27T09:26:15.474 INFO:teuthology.task.install:The correct ceph version 19.0.0-3868.ga6507388 is installed. 2024-05-27T09:26:15.476 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-27T09:26:15.476 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:26:15.476 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T09:26:15.543 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-27T09:26:15.543 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T09:26:15.574 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-27T09:26:15.574 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:26:15.575 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T09:26:15.651 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T09:26:15.755 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-27T09:26:15.755 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T09:26:15.784 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T09:26:15.853 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-27T09:26:15.853 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:26:15.853 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T09:26:15.919 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T09:26:16.034 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-27T09:26:16.034 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T09:26:16.059 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T09:26:16.123 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-27T09:26:16.123 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:26:16.123 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T09:26:16.191 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T09:26:16.297 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-27T09:26:16.297 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T09:26:16.321 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T09:26:16.385 INFO:teuthology.run_tasks:Running task ceph... 2024-05-27T09:26:16.463 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-27T09:26:16.463 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 777 /var/log/ceph 2024-05-27T09:26:16.468 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 777 /var/log/ceph 2024-05-27T09:26:16.527 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-27T09:26:16.527 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-27T09:26:16.531 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-27T09:26:16.587 INFO:tasks.ceph:Creating extra log directories... 2024-05-27T09:26:16.587 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-27T09:26:16.591 DEBUG:teuthology.orchestra.run.smithi163:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-27T09:26:16.655 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-27T09:26:16.655 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-27T09:26:16.655 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-27T09:26:16.656 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:26:16.656 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-27T09:26:16.694 DEBUG:teuthology.orchestra.run.smithi114:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-27T09:26:16.886 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-27T09:26:16.951 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-27T09:26:16.951 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-27T09:26:16.965 DEBUG:teuthology.orchestra.run.smithi163:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-27T09:26:17.072 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-27T09:26:17.123 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-27T09:26:17.123 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'Replacing daemon mds'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-27T09:26:17.123 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727681', 'branch': 'wip-vshankar-testing-20240521.105740-debug', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7727681', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-20240521.105740-debug'}, 'ceph': {'cephfs': {'max_mds': 3, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'Replacing daemon mds'], 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20240521.105740-debug', 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 2274, 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d', 'sleep_before_teardown': 0, 'subset': '442/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20240521.105740-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi114.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKf/+L1STYw31rZGPcAo5FwUu6vxD7GBLONXQiG9T5rWTrlECbcLiomfEzVFwCNwwVsVZ+tmOWYoL5g+DnNAhC4=', 'smithi163.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDHRf4tY2GXdVG2BTaLwqt8rpW1NZRe+/anqgZY1DI6ZTtB5MtdtrycJNc0mmZLgb/s5FNAdI8MfkNhPT5JOPJ4='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'Replacing daemon mds'], 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'mds_thrash': None}, {'workunit': {'clients': {'all': ['fs/snaps']}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e7ae7580c417d6ab27fda98fecadca5d9f85d63e', 'timestamp': '2024-05-27_06:46:53', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841'} 2024-05-27T09:26:17.124 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-27T09:26:17.129 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-27T09:26:17.170 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m0777 -- /var/run/ceph 2024-05-27T09:26:17.212 DEBUG:teuthology.orchestra.run.smithi163:> sudo install -d -m0777 -- /var/run/ceph 2024-05-27T09:26:17.268 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:26:17.268 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T09:26:17.306 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T09:26:17.307 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_1 2024-05-27T09:26:17.385 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T09:26:17.385 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T09:26:17.385 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-05-27T09:26:17.385 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T09:26:17.385 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T09:26:17.385 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-05-27 09:26:06.768569644 +0000 2024-05-27T09:26:17.385 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-05-27 09:23:02.552409351 +0000 2024-05-27T09:26:17.385 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-05-27 09:23:02.552409351 +0000 2024-05-27T09:26:17.385 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-05-27 09:23:02.552409351 +0000 2024-05-27T09:26:17.386 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T09:26:17.480 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-05-27T09:26:17.480 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-05-27T09:26:17.480 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000369751 s, 1.4 MB/s 2024-05-27T09:26:17.483 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T09:26:17.522 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_2 2024-05-27T09:26:17.607 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T09:26:17.608 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T09:26:17.608 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-05-27T09:26:17.608 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T09:26:17.608 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T09:26:17.608 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-05-27 09:26:06.768569644 +0000 2024-05-27T09:26:17.608 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-05-27 09:23:03.130396341 +0000 2024-05-27T09:26:17.608 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-05-27 09:23:03.130396341 +0000 2024-05-27T09:26:17.608 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-05-27 09:23:03.130396341 +0000 2024-05-27T09:26:17.608 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T09:26:17.709 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-05-27T09:26:17.710 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-05-27T09:26:17.710 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000309813 s, 1.7 MB/s 2024-05-27T09:26:17.712 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T09:26:17.752 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_3 2024-05-27T09:26:17.835 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T09:26:17.835 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T09:26:17.835 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-05-27T09:26:17.835 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T09:26:17.835 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T09:26:17.836 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-05-27 09:26:06.769569622 +0000 2024-05-27T09:26:17.836 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-05-27 09:23:03.710383288 +0000 2024-05-27T09:26:17.836 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-05-27 09:23:03.710383288 +0000 2024-05-27T09:26:17.836 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-05-27 09:23:03.710383288 +0000 2024-05-27T09:26:17.836 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T09:26:17.943 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-05-27T09:26:17.943 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-05-27T09:26:17.944 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000319419 s, 1.6 MB/s 2024-05-27T09:26:17.946 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T09:26:17.993 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_4 2024-05-27T09:26:18.033 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T09:26:18.033 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T09:26:18.033 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-05-27T09:26:18.033 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T09:26:18.033 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T09:26:18.034 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-05-27 09:26:06.770569600 +0000 2024-05-27T09:26:18.034 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-05-27 09:23:04.300369855 +0000 2024-05-27T09:26:18.034 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-05-27 09:23:04.300369855 +0000 2024-05-27T09:26:18.034 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-05-27 09:23:04.300369855 +0000 2024-05-27T09:26:18.034 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T09:26:18.132 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-05-27T09:26:18.133 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-05-27T09:26:18.133 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000295188 s, 1.7 MB/s 2024-05-27T09:26:18.135 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T09:26:18.173 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-27T09:26:18.173 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-27T09:26:18.173 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T09:26:18.191 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T09:26:18.191 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_1 2024-05-27T09:26:18.249 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T09:26:18.249 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T09:26:18.249 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-27T09:26:18.249 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T09:26:18.249 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T09:26:18.249 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-05-27 09:25:31.274483525 +0000 2024-05-27T09:26:18.249 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-05-27 09:21:27.805118175 +0000 2024-05-27T09:26:18.250 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-05-27 09:21:27.805118175 +0000 2024-05-27T09:26:18.250 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-05-27 09:21:27.805118175 +0000 2024-05-27T09:26:18.250 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T09:26:18.314 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-05-27T09:26:18.315 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-05-27T09:26:18.315 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000221016 s, 2.3 MB/s 2024-05-27T09:26:18.316 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T09:26:18.374 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_2 2024-05-27T09:26:18.430 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T09:26:18.430 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T09:26:18.431 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-27T09:26:18.431 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T09:26:18.431 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T09:26:18.431 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-05-27 09:25:31.274483525 +0000 2024-05-27T09:26:18.431 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-05-27 09:21:28.140110290 +0000 2024-05-27T09:26:18.431 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-05-27 09:21:28.140110290 +0000 2024-05-27T09:26:18.431 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-05-27 09:21:28.140110290 +0000 2024-05-27T09:26:18.431 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T09:26:18.494 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-05-27T09:26:18.494 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-05-27T09:26:18.494 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000170582 s, 3.0 MB/s 2024-05-27T09:26:18.496 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T09:26:18.554 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_3 2024-05-27T09:26:18.610 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T09:26:18.610 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T09:26:18.610 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-05-27T09:26:18.610 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T09:26:18.610 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T09:26:18.610 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-05-27 09:25:31.274483525 +0000 2024-05-27T09:26:18.610 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-05-27 09:21:28.476102381 +0000 2024-05-27T09:26:18.611 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-05-27 09:21:28.476102381 +0000 2024-05-27T09:26:18.611 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-05-27 09:21:28.476102381 +0000 2024-05-27T09:26:18.611 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T09:26:18.674 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-05-27T09:26:18.674 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-05-27T09:26:18.675 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000169449 s, 3.0 MB/s 2024-05-27T09:26:18.676 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T09:26:18.733 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_4 2024-05-27T09:26:18.789 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T09:26:18.790 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T09:26:18.790 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-05-27T09:26:18.790 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T09:26:18.790 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T09:26:18.790 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-05-27 09:25:31.275483502 +0000 2024-05-27T09:26:18.790 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-05-27 09:21:28.796094849 +0000 2024-05-27T09:26:18.790 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-05-27 09:21:28.796094849 +0000 2024-05-27T09:26:18.790 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-05-27 09:21:28.796094849 +0000 2024-05-27T09:26:18.790 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T09:26:18.852 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-05-27T09:26:18.853 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-05-27T09:26:18.853 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000234467 s, 2.2 MB/s 2024-05-27T09:26:18.854 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T09:26:18.910 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-27T09:26:18.911 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi114.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi163.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-27T09:26:18.911 INFO:tasks.ceph:Generating config... 2024-05-27T09:26:18.912 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-27T09:26:18.912 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-05-27T09:26:18.912 INFO:tasks.ceph:[client] debug client = 20 2024-05-27T09:26:18.912 INFO:tasks.ceph:[client] debug ms = 1 2024-05-27T09:26:18.913 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-27T09:26:18.913 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-27T09:26:18.913 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-05-27T09:26:18.913 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-27T09:26:18.913 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-27T09:26:18.913 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-27T09:26:18.913 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-05-27T09:26:18.913 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-05-27T09:26:18.913 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-05-27T09:26:18.913 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-05-27T09:26:18.913 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-27T09:26:18.913 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-27T09:26:18.913 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2024-05-27T09:26:18.914 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-27T09:26:18.914 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-27T09:26:18.914 INFO:tasks.ceph:[mds] mds_dir_prefetch = True 2024-05-27T09:26:18.914 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-05-27T09:26:18.914 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-27T09:26:18.914 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-27T09:26:18.914 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-27T09:26:18.914 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-27T09:26:18.914 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-27T09:26:18.914 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-27T09:26:18.914 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-27T09:26:18.914 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-27T09:26:18.914 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-27T09:26:18.915 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-27T09:26:18.915 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-27T09:26:18.915 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-27T09:26:18.915 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-05-27T09:26:18.915 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-27T09:26:18.915 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-05-27T09:26:18.915 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-05-27T09:26:18.915 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-27T09:26:18.915 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-27T09:26:18.915 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-05-27T09:26:18.915 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-27T09:26:18.916 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-27T09:26:18.916 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-27T09:26:18.916 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-05-27T09:26:18.916 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-27T09:26:18.916 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-05-27T09:26:18.916 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-27T09:26:18.916 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-27T09:26:18.916 INFO:tasks.ceph:Setting up mon.a... 2024-05-27T09:26:18.916 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-27T09:26:19.087 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/ceph.keyring 2024-05-27T09:26:19.103 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-27T09:26:19.271 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-27T09:26:19.349 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.114'), ('mon.b', '172.21.15.163'), ('mon.c', '[v2:172.21.15.163:3301,v1:172.21.15.163:6790]')] 2024-05-27T09:26:19.349 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.114,172.21.15.163,[v2:172.21.15.163:3301,v1:172.21.15.163:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-27T09:26:19.350 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:26:19.350 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-27T09:26:19.399 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.114 --add b 172.21.15.163 --addv c '[v2:172.21.15.163:3301,v1:172.21.15.163:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-27T09:26:19.622 INFO:teuthology.orchestra.run.smithi114.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-27T09:26:19.622 INFO:teuthology.orchestra.run.smithi114.stdout:monmaptool: generated fsid 2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48 2024-05-27T09:26:19.622 INFO:teuthology.orchestra.run.smithi114.stdout:setting min_mon_release = quincy 2024-05-27T09:26:19.622 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 0 2024-05-27T09:26:19.623 INFO:teuthology.orchestra.run.smithi114.stdout:fsid 2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48 2024-05-27T09:26:19.623 INFO:teuthology.orchestra.run.smithi114.stdout:last_changed 2024-05-27T09:26:19.622526+0000 2024-05-27T09:26:19.623 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-05-27T09:26:19.622526+0000 2024-05-27T09:26:19.623 INFO:teuthology.orchestra.run.smithi114.stdout:min_mon_release 17 (quincy) 2024-05-27T09:26:19.623 INFO:teuthology.orchestra.run.smithi114.stdout:election_strategy: 1 2024-05-27T09:26:19.623 INFO:teuthology.orchestra.run.smithi114.stdout:0: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.a 2024-05-27T09:26:19.623 INFO:teuthology.orchestra.run.smithi114.stdout:1: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.b 2024-05-27T09:26:19.623 INFO:teuthology.orchestra.run.smithi114.stdout:2: [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] mon.c 2024-05-27T09:26:19.623 INFO:teuthology.orchestra.run.smithi114.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-27T09:26:19.631 DEBUG:teuthology.orchestra.run.smithi114:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-27T09:26:19.679 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48... 2024-05-27T09:26:19.680 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-27T09:26:19.684 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-27T09:26:19.731 INFO:teuthology.orchestra.run.smithi163.stdout:[global] 2024-05-27T09:26:19.732 INFO:teuthology.orchestra.run.smithi163.stdout: chdir = "" 2024-05-27T09:26:19.732 INFO:teuthology.orchestra.run.smithi163.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-27T09:26:19.732 INFO:teuthology.orchestra.run.smithi163.stdout: auth supported = cephx 2024-05-27T09:26:19.732 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.732 INFO:teuthology.orchestra.run.smithi163.stdout: filestore xattr use omap = true 2024-05-27T09:26:19.732 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.732 INFO:teuthology.orchestra.run.smithi163.stdout: mon clock drift allowed = 1.000 2024-05-27T09:26:19.732 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.732 INFO:teuthology.orchestra.run.smithi163.stdout: osd crush chooseleaf type = 0 2024-05-27T09:26:19.732 INFO:teuthology.orchestra.run.smithi163.stdout: auth debug = true 2024-05-27T09:26:19.732 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.733 INFO:teuthology.orchestra.run.smithi163.stdout: ms die on old message = true 2024-05-27T09:26:19.733 INFO:teuthology.orchestra.run.smithi163.stdout: ms die on bug = true 2024-05-27T09:26:19.733 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.733 INFO:teuthology.orchestra.run.smithi163.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-27T09:26:19.733 INFO:teuthology.orchestra.run.smithi163.stdout: mon pg warn max object skew = 0 2024-05-27T09:26:19.733 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.733 INFO:teuthology.orchestra.run.smithi163.stdout: # disable pg_autoscaler by default for new pools 2024-05-27T09:26:19.733 INFO:teuthology.orchestra.run.smithi163.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-27T09:26:19.733 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.733 INFO:teuthology.orchestra.run.smithi163.stdout: osd pool default size = 2 2024-05-27T09:26:19.734 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.734 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd allow primary affinity = true 2024-05-27T09:26:19.734 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd allow pg remap = true 2024-05-27T09:26:19.734 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on legacy crush tunables = false 2024-05-27T09:26:19.734 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on crush straw calc version zero = false 2024-05-27T09:26:19.734 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on no sortbitwise = false 2024-05-27T09:26:19.734 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on osd down out interval zero = false 2024-05-27T09:26:19.734 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on too few osds = false 2024-05-27T09:26:19.734 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-27T09:26:19.734 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-27T09:26:19.735 INFO:teuthology.orchestra.run.smithi163.stdout: mon_allow_pool_size_one = true 2024-05-27T09:26:19.735 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.735 INFO:teuthology.orchestra.run.smithi163.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-27T09:26:19.735 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.735 INFO:teuthology.orchestra.run.smithi163.stdout: osd default data pool replay window = 5 2024-05-27T09:26:19.735 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.735 INFO:teuthology.orchestra.run.smithi163.stdout: mon allow pool delete = true 2024-05-27T09:26:19.735 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.735 INFO:teuthology.orchestra.run.smithi163.stdout: mon cluster log file level = debug 2024-05-27T09:26:19.735 INFO:teuthology.orchestra.run.smithi163.stdout: debug asserts on shutdown = true 2024-05-27T09:26:19.735 INFO:teuthology.orchestra.run.smithi163.stdout: mon health detail to clog = false 2024-05-27T09:26:19.736 INFO:teuthology.orchestra.run.smithi163.stdout: mon host = "172.21.15.114,172.21.15.163,[v2:172.21.15.163:3301,v1:172.21.15.163:6790]" 2024-05-27T09:26:19.736 INFO:teuthology.orchestra.run.smithi163.stdout: ms die on skipped message = False 2024-05-27T09:26:19.736 INFO:teuthology.orchestra.run.smithi163.stdout: fsid = 2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48 2024-05-27T09:26:19.736 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.736 INFO:teuthology.orchestra.run.smithi163.stdout:[osd] 2024-05-27T09:26:19.736 INFO:teuthology.orchestra.run.smithi163.stdout: osd journal size = 100 2024-05-27T09:26:19.736 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.736 INFO:teuthology.orchestra.run.smithi163.stdout: osd scrub load threshold = 5.0 2024-05-27T09:26:19.736 INFO:teuthology.orchestra.run.smithi163.stdout: osd scrub max interval = 600 2024-05-27T09:26:19.736 INFO:teuthology.orchestra.run.smithi163.stdout: osd mclock profile = high_recovery_ops 2024-05-27T09:26:19.736 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.736 INFO:teuthology.orchestra.run.smithi163.stdout: osd recover clone overlap = true 2024-05-27T09:26:19.737 INFO:teuthology.orchestra.run.smithi163.stdout: osd recovery max chunk = 1048576 2024-05-27T09:26:19.737 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.737 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug shutdown = true 2024-05-27T09:26:19.737 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug op order = true 2024-05-27T09:26:19.737 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug verify stray on activate = true 2024-05-27T09:26:19.737 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug trim objects = true 2024-05-27T09:26:19.737 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.737 INFO:teuthology.orchestra.run.smithi163.stdout: osd open classes on start = true 2024-05-27T09:26:19.737 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug pg log writeout = true 2024-05-27T09:26:19.737 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.738 INFO:teuthology.orchestra.run.smithi163.stdout: osd deep scrub update digest min age = 30 2024-05-27T09:26:19.738 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.738 INFO:teuthology.orchestra.run.smithi163.stdout: osd map max advance = 10 2024-05-27T09:26:19.738 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.738 INFO:teuthology.orchestra.run.smithi163.stdout: journal zero on create = true 2024-05-27T09:26:19.738 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.738 INFO:teuthology.orchestra.run.smithi163.stdout: filestore ondisk finisher threads = 3 2024-05-27T09:26:19.738 INFO:teuthology.orchestra.run.smithi163.stdout: filestore apply finisher threads = 3 2024-05-27T09:26:19.739 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.739 INFO:teuthology.orchestra.run.smithi163.stdout: bdev debug aio = true 2024-05-27T09:26:19.739 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug misdirected ops = true 2024-05-27T09:26:19.739 INFO:teuthology.orchestra.run.smithi163.stdout: bluestore block size = 96636764160 2024-05-27T09:26:19.739 INFO:teuthology.orchestra.run.smithi163.stdout: bluestore compression mode = aggressive 2024-05-27T09:26:19.739 INFO:teuthology.orchestra.run.smithi163.stdout: bluestore fsck on mount = True 2024-05-27T09:26:19.739 INFO:teuthology.orchestra.run.smithi163.stdout: debug bluefs = 20 2024-05-27T09:26:19.739 INFO:teuthology.orchestra.run.smithi163.stdout: debug bluestore = 20 2024-05-27T09:26:19.739 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-05-27T09:26:19.739 INFO:teuthology.orchestra.run.smithi163.stdout: debug osd = 20 2024-05-27T09:26:19.739 INFO:teuthology.orchestra.run.smithi163.stdout: debug rocksdb = 10 2024-05-27T09:26:19.739 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd backfillfull_ratio = 0.85 2024-05-27T09:26:19.739 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd full ratio = 0.9 2024-05-27T09:26:19.740 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd nearfull ratio = 0.8 2024-05-27T09:26:19.740 INFO:teuthology.orchestra.run.smithi163.stdout: osd blocked scrub grace period = 3600 2024-05-27T09:26:19.740 INFO:teuthology.orchestra.run.smithi163.stdout: osd failsafe full ratio = 0.95 2024-05-27T09:26:19.740 INFO:teuthology.orchestra.run.smithi163.stdout: osd max markdown count = 1000 2024-05-27T09:26:19.740 INFO:teuthology.orchestra.run.smithi163.stdout: osd objectstore = bluestore 2024-05-27T09:26:19.740 INFO:teuthology.orchestra.run.smithi163.stdout: osd op complaint time = 180 2024-05-27T09:26:19.740 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.740 INFO:teuthology.orchestra.run.smithi163.stdout:[mgr] 2024-05-27T09:26:19.740 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-05-27T09:26:19.740 INFO:teuthology.orchestra.run.smithi163.stdout: debug mgr = 20 2024-05-27T09:26:19.740 INFO:teuthology.orchestra.run.smithi163.stdout: debug mon = 20 2024-05-27T09:26:19.741 INFO:teuthology.orchestra.run.smithi163.stdout: debug auth = 20 2024-05-27T09:26:19.741 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min pgs per osd = 4 2024-05-27T09:26:19.741 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min bytes per osd = 10 2024-05-27T09:26:19.741 INFO:teuthology.orchestra.run.smithi163.stdout: mgr/telemetry/nag = false 2024-05-27T09:26:19.741 INFO:teuthology.orchestra.run.smithi163.stdout: debug client = 20 2024-05-27T09:26:19.741 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.741 INFO:teuthology.orchestra.run.smithi163.stdout:[mon] 2024-05-27T09:26:19.741 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-05-27T09:26:19.741 INFO:teuthology.orchestra.run.smithi163.stdout: debug mon = 20 2024-05-27T09:26:19.741 INFO:teuthology.orchestra.run.smithi163.stdout: debug paxos = 20 2024-05-27T09:26:19.741 INFO:teuthology.orchestra.run.smithi163.stdout: debug auth = 20 2024-05-27T09:26:19.742 INFO:teuthology.orchestra.run.smithi163.stdout: mon data avail warn = 5 2024-05-27T09:26:19.742 INFO:teuthology.orchestra.run.smithi163.stdout: mon mgr mkfs grace = 240 2024-05-27T09:26:19.742 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min pgs per osd = 4 2024-05-27T09:26:19.742 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd reporter subtree level = osd 2024-05-27T09:26:19.742 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd prime pg temp = true 2024-05-27T09:26:19.742 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min bytes per osd = 10 2024-05-27T09:26:19.742 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.742 INFO:teuthology.orchestra.run.smithi163.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-27T09:26:19.742 INFO:teuthology.orchestra.run.smithi163.stdout: auth mon ticket ttl = 660 # 11m 2024-05-27T09:26:19.742 INFO:teuthology.orchestra.run.smithi163.stdout: auth service ticket ttl = 240 # 4m 2024-05-27T09:26:19.743 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.743 INFO:teuthology.orchestra.run.smithi163.stdout: # don't complain about insecure global_id in the test suite 2024-05-27T09:26:19.743 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-27T09:26:19.743 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-27T09:26:19.743 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.743 INFO:teuthology.orchestra.run.smithi163.stdout: # 1m isn't quite enough 2024-05-27T09:26:19.743 INFO:teuthology.orchestra.run.smithi163.stdout: mon_down_mkfs_grace = 2m 2024-05-27T09:26:19.743 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.743 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_filestore_osds = false 2024-05-27T09:26:19.743 INFO:teuthology.orchestra.run.smithi163.stdout: mon down mkfs grace = 300 2024-05-27T09:26:19.743 INFO:teuthology.orchestra.run.smithi163.stdout: mon op complaint time = 120 2024-05-27T09:26:19.743 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-05-27T09:26:19.744 INFO:teuthology.orchestra.run.smithi163.stdout:[client] 2024-05-27T09:26:19.744 INFO:teuthology.orchestra.run.smithi163.stdout: rgw cache enabled = true 2024-05-27T09:26:19.744 INFO:teuthology.orchestra.run.smithi163.stdout: rgw enable ops log = true 2024-05-27T09:26:19.744 INFO:teuthology.orchestra.run.smithi163.stdout: rgw enable usage log = true 2024-05-27T09:26:19.744 INFO:teuthology.orchestra.run.smithi163.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-27T09:26:19.744 INFO:teuthology.orchestra.run.smithi163.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-27T09:26:19.744 INFO:teuthology.orchestra.run.smithi163.stdout: client mount timeout = 600 2024-05-27T09:26:19.744 INFO:teuthology.orchestra.run.smithi163.stdout: client_shutdown_timeout = 120 2024-05-27T09:26:19.744 INFO:teuthology.orchestra.run.smithi163.stdout: debug client = 20 2024-05-27T09:26:19.744 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-05-27T09:26:19.744 INFO:teuthology.orchestra.run.smithi163.stdout: rados mon op timeout = 900 2024-05-27T09:26:19.744 INFO:teuthology.orchestra.run.smithi163.stdout: rados osd op timeout = 900 2024-05-27T09:26:19.744 INFO:teuthology.orchestra.run.smithi163.stdout:[mon.a] 2024-05-27T09:26:19.744 INFO:teuthology.orchestra.run.smithi163.stdout:[mon.b] 2024-05-27T09:26:19.744 INFO:teuthology.orchestra.run.smithi163.stdout:[mon.c] 2024-05-27T09:26:19.745 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.a] 2024-05-27T09:26:19.745 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.c] 2024-05-27T09:26:19.745 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.e] 2024-05-27T09:26:19.745 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.b] 2024-05-27T09:26:19.745 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.d] 2024-05-27T09:26:19.745 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.f] 2024-05-27T09:26:19.745 INFO:teuthology.orchestra.run.smithi163.stdout:[mds] 2024-05-27T09:26:19.745 INFO:teuthology.orchestra.run.smithi163.stdout: debug mds = 20 2024-05-27T09:26:19.745 INFO:teuthology.orchestra.run.smithi163.stdout: debug mds balancer = 20 2024-05-27T09:26:19.746 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-05-27T09:26:19.746 INFO:teuthology.orchestra.run.smithi163.stdout: mds bal fragment size max = 10000 2024-05-27T09:26:19.746 INFO:teuthology.orchestra.run.smithi163.stdout: mds bal merge size = 5 2024-05-27T09:26:19.746 INFO:teuthology.orchestra.run.smithi163.stdout: mds bal split bits = 3 2024-05-27T09:26:19.746 INFO:teuthology.orchestra.run.smithi163.stdout: mds bal split size = 100 2024-05-27T09:26:19.746 INFO:teuthology.orchestra.run.smithi163.stdout: mds debug frag = True 2024-05-27T09:26:19.746 INFO:teuthology.orchestra.run.smithi163.stdout: mds debug scatterstat = True 2024-05-27T09:26:19.746 INFO:teuthology.orchestra.run.smithi163.stdout: mds oft prefetch dirfrags = True 2024-05-27T09:26:19.746 INFO:teuthology.orchestra.run.smithi163.stdout: mds op complaint time = 180 2024-05-27T09:26:19.746 INFO:teuthology.orchestra.run.smithi163.stdout: mds verify scatter = True 2024-05-27T09:26:19.746 INFO:teuthology.orchestra.run.smithi163.stdout: mds_dir_prefetch = True 2024-05-27T09:26:19.746 INFO:teuthology.orchestra.run.smithi163.stdout: mds_sleep_rank_change = 5000000.0 2024-05-27T09:26:19.746 INFO:teuthology.orchestra.run.smithi163.stdout: osd op complaint time = 180 2024-05-27T09:26:19.747 INFO:teuthology.orchestra.run.smithi163.stdout: rados mon op timeout = 900 2024-05-27T09:26:19.747 INFO:teuthology.orchestra.run.smithi163.stdout: rados osd op timeout = 900 2024-05-27T09:26:19.813 INFO:teuthology.orchestra.run.smithi114.stdout:[global] 2024-05-27T09:26:19.813 INFO:teuthology.orchestra.run.smithi114.stdout: chdir = "" 2024-05-27T09:26:19.813 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-27T09:26:19.813 INFO:teuthology.orchestra.run.smithi114.stdout: auth supported = cephx 2024-05-27T09:26:19.813 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.813 INFO:teuthology.orchestra.run.smithi114.stdout: filestore xattr use omap = true 2024-05-27T09:26:19.813 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.813 INFO:teuthology.orchestra.run.smithi114.stdout: mon clock drift allowed = 1.000 2024-05-27T09:26:19.814 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.814 INFO:teuthology.orchestra.run.smithi114.stdout: osd crush chooseleaf type = 0 2024-05-27T09:26:19.814 INFO:teuthology.orchestra.run.smithi114.stdout: auth debug = true 2024-05-27T09:26:19.814 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.814 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on old message = true 2024-05-27T09:26:19.814 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on bug = true 2024-05-27T09:26:19.814 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.814 INFO:teuthology.orchestra.run.smithi114.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-27T09:26:19.814 INFO:teuthology.orchestra.run.smithi114.stdout: mon pg warn max object skew = 0 2024-05-27T09:26:19.814 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.814 INFO:teuthology.orchestra.run.smithi114.stdout: # disable pg_autoscaler by default for new pools 2024-05-27T09:26:19.814 INFO:teuthology.orchestra.run.smithi114.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-27T09:26:19.814 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.815 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default size = 2 2024-05-27T09:26:19.815 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.815 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow primary affinity = true 2024-05-27T09:26:19.815 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow pg remap = true 2024-05-27T09:26:19.815 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on legacy crush tunables = false 2024-05-27T09:26:19.815 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on crush straw calc version zero = false 2024-05-27T09:26:19.815 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on no sortbitwise = false 2024-05-27T09:26:19.815 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on osd down out interval zero = false 2024-05-27T09:26:19.815 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on too few osds = false 2024-05-27T09:26:19.816 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-27T09:26:19.816 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-27T09:26:19.816 INFO:teuthology.orchestra.run.smithi114.stdout: mon_allow_pool_size_one = true 2024-05-27T09:26:19.816 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.816 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-27T09:26:19.816 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.816 INFO:teuthology.orchestra.run.smithi114.stdout: osd default data pool replay window = 5 2024-05-27T09:26:19.816 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.816 INFO:teuthology.orchestra.run.smithi114.stdout: mon allow pool delete = true 2024-05-27T09:26:19.816 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.817 INFO:teuthology.orchestra.run.smithi114.stdout: mon cluster log file level = debug 2024-05-27T09:26:19.817 INFO:teuthology.orchestra.run.smithi114.stdout: debug asserts on shutdown = true 2024-05-27T09:26:19.817 INFO:teuthology.orchestra.run.smithi114.stdout: mon health detail to clog = false 2024-05-27T09:26:19.817 INFO:teuthology.orchestra.run.smithi114.stdout: mon host = "172.21.15.114,172.21.15.163,[v2:172.21.15.163:3301,v1:172.21.15.163:6790]" 2024-05-27T09:26:19.817 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on skipped message = False 2024-05-27T09:26:19.818 INFO:teuthology.orchestra.run.smithi114.stdout: fsid = 2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48 2024-05-27T09:26:19.818 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.818 INFO:teuthology.orchestra.run.smithi114.stdout:[osd] 2024-05-27T09:26:19.818 INFO:teuthology.orchestra.run.smithi114.stdout: osd journal size = 100 2024-05-27T09:26:19.818 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.818 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub load threshold = 5.0 2024-05-27T09:26:19.818 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub max interval = 600 2024-05-27T09:26:19.818 INFO:teuthology.orchestra.run.smithi114.stdout: osd mclock profile = high_recovery_ops 2024-05-27T09:26:19.818 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.819 INFO:teuthology.orchestra.run.smithi114.stdout: osd recover clone overlap = true 2024-05-27T09:26:19.819 INFO:teuthology.orchestra.run.smithi114.stdout: osd recovery max chunk = 1048576 2024-05-27T09:26:19.819 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.819 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug shutdown = true 2024-05-27T09:26:19.819 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug op order = true 2024-05-27T09:26:19.819 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug verify stray on activate = true 2024-05-27T09:26:19.819 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug trim objects = true 2024-05-27T09:26:19.819 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.819 INFO:teuthology.orchestra.run.smithi114.stdout: osd open classes on start = true 2024-05-27T09:26:19.819 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug pg log writeout = true 2024-05-27T09:26:19.819 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.819 INFO:teuthology.orchestra.run.smithi114.stdout: osd deep scrub update digest min age = 30 2024-05-27T09:26:19.819 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.820 INFO:teuthology.orchestra.run.smithi114.stdout: osd map max advance = 10 2024-05-27T09:26:19.820 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.820 INFO:teuthology.orchestra.run.smithi114.stdout: journal zero on create = true 2024-05-27T09:26:19.820 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.820 INFO:teuthology.orchestra.run.smithi114.stdout: filestore ondisk finisher threads = 3 2024-05-27T09:26:19.820 INFO:teuthology.orchestra.run.smithi114.stdout: filestore apply finisher threads = 3 2024-05-27T09:26:19.820 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.820 INFO:teuthology.orchestra.run.smithi114.stdout: bdev debug aio = true 2024-05-27T09:26:19.820 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug misdirected ops = true 2024-05-27T09:26:19.821 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore block size = 96636764160 2024-05-27T09:26:19.821 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore compression mode = aggressive 2024-05-27T09:26:19.821 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore fsck on mount = True 2024-05-27T09:26:19.821 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluefs = 20 2024-05-27T09:26:19.821 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluestore = 20 2024-05-27T09:26:19.821 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-05-27T09:26:19.821 INFO:teuthology.orchestra.run.smithi114.stdout: debug osd = 20 2024-05-27T09:26:19.821 INFO:teuthology.orchestra.run.smithi114.stdout: debug rocksdb = 10 2024-05-27T09:26:19.821 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd backfillfull_ratio = 0.85 2024-05-27T09:26:19.821 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd full ratio = 0.9 2024-05-27T09:26:19.821 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd nearfull ratio = 0.8 2024-05-27T09:26:19.821 INFO:teuthology.orchestra.run.smithi114.stdout: osd blocked scrub grace period = 3600 2024-05-27T09:26:19.821 INFO:teuthology.orchestra.run.smithi114.stdout: osd failsafe full ratio = 0.95 2024-05-27T09:26:19.822 INFO:teuthology.orchestra.run.smithi114.stdout: osd max markdown count = 1000 2024-05-27T09:26:19.822 INFO:teuthology.orchestra.run.smithi114.stdout: osd objectstore = bluestore 2024-05-27T09:26:19.822 INFO:teuthology.orchestra.run.smithi114.stdout: osd op complaint time = 180 2024-05-27T09:26:19.822 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.822 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr] 2024-05-27T09:26:19.822 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-05-27T09:26:19.822 INFO:teuthology.orchestra.run.smithi114.stdout: debug mgr = 20 2024-05-27T09:26:19.822 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2024-05-27T09:26:19.822 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2024-05-27T09:26:19.822 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2024-05-27T09:26:19.822 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2024-05-27T09:26:19.822 INFO:teuthology.orchestra.run.smithi114.stdout: mgr/telemetry/nag = false 2024-05-27T09:26:19.823 INFO:teuthology.orchestra.run.smithi114.stdout: debug client = 20 2024-05-27T09:26:19.823 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.823 INFO:teuthology.orchestra.run.smithi114.stdout:[mon] 2024-05-27T09:26:19.823 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-05-27T09:26:19.823 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2024-05-27T09:26:19.823 INFO:teuthology.orchestra.run.smithi114.stdout: debug paxos = 20 2024-05-27T09:26:19.823 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2024-05-27T09:26:19.823 INFO:teuthology.orchestra.run.smithi114.stdout: mon data avail warn = 5 2024-05-27T09:26:19.823 INFO:teuthology.orchestra.run.smithi114.stdout: mon mgr mkfs grace = 240 2024-05-27T09:26:19.823 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2024-05-27T09:26:19.823 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd reporter subtree level = osd 2024-05-27T09:26:19.823 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd prime pg temp = true 2024-05-27T09:26:19.824 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2024-05-27T09:26:19.824 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.824 INFO:teuthology.orchestra.run.smithi114.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-27T09:26:19.824 INFO:teuthology.orchestra.run.smithi114.stdout: auth mon ticket ttl = 660 # 11m 2024-05-27T09:26:19.824 INFO:teuthology.orchestra.run.smithi114.stdout: auth service ticket ttl = 240 # 4m 2024-05-27T09:26:19.824 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.824 INFO:teuthology.orchestra.run.smithi114.stdout: # don't complain about insecure global_id in the test suite 2024-05-27T09:26:19.824 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-27T09:26:19.824 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-27T09:26:19.824 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.824 INFO:teuthology.orchestra.run.smithi114.stdout: # 1m isn't quite enough 2024-05-27T09:26:19.824 INFO:teuthology.orchestra.run.smithi114.stdout: mon_down_mkfs_grace = 2m 2024-05-27T09:26:19.824 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.824 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_filestore_osds = false 2024-05-27T09:26:19.825 INFO:teuthology.orchestra.run.smithi114.stdout: mon down mkfs grace = 300 2024-05-27T09:26:19.825 INFO:teuthology.orchestra.run.smithi114.stdout: mon op complaint time = 120 2024-05-27T09:26:19.825 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:26:19.825 INFO:teuthology.orchestra.run.smithi114.stdout:[client] 2024-05-27T09:26:19.825 INFO:teuthology.orchestra.run.smithi114.stdout: rgw cache enabled = true 2024-05-27T09:26:19.825 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable ops log = true 2024-05-27T09:26:19.825 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable usage log = true 2024-05-27T09:26:19.825 INFO:teuthology.orchestra.run.smithi114.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-27T09:26:19.825 INFO:teuthology.orchestra.run.smithi114.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-27T09:26:19.825 INFO:teuthology.orchestra.run.smithi114.stdout: client mount timeout = 600 2024-05-27T09:26:19.825 INFO:teuthology.orchestra.run.smithi114.stdout: client_shutdown_timeout = 120 2024-05-27T09:26:19.825 INFO:teuthology.orchestra.run.smithi114.stdout: debug client = 20 2024-05-27T09:26:19.825 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-05-27T09:26:19.826 INFO:teuthology.orchestra.run.smithi114.stdout: rados mon op timeout = 900 2024-05-27T09:26:19.826 INFO:teuthology.orchestra.run.smithi114.stdout: rados osd op timeout = 900 2024-05-27T09:26:19.826 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.a] 2024-05-27T09:26:19.826 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.b] 2024-05-27T09:26:19.826 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.c] 2024-05-27T09:26:19.826 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.a] 2024-05-27T09:26:19.826 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.c] 2024-05-27T09:26:19.826 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.e] 2024-05-27T09:26:19.826 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.b] 2024-05-27T09:26:19.826 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.d] 2024-05-27T09:26:19.826 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.f] 2024-05-27T09:26:19.826 INFO:teuthology.orchestra.run.smithi114.stdout:[mds] 2024-05-27T09:26:19.826 INFO:teuthology.orchestra.run.smithi114.stdout: debug mds = 20 2024-05-27T09:26:19.827 INFO:teuthology.orchestra.run.smithi114.stdout: debug mds balancer = 20 2024-05-27T09:26:19.827 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-05-27T09:26:19.827 INFO:teuthology.orchestra.run.smithi114.stdout: mds bal fragment size max = 10000 2024-05-27T09:26:19.827 INFO:teuthology.orchestra.run.smithi114.stdout: mds bal merge size = 5 2024-05-27T09:26:19.827 INFO:teuthology.orchestra.run.smithi114.stdout: mds bal split bits = 3 2024-05-27T09:26:19.827 INFO:teuthology.orchestra.run.smithi114.stdout: mds bal split size = 100 2024-05-27T09:26:19.827 INFO:teuthology.orchestra.run.smithi114.stdout: mds debug frag = True 2024-05-27T09:26:19.827 INFO:teuthology.orchestra.run.smithi114.stdout: mds debug scatterstat = True 2024-05-27T09:26:19.827 INFO:teuthology.orchestra.run.smithi114.stdout: mds oft prefetch dirfrags = True 2024-05-27T09:26:19.827 INFO:teuthology.orchestra.run.smithi114.stdout: mds op complaint time = 180 2024-05-27T09:26:19.827 INFO:teuthology.orchestra.run.smithi114.stdout: mds verify scatter = True 2024-05-27T09:26:19.827 INFO:teuthology.orchestra.run.smithi114.stdout: mds_dir_prefetch = True 2024-05-27T09:26:19.827 INFO:teuthology.orchestra.run.smithi114.stdout: mds_sleep_rank_change = 5000000.0 2024-05-27T09:26:19.828 INFO:teuthology.orchestra.run.smithi114.stdout: osd op complaint time = 180 2024-05-27T09:26:19.828 INFO:teuthology.orchestra.run.smithi114.stdout: rados mon op timeout = 900 2024-05-27T09:26:19.828 INFO:teuthology.orchestra.run.smithi114.stdout: rados osd op timeout = 900 2024-05-27T09:26:19.847 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-27T09:26:19.847 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-27T09:26:20.054 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-27T09:26:20.054 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:26:20.054 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-27T09:26:20.103 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:26:20.104 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-27T09:26:20.198 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi114.front.sepia.ceph.com 2024-05-27T09:26:20.199 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:26:20.199 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-27T09:26:20.199 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-27T09:26:20.350 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:26:20.350 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-27T09:26:20.446 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi163.front.sepia.ceph.com 2024-05-27T09:26:20.446 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-27T09:26:20.446 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-27T09:26:20.446 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-27T09:26:20.485 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-27T09:26:20.485 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-27T09:26:20.543 INFO:tasks.ceph:Setting up mon nodes... 2024-05-27T09:26:20.543 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-27T09:26:20.543 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-27T09:26:20.769 INFO:teuthology.orchestra.run.smithi114.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-27T09:26:20.785 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-27T09:26:20.873 INFO:teuthology.orchestra.run.smithi163.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-27T09:26:20.876 INFO:tasks.ceph:Setting up mds nodes... 2024-05-27T09:26:20.876 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-27T09:26:21.112 INFO:teuthology.orchestra.run.smithi114.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-27T09:26:21.127 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-27T09:26:21.209 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-27T09:26:21.436 INFO:teuthology.orchestra.run.smithi114.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-27T09:26:21.450 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-27T09:26:21.533 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-05-27T09:26:21.750 INFO:teuthology.orchestra.run.smithi114.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-05-27T09:26:21.765 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-05-27T09:26:21.848 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-27T09:26:21.916 INFO:teuthology.orchestra.run.smithi163.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-27T09:26:21.919 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-27T09:26:21.944 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-27T09:26:22.043 INFO:teuthology.orchestra.run.smithi163.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-27T09:26:22.046 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-27T09:26:22.070 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-05-27T09:26:22.169 INFO:teuthology.orchestra.run.smithi163.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-05-27T09:26:22.173 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-05-27T09:26:22.196 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-27T09:26:22.197 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-27T09:26:22.390 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-27T09:26:22.436 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-27T09:26:22.436 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi114.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi163.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-27T09:26:22.436 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-27T09:26:22.517 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-27T09:26:22.517 INFO:tasks.ceph:role: osd.0 2024-05-27T09:26:22.517 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi114.front.sepia.ceph.com 2024-05-27T09:26:22.518 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-27T09:26:22.606 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T09:26:22.606 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T09:26:22.606 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T09:26:22.607 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T09:26:22.607 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T09:26:22.607 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-05-27T09:26:22.607 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T09:26:22.607 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T09:26:22.607 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T09:26:22.607 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T09:26:23.024 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-05-27T09:26:23.032 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-05-27T09:26:23.032 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-27T09:26:23.186 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-27T09:26:23.276 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-27T09:26:23.356 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-27T09:26:23.356 INFO:tasks.ceph:role: osd.1 2024-05-27T09:26:23.356 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi114.front.sepia.ceph.com 2024-05-27T09:26:23.356 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-27T09:26:23.436 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T09:26:23.437 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T09:26:23.437 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T09:26:23.437 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T09:26:23.437 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T09:26:23.437 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-05-27T09:26:23.437 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T09:26:23.437 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T09:26:23.437 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T09:26:23.437 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T09:26:23.793 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-05-27T09:26:23.798 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-05-27T09:26:23.798 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-27T09:26:23.920 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-27T09:26:24.008 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-27T09:26:24.092 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-27T09:26:24.092 INFO:tasks.ceph:role: osd.2 2024-05-27T09:26:24.092 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi114.front.sepia.ceph.com 2024-05-27T09:26:24.093 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-27T09:26:24.180 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T09:26:24.180 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T09:26:24.180 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T09:26:24.180 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T09:26:24.180 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T09:26:24.180 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-05-27T09:26:24.180 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T09:26:24.180 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T09:26:24.181 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T09:26:24.181 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T09:26:24.573 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-05-27T09:26:24.579 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-05-27T09:26:24.580 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-27T09:26:24.742 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-27T09:26:24.831 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-27T09:26:24.907 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-27T09:26:24.907 INFO:tasks.ceph:role: osd.3 2024-05-27T09:26:24.907 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi114.front.sepia.ceph.com 2024-05-27T09:26:24.907 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-27T09:26:24.990 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T09:26:24.990 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T09:26:24.990 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T09:26:24.990 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T09:26:24.990 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T09:26:24.991 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-05-27T09:26:24.991 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T09:26:24.991 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T09:26:24.991 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T09:26:24.991 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T09:26:25.402 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-05-27T09:26:25.408 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-05-27T09:26:25.408 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-27T09:26:25.567 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-27T09:26:25.660 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T09:26:25.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:25.860+0000 7fa18081b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-27T09:26:25.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:25.861+0000 7fa18081b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-27T09:26:25.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:25.861+0000 7fa18081b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-27T09:26:25.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:25.861+0000 7fa18081b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-27T09:26:25.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:25.861+0000 7fa18081b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-27T09:26:25.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:25.865+0000 7fa18081b640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-27T09:26:29.103 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-27T09:26:29.185 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T09:26:29.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:29.387+0000 7f43c502d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-27T09:26:29.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:29.388+0000 7f43c502d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-27T09:26:29.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:29.388+0000 7f43c502d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-27T09:26:29.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:29.388+0000 7f43c502d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-27T09:26:29.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:29.388+0000 7f43c502d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-27T09:26:29.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:29.390+0000 7f43c502d640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-27T09:26:32.619 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-27T09:26:32.702 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T09:26:32.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:32.897+0000 7fd5d2215640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-27T09:26:32.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:32.897+0000 7fd5d2215640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-27T09:26:32.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:32.897+0000 7fd5d2215640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-27T09:26:32.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:32.897+0000 7fd5d2215640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-27T09:26:32.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:32.897+0000 7fd5d2215640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-27T09:26:32.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:32.900+0000 7fd5d2215640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-27T09:26:35.877 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-27T09:26:35.961 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T09:26:36.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:36.156+0000 7fb20760e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-27T09:26:36.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:36.157+0000 7fb20760e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-27T09:26:36.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:36.157+0000 7fb20760e640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-27T09:26:36.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:36.157+0000 7fb20760e640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-27T09:26:36.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:36.157+0000 7fb20760e640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-27T09:26:36.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:26:36.159+0000 7fb20760e640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-27T09:26:39.127 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-27T09:26:39.211 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-27T09:26:39.240 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-27T09:26:39.240 INFO:tasks.ceph:role: osd.4 2024-05-27T09:26:39.240 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi163.front.sepia.ceph.com 2024-05-27T09:26:39.240 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-27T09:26:39.314 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T09:26:39.314 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T09:26:39.314 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T09:26:39.314 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T09:26:39.315 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T09:26:39.315 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2024-05-27T09:26:39.315 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T09:26:39.315 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T09:26:39.315 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T09:26:39.315 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T09:26:39.827 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2024-05-27T09:26:39.829 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2024-05-27T09:26:39.829 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-27T09:26:39.965 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-27T09:26:39.999 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-27T09:26:40.066 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-27T09:26:40.066 INFO:tasks.ceph:role: osd.5 2024-05-27T09:26:40.066 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi163.front.sepia.ceph.com 2024-05-27T09:26:40.066 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-27T09:26:40.140 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T09:26:40.140 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T09:26:40.140 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T09:26:40.140 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T09:26:40.140 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T09:26:40.140 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2024-05-27T09:26:40.140 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T09:26:40.140 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T09:26:40.140 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T09:26:40.140 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T09:26:40.594 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2024-05-27T09:26:40.596 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2024-05-27T09:26:40.596 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-27T09:26:40.690 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-27T09:26:40.720 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-27T09:26:40.786 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-27T09:26:40.786 INFO:tasks.ceph:role: osd.6 2024-05-27T09:26:40.786 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi163.front.sepia.ceph.com 2024-05-27T09:26:40.786 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-27T09:26:40.859 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T09:26:40.859 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T09:26:40.859 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T09:26:40.859 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T09:26:40.859 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T09:26:40.859 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2024-05-27T09:26:40.859 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T09:26:40.859 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T09:26:40.859 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T09:26:40.859 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T09:26:41.342 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2024-05-27T09:26:41.345 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2024-05-27T09:26:41.345 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-27T09:26:41.473 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-27T09:26:41.504 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-27T09:26:41.571 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-27T09:26:41.571 INFO:tasks.ceph:role: osd.7 2024-05-27T09:26:41.571 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi163.front.sepia.ceph.com 2024-05-27T09:26:41.571 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-27T09:26:41.644 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-27T09:26:41.645 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-27T09:26:41.645 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-27T09:26:41.645 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-27T09:26:41.645 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-27T09:26:41.645 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2024-05-27T09:26:41.645 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-27T09:26:41.645 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-27T09:26:41.645 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-27T09:26:41.645 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-27T09:26:42.149 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2024-05-27T09:26:42.152 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2024-05-27T09:26:42.152 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-27T09:26:42.299 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-27T09:26:42.331 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T09:26:42.447 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:42.445+0000 7f6caec11640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-27T09:26:42.447 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:42.445+0000 7f6caec11640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-27T09:26:42.447 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:42.445+0000 7f6caec11640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-27T09:26:42.447 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:42.445+0000 7f6caec11640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-27T09:26:42.448 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:42.445+0000 7f6caec11640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-27T09:26:42.448 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:42.446+0000 7f6caec11640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-27T09:26:45.298 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-27T09:26:45.373 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T09:26:45.469 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:45.467+0000 7f4fcaa25640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-27T09:26:45.469 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:45.468+0000 7f4fcaa25640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-27T09:26:45.469 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:45.468+0000 7f4fcaa25640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-27T09:26:45.469 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:45.468+0000 7f4fcaa25640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-27T09:26:45.469 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:45.468+0000 7f4fcaa25640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-27T09:26:45.469 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:45.468+0000 7f4fcaa25640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-27T09:26:47.125 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:26:47.133 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:26:48.314 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-27T09:26:48.339 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T09:26:48.435 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:48.434+0000 7f5a98a5d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-27T09:26:48.435 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:48.434+0000 7f5a98a5d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-27T09:26:48.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:48.434+0000 7f5a98a5d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-27T09:26:48.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:48.434+0000 7f5a98a5d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-27T09:26:48.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:48.434+0000 7f5a98a5d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-27T09:26:48.436 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:48.435+0000 7f5a98a5d640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-27T09:26:51.298 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-27T09:26:51.327 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-27T09:26:51.429 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:51.428+0000 7f5c94dda640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-27T09:26:51.429 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:51.429+0000 7f5c94dda640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-27T09:26:51.430 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:51.429+0000 7f5c94dda640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-27T09:26:51.430 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:51.429+0000 7f5c94dda640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-27T09:26:51.430 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:51.429+0000 7f5c94dda640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-27T09:26:51.430 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:26:51.429+0000 7f5c94dda640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-27T09:26:54.280 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-27T09:26:54.307 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-27T09:26:54.308 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:26:54.308 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-27T09:26:54.398 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:26:54.399 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-27T09:26:54.484 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:26:54.484 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-27T09:26:54.567 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:26:54.567 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-05-27T09:26:54.650 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:26:54.650 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-27T09:26:54.728 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:26:54.729 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-27T09:26:54.855 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:26:54.856 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-27T09:26:54.934 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:26:54.934 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-27T09:26:55.016 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-27T09:26:55.016 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-27T09:26:55.044 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-27T09:26:55.044 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-27T09:26:55.112 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-27T09:26:55.112 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-27T09:26:55.179 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-27T09:26:55.180 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-05-27T09:26:55.246 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-27T09:26:55.246 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-27T09:26:55.311 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-27T09:26:55.311 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-27T09:26:55.376 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-27T09:26:55.376 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-27T09:26:55.440 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-27T09:26:55.441 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-27T09:26:55.504 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:26:55.504 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-27T09:26:55.562 INFO:tasks.ceph:Adding keys to all mons... 2024-05-27T09:26:55.563 DEBUG:teuthology.orchestra.run.smithi114:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-27T09:26:55.569 DEBUG:teuthology.orchestra.run.smithi163:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-27T09:26:55.591 INFO:teuthology.orchestra.run.smithi163.stdout:[mgr.y] 2024-05-27T09:26:55.591 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQC8UVRmYpDiLRAAgv36JI09F/UmdOuajJEXXQ== 2024-05-27T09:26:55.592 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.a] 2024-05-27T09:26:55.592 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQC9UVRmYgyzBhAABFOXoE27JVmG/Z1uufOlWg== 2024-05-27T09:26:55.592 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.c] 2024-05-27T09:26:55.592 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQC9UVRmOmcBGhAA/qPr77WfwfvJcJS2dNPiCA== 2024-05-27T09:26:55.592 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.e] 2024-05-27T09:26:55.592 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQC9UVRmE5PCLBAA5RF+A3xMesMucrAzNilgzw== 2024-05-27T09:26:55.592 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.0] 2024-05-27T09:26:55.592 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDBUVRmautVMxAADsgYc8RvOJ4YiM9d+CbQBA== 2024-05-27T09:26:55.592 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.1] 2024-05-27T09:26:55.592 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDFUVRm7bAkFxAAmh55sJVUQo8toy3hFYcx+g== 2024-05-27T09:26:55.592 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.2] 2024-05-27T09:26:55.592 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDIUVRmAgmQNRAARw+QTRv7tZwu9U9vDvba4Q== 2024-05-27T09:26:55.593 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.3] 2024-05-27T09:26:55.593 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDMUVRmA85tCRAALu/mdrJcs15NSWkc4esO5A== 2024-05-27T09:26:55.593 INFO:teuthology.orchestra.run.smithi163.stdout:[mgr.x] 2024-05-27T09:26:55.593 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQC8UVRmZH4ONBAAfy+3N8l8Ka3Yu0neJrQzgQ== 2024-05-27T09:26:55.593 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.b] 2024-05-27T09:26:55.593 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQC9UVRmleOiNhAAAB5V6a6s3s2VIboEEd0Xfg== 2024-05-27T09:26:55.593 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.d] 2024-05-27T09:26:55.593 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQC+UVRmy6SXAhAANzXee42Ol3JDBdUkvtHt9w== 2024-05-27T09:26:55.593 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.f] 2024-05-27T09:26:55.593 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQC+UVRmrG8nChAAeoW/gT3WvIcl+dnyDIW2uw== 2024-05-27T09:26:55.593 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.4] 2024-05-27T09:26:55.593 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDSUVRmZQaoGhAAE8NUjmzRA+WWTjFvjRUX+A== 2024-05-27T09:26:55.593 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.5] 2024-05-27T09:26:55.593 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDVUVRmD1rwGxAACGeAItE+8t1UduEa5MhAvw== 2024-05-27T09:26:55.593 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.6] 2024-05-27T09:26:55.594 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDYUVRmm/71GRAAN+FVv/q5BCia97y3ZFgYgA== 2024-05-27T09:26:55.594 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.7] 2024-05-27T09:26:55.594 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDbUVRmVMicGRAAkQMNN/K0hQz4b7+tBylDAA== 2024-05-27T09:26:55.594 INFO:teuthology.orchestra.run.smithi163.stdout:[client.0] 2024-05-27T09:26:55.594 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQC+UVRmdihKFxAADYNpomgY+R5hOd6q8DCQZA== 2024-05-27T09:26:55.643 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr.y] 2024-05-27T09:26:55.643 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQC8UVRmYpDiLRAAgv36JI09F/UmdOuajJEXXQ== 2024-05-27T09:26:55.643 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.a] 2024-05-27T09:26:55.643 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQC9UVRmYgyzBhAABFOXoE27JVmG/Z1uufOlWg== 2024-05-27T09:26:55.643 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.c] 2024-05-27T09:26:55.643 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQC9UVRmOmcBGhAA/qPr77WfwfvJcJS2dNPiCA== 2024-05-27T09:26:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.e] 2024-05-27T09:26:55.644 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQC9UVRmE5PCLBAA5RF+A3xMesMucrAzNilgzw== 2024-05-27T09:26:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.0] 2024-05-27T09:26:55.644 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQDBUVRmautVMxAADsgYc8RvOJ4YiM9d+CbQBA== 2024-05-27T09:26:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.1] 2024-05-27T09:26:55.644 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQDFUVRm7bAkFxAAmh55sJVUQo8toy3hFYcx+g== 2024-05-27T09:26:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.2] 2024-05-27T09:26:55.644 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQDIUVRmAgmQNRAARw+QTRv7tZwu9U9vDvba4Q== 2024-05-27T09:26:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.3] 2024-05-27T09:26:55.644 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQDMUVRmA85tCRAALu/mdrJcs15NSWkc4esO5A== 2024-05-27T09:26:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr.x] 2024-05-27T09:26:55.644 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQC8UVRmZH4ONBAAfy+3N8l8Ka3Yu0neJrQzgQ== 2024-05-27T09:26:55.644 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.b] 2024-05-27T09:26:55.644 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQC9UVRmleOiNhAAAB5V6a6s3s2VIboEEd0Xfg== 2024-05-27T09:26:55.645 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.d] 2024-05-27T09:26:55.645 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQC+UVRmy6SXAhAANzXee42Ol3JDBdUkvtHt9w== 2024-05-27T09:26:55.645 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.f] 2024-05-27T09:26:55.645 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQC+UVRmrG8nChAAeoW/gT3WvIcl+dnyDIW2uw== 2024-05-27T09:26:55.645 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.4] 2024-05-27T09:26:55.645 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQDSUVRmZQaoGhAAE8NUjmzRA+WWTjFvjRUX+A== 2024-05-27T09:26:55.645 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.5] 2024-05-27T09:26:55.645 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQDVUVRmD1rwGxAACGeAItE+8t1UduEa5MhAvw== 2024-05-27T09:26:55.645 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.6] 2024-05-27T09:26:55.645 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQDYUVRmm/71GRAAN+FVv/q5BCia97y3ZFgYgA== 2024-05-27T09:26:55.645 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.7] 2024-05-27T09:26:55.645 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQDbUVRmVMicGRAAkQMNN/K0hQz4b7+tBylDAA== 2024-05-27T09:26:55.645 INFO:teuthology.orchestra.run.smithi114.stdout:[client.0] 2024-05-27T09:26:55.646 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQC+UVRmdihKFxAADYNpomgY+R5hOd6q8DCQZA== 2024-05-27T09:26:55.647 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-27T09:26:55.694 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-27T09:26:55.906 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T09:26:55.913 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T09:26:56.114 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T09:26:56.122 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T09:26:56.327 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T09:26:56.332 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T09:26:56.537 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T09:26:56.542 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T09:26:56.744 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T09:26:56.749 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T09:26:56.950 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T09:26:56.955 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T09:26:57.156 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T09:26:57.162 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T09:26:57.363 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-27T09:26:57.369 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-27T09:26:57.572 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T09:26:57.579 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T09:26:57.787 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T09:26:57.793 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T09:26:57.994 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T09:26:57.999 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-27T09:26:58.204 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T09:26:58.211 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T09:26:58.417 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T09:26:58.422 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T09:26:58.625 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T09:26:58.631 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T09:26:58.835 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T09:26:58.842 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-27T09:26:59.042 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-27T09:26:59.047 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-27T09:26:59.250 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-27T09:26:59.251 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-27T09:26:59.332 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-27T09:26:59.560 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-27T09:26:59.633 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-27T09:26:59.657 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-27T09:26:59.764 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-27T09:26:59.787 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-27T09:26:59.850 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-27T09:26:59.953 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-27T09:26:59.977 DEBUG:teuthology.orchestra.run.smithi114:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-27T09:26:59.985 DEBUG:teuthology.orchestra.run.smithi163:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-27T09:27:00.035 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-27T09:27:00.035 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-27T09:27:00.035 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-27T09:27:00.040 INFO:tasks.ceph.mon.a:Started 2024-05-27T09:27:00.040 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-27T09:27:00.040 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-27T09:27:00.073 INFO:tasks.ceph.mon.b:Started 2024-05-27T09:27:00.074 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-27T09:27:00.074 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-27T09:27:00.076 INFO:tasks.ceph.mon.c:Started 2024-05-27T09:27:00.076 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-27T09:27:00.076 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-27T09:27:00.076 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-27T09:27:00.081 INFO:tasks.ceph.mgr.y:Started 2024-05-27T09:27:00.081 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-27T09:27:00.081 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-27T09:27:00.083 INFO:tasks.ceph.mgr.x:Started 2024-05-27T09:27:00.083 INFO:tasks.ceph:Setting crush tunables to default 2024-05-27T09:27:00.083 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd crush tunables default 2024-05-27T09:27:00.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.327+0000 7f2336c00640 1 Processor -- start 2024-05-27T09:27:00.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.327+0000 7f2336c00640 1 -- start start 2024-05-27T09:27:00.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.329+0000 7f2336c00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f23300c6d10 0x7f233015fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:00.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.329+0000 7f2336c00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f23301569a0 0x7f233015ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:00.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.329+0000 7f2336c00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2330160180 con 0x7f23300ccba0 2024-05-27T09:27:00.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.329+0000 7f2336c00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2330160460 con 0x7f23301569a0 2024-05-27T09:27:00.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.329+0000 7f2336c00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f2330160740 con 0x7f23300c6d10 2024-05-27T09:27:00.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.329+0000 7f2335800640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f23300c6d10 0x7f233015fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:00.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.329+0000 7f2334e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f23301569a0 0x7f233015ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:00.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.329+0000 7f2335800640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f23300c6d10 0x7f233015fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:51698/0 (socket says 172.21.15.114:51698) 2024-05-27T09:27:00.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.329+0000 7f2335800640 1 -- 172.21.15.114:0/1923775740 learned_addr learned my addr 172.21.15.114:0/1923775740 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:00.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.330+0000 7f2335800640 1 -- 172.21.15.114:0/1923775740 >> v2:172.21.15.114:3300/0 conn(0x7f23301569a0 msgr2=0x7f233015ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:00.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.330+0000 7f2335800640 1 --2- 172.21.15.114:0/1923775740 >> v2:172.21.15.114:3300/0 conn(0x7f23301569a0 0x7f233015ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:00.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.330+0000 7f2335800640 1 -- 172.21.15.114:0/1923775740 >> v1:172.21.15.114:6789/0 conn(0x7f23300ccba0 legacy=0x7f23300d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:00.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.330+0000 7f2335800640 1 -- 172.21.15.114:0/1923775740 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2330160a20 con 0x7f23300c6d10 2024-05-27T09:27:00.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.331+0000 7f2335800640 1 --2- 172.21.15.114:0/1923775740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23300c6d10 0x7f233015fb40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f23300cbff0 tx=0x7f232402a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18841a44afa141bf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:00.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.333+0000 7f232fe00640 1 -- 172.21.15.114:0/1923775740 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324037070 con 0x7f23300c6d10 2024-05-27T09:27:00.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.333+0000 7f232fe00640 1 -- 172.21.15.114:0/1923775740 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f232402ae50 con 0x7f23300c6d10 2024-05-27T09:27:00.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.333+0000 7f232fe00640 1 -- 172.21.15.114:0/1923775740 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324032860 con 0x7f23300c6d10 2024-05-27T09:27:00.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.334+0000 7f2336c00640 1 -- 172.21.15.114:0/1923775740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23300c6d10 msgr2=0x7f233015fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:00.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.334+0000 7f2336c00640 1 --2- 172.21.15.114:0/1923775740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23300c6d10 0x7f233015fb40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f23300cbff0 tx=0x7f232402a300 comp rx=0 tx=0).stop 2024-05-27T09:27:00.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.335+0000 7f2336c00640 1 -- 172.21.15.114:0/1923775740 shutdown_connections 2024-05-27T09:27:00.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.335+0000 7f2336c00640 1 --2- 172.21.15.114:0/1923775740 >> v2:172.21.15.114:3300/0 conn(0x7f23301569a0 0x7f233015ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:00.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.335+0000 7f2336c00640 1 --2- 172.21.15.114:0/1923775740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23300c6d10 0x7f233015fb40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:00.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.335+0000 7f2336c00640 1 -- 172.21.15.114:0/1923775740 >> 172.21.15.114:0/1923775740 conn(0x7f2330068600 msgr2=0x7f2330068a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:00.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.335+0000 7f2336c00640 1 -- 172.21.15.114:0/1923775740 shutdown_connections 2024-05-27T09:27:00.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.335+0000 7f2336c00640 1 -- 172.21.15.114:0/1923775740 wait complete. 2024-05-27T09:27:00.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.338+0000 7f2336c00640 1 Processor -- start 2024-05-27T09:27:00.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.338+0000 7f2336c00640 1 -- start start 2024-05-27T09:27:00.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.340+0000 7f2336c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23300c6d10 0x7f23301f6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:00.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.340+0000 7f2336c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23300ccba0 0x7f233015dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:00.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.340+0000 7f2336c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23301569a0 0x7f233015e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:00.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.340+0000 7f2336c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f23300cd9a0 con 0x7f23301569a0 2024-05-27T09:27:00.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.340+0000 7f2336c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f233015c940 con 0x7f23300ccba0 2024-05-27T09:27:00.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.340+0000 7f2336c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f23300cebe0 con 0x7f23300c6d10 2024-05-27T09:27:00.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.341+0000 7f2335800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23300c6d10 0x7f23301f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:00.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.341+0000 7f2336200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23301569a0 0x7f233015e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:00.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.341+0000 7f2335800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23300c6d10 0x7f23301f6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:40580/0 (socket says 172.21.15.114:40580) 2024-05-27T09:27:00.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.341+0000 7f2334e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23300ccba0 0x7f233015dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:00.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.341+0000 7f2335800640 1 -- 172.21.15.114:0/1159452013 learned_addr learned my addr 172.21.15.114:0/1159452013 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:00.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.341+0000 7f2335800640 1 -- 172.21.15.114:0/1159452013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23300ccba0 msgr2=0x7f233015dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:00.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.341+0000 7f2335800640 1 --2- 172.21.15.114:0/1159452013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23300ccba0 0x7f233015dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:00.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.341+0000 7f2335800640 1 -- 172.21.15.114:0/1159452013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23301569a0 msgr2=0x7f233015e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:00.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.341+0000 7f2335800640 1 --2- 172.21.15.114:0/1159452013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23301569a0 0x7f233015e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:00.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.341+0000 7f2335800640 1 -- 172.21.15.114:0/1159452013 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23301f6750 con 0x7f23300c6d10 2024-05-27T09:27:00.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.341+0000 7f2335800640 1 --2- 172.21.15.114:0/1159452013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23300c6d10 0x7f23301f6210 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2324000c00 tx=0x7f23240046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:00.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.342+0000 7f232d600640 1 -- 172.21.15.114:0/1159452013 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324041020 con 0x7f23300c6d10 2024-05-27T09:27:00.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.342+0000 7f232d600640 1 -- 172.21.15.114:0/1159452013 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f232403c890 con 0x7f23300c6d10 2024-05-27T09:27:00.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.342+0000 7f232d600640 1 -- 172.21.15.114:0/1159452013 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324037040 con 0x7f23300c6d10 2024-05-27T09:27:00.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.342+0000 7f2336c00640 1 -- 172.21.15.114:0/1159452013 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f233015dff0 con 0x7f23300c6d10 2024-05-27T09:27:00.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.342+0000 7f2336c00640 1 -- 172.21.15.114:0/1159452013 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f233015f960 con 0x7f23300c6d10 2024-05-27T09:27:00.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.342+0000 7f232d600640 1 -- 172.21.15.114:0/1159452013 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2324044020 con 0x7f23300c6d10 2024-05-27T09:27:00.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.342+0000 7f232d600640 1 -- 172.21.15.114:0/1159452013 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f232403ddf0 con 0x7f23300c6d10 2024-05-27T09:27:00.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.343+0000 7f2336c00640 1 -- 172.21.15.114:0/1159452013 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2330050860 con 0x7f23300c6d10 2024-05-27T09:27:00.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.347+0000 7f232d600640 1 -- 172.21.15.114:0/1159452013 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f232404b030 con 0x7f23300c6d10 2024-05-27T09:27:00.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.432+0000 7f2336c00640 1 -- 172.21.15.114:0/1159452013 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f23301f5b50 con 0x7f23300c6d10 2024-05-27T09:27:00.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.438+0000 7f232d600640 1 -- 172.21.15.114:0/1159452013 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f232403ccd0 con 0x7f23300c6d10 2024-05-27T09:27:00.439 INFO:teuthology.orchestra.run.smithi114.stderr:adjusted tunables profile to default 2024-05-27T09:27:00.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.439+0000 7f2336c00640 1 -- 172.21.15.114:0/1159452013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23300c6d10 msgr2=0x7f23301f6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:00.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.439+0000 7f2336c00640 1 --2- 172.21.15.114:0/1159452013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23300c6d10 0x7f23301f6210 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2324000c00 tx=0x7f23240046c0 comp rx=0 tx=0).stop 2024-05-27T09:27:00.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.439+0000 7f2336c00640 1 -- 172.21.15.114:0/1159452013 shutdown_connections 2024-05-27T09:27:00.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.439+0000 7f2336c00640 1 --2- 172.21.15.114:0/1159452013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23301569a0 0x7f233015e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:00.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.439+0000 7f2336c00640 1 --2- 172.21.15.114:0/1159452013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23300ccba0 0x7f233015dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:00.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.439+0000 7f2336c00640 1 --2- 172.21.15.114:0/1159452013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23300c6d10 0x7f23301f6210 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:00.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.439+0000 7f2336c00640 1 -- 172.21.15.114:0/1159452013 >> 172.21.15.114:0/1159452013 conn(0x7f2330068600 msgr2=0x7f23301553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:00.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.440+0000 7f2336c00640 1 -- 172.21.15.114:0/1159452013 shutdown_connections 2024-05-27T09:27:00.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:00.440+0000 7f2336c00640 1 -- 172.21.15.114:0/1159452013 wait complete. 2024-05-27T09:27:00.464 INFO:tasks.ceph:check_enable_crimson: False 2024-05-27T09:27:00.464 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-27T09:27:00.464 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:27:00.464 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-27T09:27:00.477 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:00.477+0000 7fe8c8e3d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T09:27:00.488 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:00.487+0000 7f98ecc38040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T09:27:00.530 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:27:00.530 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-27T09:27:00.642 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:00.642+0000 7fe8c8e3d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T09:27:00.692 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:27:00.692 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-27T09:27:00.692 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:00.677+0000 7f98ecc38040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T09:27:00.696 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:00.696+0000 7fe8c8e3d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T09:27:00.740 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:00.739+0000 7f98ecc38040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T09:27:00.804 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:27:00.804 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-27T09:27:00.917 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-27T09:27:00.917 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-27T09:27:00.935 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:00.935+0000 7fe8c8e3d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T09:27:00.941 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-27T09:27:00.941 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-27T09:27:00.984 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:00.983+0000 7fe8c8e3d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T09:27:01.005 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-27T09:27:01.005 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-27T09:27:01.033 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:01.033+0000 7f98ecc38040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T09:27:01.069 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-05-27T09:27:01.069 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-27T09:27:01.086 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:01.086+0000 7f98ecc38040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T09:27:01.135 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 3561c750-1d63-42f6-a6ae-0ee63721e15f 0 2024-05-27T09:27:01.284 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.283+0000 7f043fa61640 1 Processor -- start 2024-05-27T09:27:01.284 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.283+0000 7f043fa61640 1 -- start start 2024-05-27T09:27:01.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.285+0000 7f043fa61640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04380cbed0 0x7f04380c5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:01.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.285+0000 7f043fa61640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f043815e640 0x7f04381548f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:01.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.285+0000 7f043fa61640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f043815a600 0x7f04380cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:01.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.285+0000 7f043fa61640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f043815dbf0 con 0x7f043815a600 2024-05-27T09:27:01.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.285+0000 7f043fa61640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f043815e1b0 con 0x7f043815e640 2024-05-27T09:27:01.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.285+0000 7f043fa61640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f043815ea50 con 0x7f04380cbed0 2024-05-27T09:27:01.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.285+0000 7f043cffc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f043815e640 0x7f04381548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:01.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.285+0000 7f043d7fd640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04380cbed0 0x7f04380c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:01.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.285+0000 7f043cffc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f043815e640 0x7f04381548f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56614/0 (socket says 172.21.15.163:56614) 2024-05-27T09:27:01.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.285+0000 7f043cffc640 1 -- 172.21.15.163:0/3133098867 learned_addr learned my addr 172.21.15.163:0/3133098867 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-27T09:27:01.287 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.285+0000 7f043d7fd640 1 --2- 172.21.15.163:0/3133098867 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04380cbed0 0x7f04380c5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:60328/0 (socket says 172.21.15.163:60328) 2024-05-27T09:27:01.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.285+0000 7f043cffc640 1 -- 172.21.15.163:0/3133098867 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04380cbed0 msgr2=0x7f04380c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:01.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.285+0000 7f043cffc640 1 --2- 172.21.15.163:0/3133098867 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04380cbed0 0x7f04380c5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.285+0000 7f043cffc640 1 -- 172.21.15.163:0/3133098867 >> v2:172.21.15.114:3300/0 conn(0x7f043815a600 msgr2=0x7f04380cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:01.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.285+0000 7f043cffc640 1 --2- 172.21.15.163:0/3133098867 >> v2:172.21.15.114:3300/0 conn(0x7f043815a600 0x7f04380cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.285+0000 7f043cffc640 1 -- 172.21.15.163:0/3133098867 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043815add0 con 0x7f043815e640 2024-05-27T09:27:01.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.285+0000 7f043d7fd640 1 --2- 172.21.15.163:0/3133098867 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04380cbed0 0x7f04380c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:01.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.286+0000 7f043cffc640 1 --2- 172.21.15.163:0/3133098867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f043815e640 0x7f04381548f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f04380d0830 tx=0x7f042802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92f3bf6335679f8b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:01.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.286+0000 7f042ffff640 1 -- 172.21.15.163:0/3133098867 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428037070 con 0x7f043815e640 2024-05-27T09:27:01.288 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.286+0000 7f042ffff640 1 -- 172.21.15.163:0/3133098867 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0428032440 con 0x7f043815e640 2024-05-27T09:27:01.289 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.286+0000 7f042ffff640 1 -- 172.21.15.163:0/3133098867 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428032810 con 0x7f043815e640 2024-05-27T09:27:01.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.287+0000 7f043fa61640 1 -- 172.21.15.163:0/3133098867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f043815e640 msgr2=0x7f04381548f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:01.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.287+0000 7f043fa61640 1 --2- 172.21.15.163:0/3133098867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f043815e640 0x7f04381548f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f04380d0830 tx=0x7f042802a270 comp rx=0 tx=0).stop 2024-05-27T09:27:01.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.287+0000 7f043fa61640 1 -- 172.21.15.163:0/3133098867 shutdown_connections 2024-05-27T09:27:01.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.287+0000 7f043fa61640 1 --2- 172.21.15.163:0/3133098867 >> v2:172.21.15.114:3300/0 conn(0x7f043815a600 0x7f04380cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.287+0000 7f043fa61640 1 --2- 172.21.15.163:0/3133098867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f043815e640 0x7f04381548f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.287+0000 7f043fa61640 1 --2- 172.21.15.163:0/3133098867 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04380cbed0 0x7f04380c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.287+0000 7f043fa61640 1 -- 172.21.15.163:0/3133098867 >> 172.21.15.163:0/3133098867 conn(0x7f0438068600 msgr2=0x7f0438068a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:01.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.287+0000 7f043fa61640 1 -- 172.21.15.163:0/3133098867 shutdown_connections 2024-05-27T09:27:01.290 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.287+0000 7f043fa61640 1 -- 172.21.15.163:0/3133098867 wait complete. 2024-05-27T09:27:01.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.289+0000 7f043fa61640 1 Processor -- start 2024-05-27T09:27:01.291 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.289+0000 7f043fa61640 1 -- start start 2024-05-27T09:27:01.293 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.290+0000 7f043fa61640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04380cbed0 0x7f04380cef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:01.293 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.290+0000 7f043fa61640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f043815a600 0x7f04380cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:01.293 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.290+0000 7f043fa61640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f043815e640 0x7f0438151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:01.293 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.290+0000 7f043fa61640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f043815de40 con 0x7f043815a600 2024-05-27T09:27:01.293 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.290+0000 7f043fa61640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f043815b390 con 0x7f04380cbed0 2024-05-27T09:27:01.293 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.290+0000 7f043fa61640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f043815aa10 con 0x7f043815e640 2024-05-27T09:27:01.293 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.290+0000 7f043d7fd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04380cbed0 0x7f04380cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:01.293 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.290+0000 7f043d7fd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04380cbed0 0x7f04380cef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56616/0 (socket says 172.21.15.163:56616) 2024-05-27T09:27:01.294 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.290+0000 7f043d7fd640 1 -- 172.21.15.163:0/1816786777 learned_addr learned my addr 172.21.15.163:0/1816786777 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-27T09:27:01.294 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.290+0000 7f043dffe640 1 --2- 172.21.15.163:0/1816786777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f043815e640 0x7f0438151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:01.294 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.290+0000 7f043cffc640 1 --2- 172.21.15.163:0/1816786777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f043815a600 0x7f04380cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:01.294 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.290+0000 7f043d7fd640 1 -- 172.21.15.163:0/1816786777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f043815e640 msgr2=0x7f0438151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:01.294 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.290+0000 7f043d7fd640 1 --2- 172.21.15.163:0/1816786777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f043815e640 0x7f0438151320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.294 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.290+0000 7f043d7fd640 1 -- 172.21.15.163:0/1816786777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f043815a600 msgr2=0x7f04380cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:01.294 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.290+0000 7f043d7fd640 1 --2- 172.21.15.163:0/1816786777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f043815a600 0x7f04380cc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.294 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.290+0000 7f043d7fd640 1 -- 172.21.15.163:0/1816786777 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04380cf4c0 con 0x7f04380cbed0 2024-05-27T09:27:01.294 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.291+0000 7f043d7fd640 1 --2- 172.21.15.163:0/1816786777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04380cbed0 0x7f04380cef80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f043400e9c0 tx=0x7f043400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:01.294 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.291+0000 7f042dffb640 1 -- 172.21.15.163:0/1816786777 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434017070 con 0x7f04380cbed0 2024-05-27T09:27:01.294 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.291+0000 7f042dffb640 1 -- 172.21.15.163:0/1816786777 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f043400ca60 con 0x7f04380cbed0 2024-05-27T09:27:01.294 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.291+0000 7f042dffb640 1 -- 172.21.15.163:0/1816786777 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043400ce30 con 0x7f04380cbed0 2024-05-27T09:27:01.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.291+0000 7f043fa61640 1 -- 172.21.15.163:0/1816786777 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04380c6eb0 con 0x7f04380cbed0 2024-05-27T09:27:01.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.291+0000 7f043fa61640 1 -- 172.21.15.163:0/1816786777 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04380c8b60 con 0x7f04380cbed0 2024-05-27T09:27:01.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.291+0000 7f042dffb640 1 -- 172.21.15.163:0/1816786777 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f043401a070 con 0x7f04380cbed0 2024-05-27T09:27:01.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.291+0000 7f042dffb640 1 -- 172.21.15.163:0/1816786777 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f043401c820 con 0x7f04380cbed0 2024-05-27T09:27:01.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.292+0000 7f043fa61640 1 -- 172.21.15.163:0/1816786777 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04380c88f0 con 0x7f04380cbed0 2024-05-27T09:27:01.295 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.294+0000 7f042dffb640 1 -- 172.21.15.163:0/1816786777 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0434013470 con 0x7f04380cbed0 2024-05-27T09:27:01.367 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.367+0000 7f043fa61640 1 -- 172.21.15.163:0/1816786777 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3561c750-1d63-42f6-a6ae-0ee63721e15f", "id": 0} v 0) -- 0x7f043806a380 con 0x7f04380cbed0 2024-05-27T09:27:01.372 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.372+0000 7f042dffb640 1 -- 172.21.15.163:0/1816786777 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3561c750-1d63-42f6-a6ae-0ee63721e15f", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f043401caf0 con 0x7f04380cbed0 2024-05-27T09:27:01.372 INFO:teuthology.orchestra.run.smithi163.stdout:0 2024-05-27T09:27:01.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.373+0000 7f043fa61640 1 -- 172.21.15.163:0/1816786777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04380cbed0 msgr2=0x7f04380cef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:01.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.373+0000 7f043fa61640 1 --2- 172.21.15.163:0/1816786777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04380cbed0 0x7f04380cef80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f043400e9c0 tx=0x7f043400ee80 comp rx=0 tx=0).stop 2024-05-27T09:27:01.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.373+0000 7f043fa61640 1 -- 172.21.15.163:0/1816786777 shutdown_connections 2024-05-27T09:27:01.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.373+0000 7f043fa61640 1 --2- 172.21.15.163:0/1816786777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f043815e640 0x7f0438151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.373+0000 7f043fa61640 1 --2- 172.21.15.163:0/1816786777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f043815a600 0x7f04380cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.373+0000 7f043fa61640 1 --2- 172.21.15.163:0/1816786777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04380cbed0 0x7f04380cef80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.373+0000 7f043fa61640 1 -- 172.21.15.163:0/1816786777 >> 172.21.15.163:0/1816786777 conn(0x7f0438068600 msgr2=0x7f04381592a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:01.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.373+0000 7f043fa61640 1 -- 172.21.15.163:0/1816786777 shutdown_connections 2024-05-27T09:27:01.374 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.373+0000 7f043fa61640 1 -- 172.21.15.163:0/1816786777 wait complete. 2024-05-27T09:27:01.384 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 9a407d04-38c0-4676-928f-d04e8116bd19 1 2024-05-27T09:27:01.493 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.492+0000 7f8ac2d4d640 1 Processor -- start 2024-05-27T09:27:01.493 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.492+0000 7f8ac2d4d640 1 -- start start 2024-05-27T09:27:01.495 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.493+0000 7f8ac2d4d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8abc160860 0x7f8abc15d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:01.495 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.493+0000 7f8ac2d4d640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8abc156760 0x7f8abc0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:01.495 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.493+0000 7f8ac2d4d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8abc0c73b0 0x7f8abc0c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:01.495 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.493+0000 7f8ac2d4d640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f8abc0cc6f0 con 0x7f8abc156760 2024-05-27T09:27:01.495 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.493+0000 7f8ac2d4d640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f8abc0cc990 con 0x7f8abc160860 2024-05-27T09:27:01.495 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.493+0000 7f8ac2d4d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f8abc0ccc30 con 0x7f8abc0c73b0 2024-05-27T09:27:01.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.493+0000 7f8ac12ea640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8abc0c73b0 0x7f8abc0c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:01.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.493+0000 7f8ac12ea640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8abc0c73b0 0x7f8abc0c77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:60354/0 (socket says 172.21.15.163:60354) 2024-05-27T09:27:01.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.493+0000 7f8ac12ea640 1 -- 172.21.15.163:0/2748491563 learned_addr learned my addr 172.21.15.163:0/2748491563 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-27T09:27:01.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.493+0000 7f8ac0ae9640 1 --2- 172.21.15.163:0/2748491563 >> v2:172.21.15.163:3300/0 conn(0x7f8abc160860 0x7f8abc15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:01.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.494+0000 7f8ac12ea640 1 -- 172.21.15.163:0/2748491563 >> v2:172.21.15.163:3300/0 conn(0x7f8abc160860 msgr2=0x7f8abc15d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:01.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.494+0000 7f8ac12ea640 1 --2- 172.21.15.163:0/2748491563 >> v2:172.21.15.163:3300/0 conn(0x7f8abc160860 0x7f8abc15d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.494+0000 7f8ac12ea640 1 -- 172.21.15.163:0/2748491563 >> v2:172.21.15.114:3300/0 conn(0x7f8abc156760 msgr2=0x7f8abc0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:01.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.494+0000 7f8ac12ea640 1 --2- 172.21.15.163:0/2748491563 >> v2:172.21.15.114:3300/0 conn(0x7f8abc156760 0x7f8abc0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.494+0000 7f8ac12ea640 1 -- 172.21.15.163:0/2748491563 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8abc0b5a30 con 0x7f8abc0c73b0 2024-05-27T09:27:01.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.494+0000 7f8ac0ae9640 1 --2- 172.21.15.163:0/2748491563 >> v2:172.21.15.163:3300/0 conn(0x7f8abc160860 0x7f8abc15d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:01.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.494+0000 7f8ac12ea640 1 --2- 172.21.15.163:0/2748491563 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8abc0c73b0 0x7f8abc0c77c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8abc069210 tx=0x7f8ab402a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b862e2fc66bf9cd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:01.496 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.494+0000 7f8ab37fe640 1 -- 172.21.15.163:0/2748491563 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4036070 con 0x7f8abc0c73b0 2024-05-27T09:27:01.497 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.495+0000 7f8ab37fe640 1 -- 172.21.15.163:0/2748491563 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ab4031600 con 0x7f8abc0c73b0 2024-05-27T09:27:01.497 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.495+0000 7f8ab37fe640 1 -- 172.21.15.163:0/2748491563 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40319d0 con 0x7f8abc0c73b0 2024-05-27T09:27:01.498 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.495+0000 7f8ac2d4d640 1 -- 172.21.15.163:0/2748491563 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8abc0c73b0 msgr2=0x7f8abc0c77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:01.498 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.495+0000 7f8ac2d4d640 1 --2- 172.21.15.163:0/2748491563 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8abc0c73b0 0x7f8abc0c77c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8abc069210 tx=0x7f8ab402a6a0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.499 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.495+0000 7f8ac2d4d640 1 -- 172.21.15.163:0/2748491563 shutdown_connections 2024-05-27T09:27:01.499 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.495+0000 7f8ac2d4d640 1 --2- 172.21.15.163:0/2748491563 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8abc0c73b0 0x7f8abc0c77c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.499 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.495+0000 7f8ac2d4d640 1 --2- 172.21.15.163:0/2748491563 >> v2:172.21.15.114:3300/0 conn(0x7f8abc156760 0x7f8abc0b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.499 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.495+0000 7f8ac2d4d640 1 --2- 172.21.15.163:0/2748491563 >> v2:172.21.15.163:3300/0 conn(0x7f8abc160860 0x7f8abc15d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.499 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.495+0000 7f8ac2d4d640 1 -- 172.21.15.163:0/2748491563 >> 172.21.15.163:0/2748491563 conn(0x7f8abc06f2f0 msgr2=0x7f8abc068600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:01.499 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.495+0000 7f8ac2d4d640 1 -- 172.21.15.163:0/2748491563 shutdown_connections 2024-05-27T09:27:01.499 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.495+0000 7f8ac2d4d640 1 -- 172.21.15.163:0/2748491563 wait complete. 2024-05-27T09:27:01.499 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.497+0000 7f8ac2d4d640 1 Processor -- start 2024-05-27T09:27:01.499 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.497+0000 7f8ac2d4d640 1 -- start start 2024-05-27T09:27:01.501 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ac2d4d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8abc0c73b0 0x7f8abc0c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:01.501 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ac2d4d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8abc156760 0x7f8abc1f32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:01.501 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ac2d4d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8abc160860 0x7f8abc14a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:01.501 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ac2d4d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc160340 con 0x7f8abc160860 2024-05-27T09:27:01.501 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ac2d4d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc160060 con 0x7f8abc156760 2024-05-27T09:27:01.502 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ac2d4d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f8abc0b4a00 con 0x7f8abc0c73b0 2024-05-27T09:27:01.502 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ac0ae9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8abc0c73b0 0x7f8abc0c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:01.502 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ab3fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8abc156760 0x7f8abc1f32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:01.502 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ac0ae9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8abc0c73b0 0x7f8abc0c9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:60368/0 (socket says 172.21.15.163:60368) 2024-05-27T09:27:01.502 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ab3fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8abc156760 0x7f8abc1f32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56636/0 (socket says 172.21.15.163:56636) 2024-05-27T09:27:01.502 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ac0ae9640 1 -- 172.21.15.163:0/1172288256 learned_addr learned my addr 172.21.15.163:0/1172288256 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-27T09:27:01.502 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ac0ae9640 1 -- 172.21.15.163:0/1172288256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8abc156760 msgr2=0x7f8abc1f32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:01.502 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ac0ae9640 1 --2- 172.21.15.163:0/1172288256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8abc156760 0x7f8abc1f32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.502 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ac0ae9640 1 -- 172.21.15.163:0/1172288256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8abc160860 msgr2=0x7f8abc14a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:01.502 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ac0ae9640 1 --2- 172.21.15.163:0/1172288256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8abc160860 0x7f8abc14a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.502 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ac0ae9640 1 -- 172.21.15.163:0/1172288256 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8abc1f37f0 con 0x7f8abc0c73b0 2024-05-27T09:27:01.503 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ac0ae9640 1 --2- 172.21.15.163:0/1172288256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8abc0c73b0 0x7f8abc0c9410 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800e990 tx=0x7f8ab800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:01.503 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ab17fa640 1 -- 172.21.15.163:0/1172288256 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8017070 con 0x7f8abc0c73b0 2024-05-27T09:27:01.503 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ab17fa640 1 -- 172.21.15.163:0/1172288256 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ab800c9e0 con 0x7f8abc0c73b0 2024-05-27T09:27:01.503 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ac2d4d640 1 -- 172.21.15.163:0/1172288256 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8abc0ca540 con 0x7f8abc0c73b0 2024-05-27T09:27:01.503 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ab17fa640 1 -- 172.21.15.163:0/1172288256 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab800cdb0 con 0x7f8abc0c73b0 2024-05-27T09:27:01.503 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ac2d4d640 1 -- 172.21.15.163:0/1172288256 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8abc0cab00 con 0x7f8abc0c73b0 2024-05-27T09:27:01.503 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.499+0000 7f8ab17fa640 1 -- 172.21.15.163:0/1172288256 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8ab801a070 con 0x7f8abc0c73b0 2024-05-27T09:27:01.503 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.500+0000 7f8ab17fa640 1 -- 172.21.15.163:0/1172288256 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f8ab801c800 con 0x7f8abc0c73b0 2024-05-27T09:27:01.503 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.500+0000 7f8ac2d4d640 1 -- 172.21.15.163:0/1172288256 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8abc06a380 con 0x7f8abc0c73b0 2024-05-27T09:27:01.504 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.502+0000 7f8ab17fa640 1 -- 172.21.15.163:0/1172288256 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f8ab801ca00 con 0x7f8abc0c73b0 2024-05-27T09:27:01.576 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.576+0000 7f8ac2d4d640 1 -- 172.21.15.163:0/1172288256 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9a407d04-38c0-4676-928f-d04e8116bd19", "id": 1} v 0) -- 0x7f8abc0c6c10 con 0x7f8abc0c73b0 2024-05-27T09:27:01.581 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.581+0000 7f8ab17fa640 1 -- 172.21.15.163:0/1172288256 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9a407d04-38c0-4676-928f-d04e8116bd19", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f8ab801cbe0 con 0x7f8abc0c73b0 2024-05-27T09:27:01.582 INFO:teuthology.orchestra.run.smithi163.stdout:1 2024-05-27T09:27:01.583 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.582+0000 7f8ac2d4d640 1 -- 172.21.15.163:0/1172288256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8abc0c73b0 msgr2=0x7f8abc0c9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:01.583 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.582+0000 7f8ac2d4d640 1 --2- 172.21.15.163:0/1172288256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8abc0c73b0 0x7f8abc0c9410 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800e990 tx=0x7f8ab800ee50 comp rx=0 tx=0).stop 2024-05-27T09:27:01.583 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.582+0000 7f8ac2d4d640 1 -- 172.21.15.163:0/1172288256 shutdown_connections 2024-05-27T09:27:01.583 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.582+0000 7f8ac2d4d640 1 --2- 172.21.15.163:0/1172288256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8abc160860 0x7f8abc14a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.583 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.582+0000 7f8ac2d4d640 1 --2- 172.21.15.163:0/1172288256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8abc156760 0x7f8abc1f32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.583 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.582+0000 7f8ac2d4d640 1 --2- 172.21.15.163:0/1172288256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8abc0c73b0 0x7f8abc0c9410 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.583 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.582+0000 7f8ac2d4d640 1 -- 172.21.15.163:0/1172288256 >> 172.21.15.163:0/1172288256 conn(0x7f8abc06f2f0 msgr2=0x7f8abc14c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:01.583 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.582+0000 7f8ac2d4d640 1 -- 172.21.15.163:0/1172288256 shutdown_connections 2024-05-27T09:27:01.583 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.582+0000 7f8ac2d4d640 1 -- 172.21.15.163:0/1172288256 wait complete. 2024-05-27T09:27:01.594 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 255a1cc6-9357-43f7-9db5-d760f4806ab2 2 2024-05-27T09:27:01.702 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.702+0000 7fdc70e61640 1 Processor -- start 2024-05-27T09:27:01.703 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.702+0000 7fdc70e61640 1 -- start start 2024-05-27T09:27:01.704 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.703+0000 7fdc70e61640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc6c0ca6e0 0x7fdc6c15cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:01.704 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.703+0000 7fdc70e61640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc6c0c5880 con 0x7fdc6c15d2d0 2024-05-27T09:27:01.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.703+0000 7fdc70e61640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc6c160940 con 0x7fdc6c156cd0 2024-05-27T09:27:01.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.703+0000 7fdc70e61640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fdc6c0c5b20 con 0x7fdc6c0ca6e0 2024-05-27T09:27:01.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.704+0000 7fdc69d9b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc6c0ca6e0 0x7fdc6c15cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:01.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.704+0000 7fdc6a59c640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fdc6c156cd0 0x7fdc6c073ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:58794/0 (socket says 172.21.15.163:58794) 2024-05-27T09:27:01.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.704+0000 7fdc6a59c640 1 -- 172.21.15.163:0/1458581537 learned_addr learned my addr 172.21.15.163:0/1458581537 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-27T09:27:01.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.704+0000 7fdc69d9b640 1 -- 172.21.15.163:0/1458581537 >> v1:172.21.15.163:6789/0 conn(0x7fdc6c156cd0 legacy=0x7fdc6c073ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:01.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.704+0000 7fdc69d9b640 1 -- 172.21.15.163:0/1458581537 >> v1:172.21.15.114:6789/0 conn(0x7fdc6c15d2d0 legacy=0x7fdc6c1670c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:01.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.704+0000 7fdc69d9b640 1 -- 172.21.15.163:0/1458581537 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6c0c5e00 con 0x7fdc6c0ca6e0 2024-05-27T09:27:01.705 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.704+0000 7fdc6959a640 1 -- 172.21.15.163:0/1458581537 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2366182492 0 0) 0x7fdc6c160940 con 0x7fdc6c156cd0 2024-05-27T09:27:01.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.705+0000 7fdc69d9b640 1 --2- 172.21.15.163:0/1458581537 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc6c0ca6e0 0x7fdc6c15cd90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c0097d0 tx=0x7fdc5c02a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39bd969db4e12233 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:01.706 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.705+0000 7fdc6959a640 1 -- 172.21.15.163:0/1458581537 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c037070 con 0x7fdc6c0ca6e0 2024-05-27T09:27:01.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.705+0000 7fdc6959a640 1 -- 172.21.15.163:0/1458581537 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc5c032440 con 0x7fdc6c0ca6e0 2024-05-27T09:27:01.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.705+0000 7fdc6959a640 1 -- 172.21.15.163:0/1458581537 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c032810 con 0x7fdc6c0ca6e0 2024-05-27T09:27:01.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.705+0000 7fdc70e61640 1 -- 172.21.15.163:0/1458581537 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc6c0ca6e0 msgr2=0x7fdc6c15cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:01.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.705+0000 7fdc70e61640 1 --2- 172.21.15.163:0/1458581537 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc6c0ca6e0 0x7fdc6c15cd90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c0097d0 tx=0x7fdc5c02a200 comp rx=0 tx=0).stop 2024-05-27T09:27:01.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.706+0000 7fdc70e61640 1 -- 172.21.15.163:0/1458581537 shutdown_connections 2024-05-27T09:27:01.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.706+0000 7fdc70e61640 1 --2- 172.21.15.163:0/1458581537 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc6c0ca6e0 0x7fdc6c15cd90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.706+0000 7fdc70e61640 1 -- 172.21.15.163:0/1458581537 >> 172.21.15.163:0/1458581537 conn(0x7fdc6c0cade0 msgr2=0x7fdc6c0cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:01.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.706+0000 7fdc70e61640 1 -- 172.21.15.163:0/1458581537 shutdown_connections 2024-05-27T09:27:01.707 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.706+0000 7fdc70e61640 1 -- 172.21.15.163:0/1458581537 wait complete. 2024-05-27T09:27:01.708 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.707+0000 7fdc70e61640 1 Processor -- start 2024-05-27T09:27:01.708 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.708+0000 7fdc70e61640 1 -- start start 2024-05-27T09:27:01.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.709+0000 7fdc70e61640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc6c0ca6e0 0x7fdc6c15e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:01.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.709+0000 7fdc70e61640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc6c156cd0 0x7fdc6c155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:01.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.709+0000 7fdc70e61640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc6c15d2d0 0x7fdc6c155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:01.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.709+0000 7fdc70e61640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c15c820 con 0x7fdc6c156cd0 2024-05-27T09:27:01.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.709+0000 7fdc70e61640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c160c20 con 0x7fdc6c15d2d0 2024-05-27T09:27:01.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.709+0000 7fdc70e61640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fdc6c06dbb0 con 0x7fdc6c0ca6e0 2024-05-27T09:27:01.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.709+0000 7fdc6a59c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc6c0ca6e0 0x7fdc6c15e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:01.711 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.709+0000 7fdc6ad9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc6c15d2d0 0x7fdc6c155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:01.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.709+0000 7fdc6a59c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc6c0ca6e0 0x7fdc6c15e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:60382/0 (socket says 172.21.15.163:60382) 2024-05-27T09:27:01.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.709+0000 7fdc6a59c640 1 -- 172.21.15.163:0/2031081046 learned_addr learned my addr 172.21.15.163:0/2031081046 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-27T09:27:01.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.709+0000 7fdc6a59c640 1 -- 172.21.15.163:0/2031081046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc6c15d2d0 msgr2=0x7fdc6c155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:01.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.709+0000 7fdc6a59c640 1 --2- 172.21.15.163:0/2031081046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc6c15d2d0 0x7fdc6c155ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.709+0000 7fdc6a59c640 1 -- 172.21.15.163:0/2031081046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc6c156cd0 msgr2=0x7fdc6c155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:01.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.709+0000 7fdc6a59c640 1 --2- 172.21.15.163:0/2031081046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc6c156cd0 0x7fdc6c155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.709+0000 7fdc6a59c640 1 -- 172.21.15.163:0/2031081046 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6c15eed0 con 0x7fdc6c0ca6e0 2024-05-27T09:27:01.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.709+0000 7fdc6ad9d640 1 --2- 172.21.15.163:0/2031081046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc6c15d2d0 0x7fdc6c155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:01.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.709+0000 7fdc6a59c640 1 --2- 172.21.15.163:0/2031081046 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc6c0ca6e0 0x7fdc6c15e990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000c410 tx=0x7fdc6000c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:01.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.710+0000 7fdc5affd640 1 -- 172.21.15.163:0/2031081046 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc600074e0 con 0x7fdc6c0ca6e0 2024-05-27T09:27:01.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.710+0000 7fdc5affd640 1 -- 172.21.15.163:0/2031081046 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc600047e0 con 0x7fdc6c0ca6e0 2024-05-27T09:27:01.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.710+0000 7fdc5affd640 1 -- 172.21.15.163:0/2031081046 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc60004bd0 con 0x7fdc6c0ca6e0 2024-05-27T09:27:01.712 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.710+0000 7fdc70e61640 1 -- 172.21.15.163:0/2031081046 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc6c15f570 con 0x7fdc6c0ca6e0 2024-05-27T09:27:01.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.710+0000 7fdc70e61640 1 -- 172.21.15.163:0/2031081046 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc6c15deb0 con 0x7fdc6c0ca6e0 2024-05-27T09:27:01.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.710+0000 7fdc5affd640 1 -- 172.21.15.163:0/2031081046 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdc60011070 con 0x7fdc6c0ca6e0 2024-05-27T09:27:01.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.710+0000 7fdc5affd640 1 -- 172.21.15.163:0/2031081046 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fdc600137e0 con 0x7fdc6c0ca6e0 2024-05-27T09:27:01.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.711+0000 7fdc70e61640 1 -- 172.21.15.163:0/2031081046 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc6c15d0a0 con 0x7fdc6c0ca6e0 2024-05-27T09:27:01.713 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.713+0000 7fdc5affd640 1 -- 172.21.15.163:0/2031081046 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fdc6c15d0a0 con 0x7fdc6c0ca6e0 2024-05-27T09:27:01.750 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:01.749+0000 7fe8c8e3d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T09:27:01.787 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.787+0000 7fdc70e61640 1 -- 172.21.15.163:0/2031081046 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd new", "uuid": "255a1cc6-9357-43f7-9db5-d760f4806ab2", "id": 2} v 0) -- 0x7fdc6c06a380 con 0x7fdc6c0ca6e0 2024-05-27T09:27:01.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.793+0000 7fdc5affd640 1 -- 172.21.15.163:0/2031081046 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "255a1cc6-9357-43f7-9db5-d760f4806ab2", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fdc600133c0 con 0x7fdc6c0ca6e0 2024-05-27T09:27:01.794 INFO:teuthology.orchestra.run.smithi163.stdout:2 2024-05-27T09:27:01.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.794+0000 7fdc70e61640 1 -- 172.21.15.163:0/2031081046 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc6c0ca6e0 msgr2=0x7fdc6c15e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:01.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.794+0000 7fdc70e61640 1 --2- 172.21.15.163:0/2031081046 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc6c0ca6e0 0x7fdc6c15e990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000c410 tx=0x7fdc6000c8d0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.794+0000 7fdc70e61640 1 -- 172.21.15.163:0/2031081046 shutdown_connections 2024-05-27T09:27:01.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.794+0000 7fdc70e61640 1 --2- 172.21.15.163:0/2031081046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc6c15d2d0 0x7fdc6c155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.794+0000 7fdc70e61640 1 --2- 172.21.15.163:0/2031081046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc6c156cd0 0x7fdc6c155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.794+0000 7fdc70e61640 1 --2- 172.21.15.163:0/2031081046 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc6c0ca6e0 0x7fdc6c15e990 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.794+0000 7fdc70e61640 1 -- 172.21.15.163:0/2031081046 >> 172.21.15.163:0/2031081046 conn(0x7fdc6c0cade0 msgr2=0x7fdc6c0c8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:01.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.794+0000 7fdc70e61640 1 -- 172.21.15.163:0/2031081046 shutdown_connections 2024-05-27T09:27:01.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.794+0000 7fdc70e61640 1 -- 172.21.15.163:0/2031081046 wait complete. 2024-05-27T09:27:01.805 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new cb4d7692-9a17-41b9-927e-d8eac3e217c2 3 2024-05-27T09:27:01.844 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:01.843+0000 7fe8c8e3d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T09:27:01.890 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:01.890+0000 7fe8c8e3d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T09:27:01.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.914+0000 7fa73e682640 1 Processor -- start 2024-05-27T09:27:01.915 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.914+0000 7fa73e682640 1 -- start start 2024-05-27T09:27:01.916 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.915+0000 7fa73e682640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7380ca810 0x7fa73806c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:01.917 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.915+0000 7fa73e682640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa7380c5d10 0x7fa7381537b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:01.917 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.915+0000 7fa73e682640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa7380c7f10 0x7fa73815a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:01.917 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.915+0000 7fa73e682640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa7380c9dc0 con 0x7fa7380c7f10 2024-05-27T09:27:01.917 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.915+0000 7fa73e682640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fa738159bc0 con 0x7fa7380c5d10 2024-05-27T09:27:01.917 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.915+0000 7fa73e682640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa7381595b0 con 0x7fa7380ca810 2024-05-27T09:27:01.917 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.916+0000 7fa7377fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa7380c5d10 0x7fa7381537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:01.917 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.916+0000 7fa737fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7380ca810 0x7fa73806c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:01.917 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.916+0000 7fa737fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7380ca810 0x7fa73806c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:60384/0 (socket says 172.21.15.163:60384) 2024-05-27T09:27:01.917 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.916+0000 7fa737fff640 1 -- 172.21.15.163:0/2167885633 learned_addr learned my addr 172.21.15.163:0/2167885633 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-27T09:27:01.917 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.916+0000 7fa7377fe640 1 -- 172.21.15.163:0/2167885633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7380ca810 msgr2=0x7fa73806c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:01.917 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.916+0000 7fa7377fe640 1 --2- 172.21.15.163:0/2167885633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7380ca810 0x7fa73806c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.917 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.916+0000 7fa7377fe640 1 -- 172.21.15.163:0/2167885633 >> v2:172.21.15.114:3300/0 conn(0x7fa7380c7f10 msgr2=0x7fa73815a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:01.918 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.916+0000 7fa7377fe640 1 --2- 172.21.15.163:0/2167885633 >> v2:172.21.15.114:3300/0 conn(0x7fa7380c7f10 0x7fa73815a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.918 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.916+0000 7fa7377fe640 1 -- 172.21.15.163:0/2167885633 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7380c9ff0 con 0x7fa7380c5d10 2024-05-27T09:27:01.918 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.916+0000 7fa737fff640 1 --2- 172.21.15.163:0/2167885633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7380ca810 0x7fa73806c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:01.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.917+0000 7fa7377fe640 1 --2- 172.21.15.163:0/2167885633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7380c5d10 0x7fa7381537b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa738074bb0 tx=0x7fa72802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20d9559935a0ae38 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:01.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.917+0000 7fa736ffd640 1 -- 172.21.15.163:0/2167885633 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728037070 con 0x7fa7380c5d10 2024-05-27T09:27:01.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.917+0000 7fa736ffd640 1 -- 172.21.15.163:0/2167885633 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa728032440 con 0x7fa7380c5d10 2024-05-27T09:27:01.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.917+0000 7fa736ffd640 1 -- 172.21.15.163:0/2167885633 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728032810 con 0x7fa7380c5d10 2024-05-27T09:27:01.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.918+0000 7fa73e682640 1 -- 172.21.15.163:0/2167885633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7380c5d10 msgr2=0x7fa7381537b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:01.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.918+0000 7fa73e682640 1 --2- 172.21.15.163:0/2167885633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7380c5d10 0x7fa7381537b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa738074bb0 tx=0x7fa72802a1c0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.919 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.918+0000 7fa73e682640 1 -- 172.21.15.163:0/2167885633 shutdown_connections 2024-05-27T09:27:01.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.918+0000 7fa73e682640 1 --2- 172.21.15.163:0/2167885633 >> v2:172.21.15.114:3300/0 conn(0x7fa7380c7f10 0x7fa73815a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.918+0000 7fa73e682640 1 --2- 172.21.15.163:0/2167885633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7380c5d10 0x7fa7381537b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.918+0000 7fa73e682640 1 --2- 172.21.15.163:0/2167885633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7380ca810 0x7fa73806c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.918+0000 7fa73e682640 1 -- 172.21.15.163:0/2167885633 >> 172.21.15.163:0/2167885633 conn(0x7fa7380cad40 msgr2=0x7fa7380cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:01.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.918+0000 7fa73e682640 1 -- 172.21.15.163:0/2167885633 shutdown_connections 2024-05-27T09:27:01.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.918+0000 7fa73e682640 1 -- 172.21.15.163:0/2167885633 wait complete. 2024-05-27T09:27:01.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.919+0000 7fa73e682640 1 Processor -- start 2024-05-27T09:27:01.920 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.920+0000 7fa73e682640 1 -- start start 2024-05-27T09:27:01.923 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.921+0000 7fa73e682640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7380c5d10 0x7fa738157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:01.923 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.921+0000 7fa73e682640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7380c7f10 0x7fa7381e3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:01.923 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.921+0000 7fa73e682640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7380ca810 0x7fa7380cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:01.923 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.921+0000 7fa73e682640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa7380c85f0 con 0x7fa7380c5d10 2024-05-27T09:27:01.923 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.921+0000 7fa73e682640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa738159850 con 0x7fa7380c7f10 2024-05-27T09:27:01.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.921+0000 7fa73e682640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa7380c6f90 con 0x7fa7380ca810 2024-05-27T09:27:01.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.921+0000 7fa7377fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7380c7f10 0x7fa7381e3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:01.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.921+0000 7fa73cc1f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7380ca810 0x7fa7380cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:01.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.921+0000 7fa73cc1f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7380ca810 0x7fa7380cc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:60398/0 (socket says 172.21.15.163:60398) 2024-05-27T09:27:01.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.921+0000 7fa73cc1f640 1 -- 172.21.15.163:0/840244606 learned_addr learned my addr 172.21.15.163:0/840244606 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-27T09:27:01.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.921+0000 7fa73cc1f640 1 -- 172.21.15.163:0/840244606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7380c7f10 msgr2=0x7fa7381e3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:01.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.921+0000 7fa73cc1f640 1 --2- 172.21.15.163:0/840244606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7380c7f10 0x7fa7381e3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.921+0000 7fa73cc1f640 1 -- 172.21.15.163:0/840244606 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7380c5d10 msgr2=0x7fa738157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:01.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.921+0000 7fa73cc1f640 1 --2- 172.21.15.163:0/840244606 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7380c5d10 0x7fa738157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:01.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.921+0000 7fa73cc1f640 1 -- 172.21.15.163:0/840244606 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7381f68d0 con 0x7fa7380ca810 2024-05-27T09:27:01.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.921+0000 7fa7377fe640 1 --2- 172.21.15.163:0/840244606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7380c7f10 0x7fa7381e3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:01.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.922+0000 7fa73cc1f640 1 --2- 172.21.15.163:0/840244606 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7380ca810 0x7fa7380cc100 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa73000ba10 tx=0x7fa73000bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:01.924 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.922+0000 7fa734ff9640 1 -- 172.21.15.163:0/840244606 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa730019070 con 0x7fa7380ca810 2024-05-27T09:27:01.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.922+0000 7fa734ff9640 1 -- 172.21.15.163:0/840244606 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa73000cb20 con 0x7fa7380ca810 2024-05-27T09:27:01.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.922+0000 7fa73e682640 1 -- 172.21.15.163:0/840244606 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa738157f60 con 0x7fa7380ca810 2024-05-27T09:27:01.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.922+0000 7fa734ff9640 1 -- 172.21.15.163:0/840244606 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa730014440 con 0x7fa7380ca810 2024-05-27T09:27:01.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.922+0000 7fa73e682640 1 -- 172.21.15.163:0/840244606 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7380cdf20 con 0x7fa7380ca810 2024-05-27T09:27:01.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.922+0000 7fa734ff9640 1 -- 172.21.15.163:0/840244606 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa73001c070 con 0x7fa7380ca810 2024-05-27T09:27:01.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.922+0000 7fa734ff9640 1 -- 172.21.15.163:0/840244606 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fa73001d840 con 0x7fa7380ca810 2024-05-27T09:27:01.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.923+0000 7fa73e682640 1 -- 172.21.15.163:0/840244606 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa73806a380 con 0x7fa7380ca810 2024-05-27T09:27:01.925 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.925+0000 7fa734ff9640 1 -- 172.21.15.163:0/840244606 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa730023070 con 0x7fa7380ca810 2024-05-27T09:27:01.998 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:01.998+0000 7fa73e682640 1 -- 172.21.15.163:0/840244606 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd new", "uuid": "cb4d7692-9a17-41b9-927e-d8eac3e217c2", "id": 3} v 0) -- 0x7fa738071fd0 con 0x7fa7380ca810 2024-05-27T09:27:02.005 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.005+0000 7fa734ff9640 1 -- 172.21.15.163:0/840244606 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cb4d7692-9a17-41b9-927e-d8eac3e217c2", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fa7381f68d0 con 0x7fa7380ca810 2024-05-27T09:27:02.005 INFO:teuthology.orchestra.run.smithi163.stdout:3 2024-05-27T09:27:02.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.005+0000 7fa73e682640 1 -- 172.21.15.163:0/840244606 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7380ca810 msgr2=0x7fa7380cc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:02.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.005+0000 7fa73e682640 1 --2- 172.21.15.163:0/840244606 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7380ca810 0x7fa7380cc100 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa73000ba10 tx=0x7fa73000bed0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.005+0000 7fa73e682640 1 -- 172.21.15.163:0/840244606 shutdown_connections 2024-05-27T09:27:02.006 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.005+0000 7fa73e682640 1 --2- 172.21.15.163:0/840244606 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7380ca810 0x7fa7380cc100 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.007 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.005+0000 7fa73e682640 1 --2- 172.21.15.163:0/840244606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7380c7f10 0x7fa7381e3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.007 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.005+0000 7fa73e682640 1 --2- 172.21.15.163:0/840244606 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa7380c5d10 0x7fa738157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.007 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.005+0000 7fa73e682640 1 -- 172.21.15.163:0/840244606 >> 172.21.15.163:0/840244606 conn(0x7fa7380cad40 msgr2=0x7fa738150480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:02.007 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.006+0000 7fa73e682640 1 -- 172.21.15.163:0/840244606 shutdown_connections 2024-05-27T09:27:02.007 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.006+0000 7fa73e682640 1 -- 172.21.15.163:0/840244606 wait complete. 2024-05-27T09:27:02.017 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new cdef1940-3453-4fbd-b0f1-459f5f5c7142 4 2024-05-27T09:27:02.065 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:02.065+0000 7fe8c8e3d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T09:27:02.068 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:02.068+0000 7f98ecc38040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T09:27:02.141 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:02.141+0000 7fe8c8e3d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T09:27:02.166 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.165+0000 7f1bd2976640 1 Processor -- start 2024-05-27T09:27:02.166 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.165+0000 7f1bd2976640 1 -- start start 2024-05-27T09:27:02.168 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.167+0000 7f1bd2976640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1bcc0cb260 0x7f1bcc0cb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:02.168 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.167+0000 7f1bd2976640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bcc0cf8a0 0x7f1bcc0cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:02.168 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.167+0000 7f1bd2976640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bcc156f20 con 0x7f1bcc0c6840 2024-05-27T09:27:02.168 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.167+0000 7f1bd2976640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f1bcc15fd90 con 0x7f1bcc0cb260 2024-05-27T09:27:02.168 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.167+0000 7f1bd2976640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f1bcc0c5350 con 0x7f1bcc0cf8a0 2024-05-27T09:27:02.168 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.167+0000 7f1bcb7fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1bcc0cb260 0x7f1bcc0cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:02.168 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.167+0000 7f1bd0f13640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bcc0cf8a0 0x7f1bcc0cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:02.168 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.167+0000 7f1bcb7fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1bcc0cb260 0x7f1bcc0cb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56654/0 (socket says 172.21.15.163:56654) 2024-05-27T09:27:02.168 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.167+0000 7f1bcb7fe640 1 -- 172.21.15.163:0/3727440987 learned_addr learned my addr 172.21.15.163:0/3727440987 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-27T09:27:02.169 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.167+0000 7f1bcbfff640 1 --1- 172.21.15.163:0/3727440987 >> v1:172.21.15.163:6789/0 conn(0x7f1bcc0c6840 0x7f1bcc160500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:58806/0 (socket says 172.21.15.163:58806) 2024-05-27T09:27:02.169 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.167+0000 7f1bcb7fe640 1 -- 172.21.15.163:0/3727440987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bcc0cf8a0 msgr2=0x7f1bcc0cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:02.169 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.167+0000 7f1bcb7fe640 1 --2- 172.21.15.163:0/3727440987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bcc0cf8a0 0x7f1bcc0cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.169 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.167+0000 7f1bcb7fe640 1 -- 172.21.15.163:0/3727440987 >> v1:172.21.15.163:6789/0 conn(0x7f1bcc0c6840 legacy=0x7f1bcc160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:02.169 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.167+0000 7f1bcaffd640 1 -- 172.21.15.163:0/3727440987 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3613479478 0 0) 0x7f1bcc156f20 con 0x7f1bcc0c6840 2024-05-27T09:27:02.169 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.167+0000 7f1bcb7fe640 1 -- 172.21.15.163:0/3727440987 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bcc06e180 con 0x7f1bcc0cb260 2024-05-27T09:27:02.169 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.167+0000 7f1bd0f13640 1 --2- 172.21.15.163:0/3727440987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bcc0cf8a0 0x7f1bcc0cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:02.170 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.168+0000 7f1bcb7fe640 1 --2- 172.21.15.163:0/3727440987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bcc0cb260 0x7f1bcc0cb670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc009850 tx=0x7f1bbc02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fd2bc6b0956e277 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:02.170 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.168+0000 7f1bcaffd640 1 -- 172.21.15.163:0/3727440987 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc004030 con 0x7f1bcc0cb260 2024-05-27T09:27:02.170 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.168+0000 7f1bcaffd640 1 -- 172.21.15.163:0/3727440987 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bbc0041d0 con 0x7f1bcc0cb260 2024-05-27T09:27:02.171 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.168+0000 7f1bcaffd640 1 -- 172.21.15.163:0/3727440987 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc003610 con 0x7f1bcc0cb260 2024-05-27T09:27:02.171 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.169+0000 7f1bd2976640 1 -- 172.21.15.163:0/3727440987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bcc0cb260 msgr2=0x7f1bcc0cb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:02.171 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.169+0000 7f1bd2976640 1 --2- 172.21.15.163:0/3727440987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bcc0cb260 0x7f1bcc0cb670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc009850 tx=0x7f1bbc02a230 comp rx=0 tx=0).stop 2024-05-27T09:27:02.171 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.169+0000 7f1bd2976640 1 -- 172.21.15.163:0/3727440987 shutdown_connections 2024-05-27T09:27:02.171 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.169+0000 7f1bd2976640 1 --2- 172.21.15.163:0/3727440987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bcc0cf8a0 0x7f1bcc0cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.171 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.169+0000 7f1bd2976640 1 --2- 172.21.15.163:0/3727440987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bcc0cb260 0x7f1bcc0cb670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.171 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.169+0000 7f1bd2976640 1 -- 172.21.15.163:0/3727440987 >> 172.21.15.163:0/3727440987 conn(0x7f1bcc0731e0 msgr2=0x7f1bcc071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:02.171 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.169+0000 7f1bd2976640 1 -- 172.21.15.163:0/3727440987 shutdown_connections 2024-05-27T09:27:02.171 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.169+0000 7f1bd2976640 1 -- 172.21.15.163:0/3727440987 wait complete. 2024-05-27T09:27:02.172 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.171+0000 7f1bd2976640 1 Processor -- start 2024-05-27T09:27:02.172 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.171+0000 7f1bd2976640 1 -- start start 2024-05-27T09:27:02.173 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:02.172+0000 7f98ecc38040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T09:27:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.172+0000 7f1bd2976640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1bcc0c6840 0x7f1bcc15a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.172+0000 7f1bd2976640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bcc0cb260 0x7f1bcc159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.172+0000 7f1bd2976640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bcc0cf8a0 0x7f1bcc152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.172+0000 7f1bd2976640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc06ea20 con 0x7f1bcc0c6840 2024-05-27T09:27:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.172+0000 7f1bd2976640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc06e740 con 0x7f1bcc0cb260 2024-05-27T09:27:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.172+0000 7f1bd2976640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f1bcc15ffe0 con 0x7f1bcc0cf8a0 2024-05-27T09:27:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.172+0000 7f1bcb7fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bcc0cb260 0x7f1bcc159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.172+0000 7f1bcb7fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bcc0cb260 0x7f1bcc159b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56660/0 (socket says 172.21.15.163:56660) 2024-05-27T09:27:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.172+0000 7f1bcb7fe640 1 -- 172.21.15.163:0/3507805278 learned_addr learned my addr 172.21.15.163:0/3507805278 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-27T09:27:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.172+0000 7f1bd0f13640 1 --2- 172.21.15.163:0/3507805278 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bcc0cf8a0 0x7f1bcc152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.173+0000 7f1bcbfff640 1 --2- 172.21.15.163:0/3507805278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1bcc0c6840 0x7f1bcc15a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:02.176 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.173+0000 7f1bcb7fe640 1 -- 172.21.15.163:0/3507805278 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bcc0cf8a0 msgr2=0x7f1bcc152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:02.176 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.173+0000 7f1bcb7fe640 1 --2- 172.21.15.163:0/3507805278 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bcc0cf8a0 0x7f1bcc152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.176 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.173+0000 7f1bcb7fe640 1 -- 172.21.15.163:0/3507805278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1bcc0c6840 msgr2=0x7f1bcc15a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:02.176 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.173+0000 7f1bcb7fe640 1 --2- 172.21.15.163:0/3507805278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1bcc0c6840 0x7f1bcc15a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.176 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.173+0000 7f1bcb7fe640 1 -- 172.21.15.163:0/3507805278 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bcc1e6f20 con 0x7f1bcc0cb260 2024-05-27T09:27:02.176 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.173+0000 7f1bd0f13640 1 --2- 172.21.15.163:0/3507805278 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bcc0cf8a0 0x7f1bcc152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:02.176 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.173+0000 7f1bcb7fe640 1 --2- 172.21.15.163:0/3507805278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bcc0cb260 0x7f1bcc159b00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc000c00 tx=0x7f1bbc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:02.176 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.173+0000 7f1bc8ff9640 1 -- 172.21.15.163:0/3507805278 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc041020 con 0x7f1bcc0cb260 2024-05-27T09:27:02.176 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.173+0000 7f1bc8ff9640 1 -- 172.21.15.163:0/3507805278 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bbc003c50 con 0x7f1bcc0cb260 2024-05-27T09:27:02.176 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.173+0000 7f1bd2976640 1 -- 172.21.15.163:0/3507805278 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bcc1e5650 con 0x7f1bcc0cb260 2024-05-27T09:27:02.176 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.173+0000 7f1bc8ff9640 1 -- 172.21.15.163:0/3507805278 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc004030 con 0x7f1bcc0cb260 2024-05-27T09:27:02.176 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.173+0000 7f1bd2976640 1 -- 172.21.15.163:0/3507805278 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bcc153000 con 0x7f1bcc0cb260 2024-05-27T09:27:02.176 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.173+0000 7f1bc8ff9640 1 -- 172.21.15.163:0/3507805278 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1bbc044020 con 0x7f1bcc0cb260 2024-05-27T09:27:02.177 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.173+0000 7f1bc8ff9640 1 -- 172.21.15.163:0/3507805278 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f1bbc03c960 con 0x7f1bcc0cb260 2024-05-27T09:27:02.177 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.174+0000 7f1bd2976640 1 -- 172.21.15.163:0/3507805278 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bcc06a380 con 0x7f1bcc0cb260 2024-05-27T09:27:02.177 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.176+0000 7f1bc8ff9640 1 -- 172.21.15.163:0/3507805278 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f1bbc04b030 con 0x7f1bcc0cb260 2024-05-27T09:27:02.197 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:02.197+0000 7fe8c8e3d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T09:27:02.223 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:02.223+0000 7f98ecc38040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T09:27:02.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.249+0000 7f1bd2976640 1 -- 172.21.15.163:0/3507805278 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cdef1940-3453-4fbd-b0f1-459f5f5c7142", "id": 4} v 0) -- 0x7f1bcc0cb040 con 0x7f1bcc0cb260 2024-05-27T09:27:02.256 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.256+0000 7f1bc8ff9640 1 -- 172.21.15.163:0/3507805278 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cdef1940-3453-4fbd-b0f1-459f5f5c7142", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f1bbc03cc80 con 0x7f1bcc0cb260 2024-05-27T09:27:02.256 INFO:teuthology.orchestra.run.smithi163.stdout:4 2024-05-27T09:27:02.258 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.257+0000 7f1bd2976640 1 -- 172.21.15.163:0/3507805278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bcc0cb260 msgr2=0x7f1bcc159b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:02.258 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.257+0000 7f1bd2976640 1 --2- 172.21.15.163:0/3507805278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bcc0cb260 0x7f1bcc159b00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc000c00 tx=0x7f1bbc0046c0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.258 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.257+0000 7f1bd2976640 1 -- 172.21.15.163:0/3507805278 shutdown_connections 2024-05-27T09:27:02.258 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.257+0000 7f1bd2976640 1 --2- 172.21.15.163:0/3507805278 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bcc0cf8a0 0x7f1bcc152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.258 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.257+0000 7f1bd2976640 1 --2- 172.21.15.163:0/3507805278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bcc0cb260 0x7f1bcc159b00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.258 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.257+0000 7f1bd2976640 1 --2- 172.21.15.163:0/3507805278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1bcc0c6840 0x7f1bcc15a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.258 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.257+0000 7f1bd2976640 1 -- 172.21.15.163:0/3507805278 >> 172.21.15.163:0/3507805278 conn(0x7f1bcc0731e0 msgr2=0x7f1bcc153960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:02.258 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.257+0000 7f1bd2976640 1 -- 172.21.15.163:0/3507805278 shutdown_connections 2024-05-27T09:27:02.258 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.257+0000 7f1bd2976640 1 -- 172.21.15.163:0/3507805278 wait complete. 2024-05-27T09:27:02.268 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 220a7b32-abcc-492f-bca8-4615cc2ea111 5 2024-05-27T09:27:02.315 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:02.315+0000 7fe8c8e3d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T09:27:02.415 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:02.414+0000 7f98ecc38040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T09:27:02.419 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.418+0000 7f2cfa90d640 1 Processor -- start 2024-05-27T09:27:02.419 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.418+0000 7f2cfa90d640 1 -- start start 2024-05-27T09:27:02.421 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.420+0000 7f2cfa90d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2cf40ca3d0 0x7f2cf41569e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:02.421 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.420+0000 7f2cfa90d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf40c7e40 0x7f2cf40715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:02.421 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.420+0000 7f2cfa90d640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cf40c7250 con 0x7f2cf40711c0 2024-05-27T09:27:02.421 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.420+0000 7f2cfa90d640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f2cf415d270 con 0x7f2cf40ca3d0 2024-05-27T09:27:02.421 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.420+0000 7f2cfa90d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f2cf415d530 con 0x7f2cf40c7e40 2024-05-27T09:27:02.421 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.420+0000 7f2cf8eaa640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf40c7e40 0x7f2cf40715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:02.421 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.420+0000 7f2cf3fff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2cf40ca3d0 0x7f2cf41569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:02.421 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.420+0000 7f2cf37fe640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2cf40711c0 0x7f2cf40c7730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:58814/0 (socket says 172.21.15.163:58814) 2024-05-27T09:27:02.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.420+0000 7f2cf3fff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2cf40ca3d0 0x7f2cf41569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56666/0 (socket says 172.21.15.163:56666) 2024-05-27T09:27:02.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.420+0000 7f2cf8eaa640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf40c7e40 0x7f2cf40715d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:60436/0 (socket says 172.21.15.163:60436) 2024-05-27T09:27:02.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.420+0000 7f2cf37fe640 1 -- 172.21.15.163:0/3511165115 learned_addr learned my addr 172.21.15.163:0/3511165115 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-27T09:27:02.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.420+0000 7f2cf8eaa640 1 -- 172.21.15.163:0/3511165115 >> v2:172.21.15.163:3300/0 conn(0x7f2cf40ca3d0 msgr2=0x7f2cf41569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:02.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.420+0000 7f2cf8eaa640 1 --2- 172.21.15.163:0/3511165115 >> v2:172.21.15.163:3300/0 conn(0x7f2cf40ca3d0 0x7f2cf41569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.420+0000 7f2cf8eaa640 1 -- 172.21.15.163:0/3511165115 >> v1:172.21.15.163:6789/0 conn(0x7f2cf40711c0 legacy=0x7f2cf40c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:02.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.420+0000 7f2cf8eaa640 1 -- 172.21.15.163:0/3511165115 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf40c5570 con 0x7f2cf40c7e40 2024-05-27T09:27:02.422 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.420+0000 7f2cf3fff640 1 --2- 172.21.15.163:0/3511165115 >> v2:172.21.15.163:3300/0 conn(0x7f2cf40ca3d0 0x7f2cf41569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:02.424 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.421+0000 7f2cf8eaa640 1 --2- 172.21.15.163:0/3511165115 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf40c7e40 0x7f2cf40715d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2cec009d90 tx=0x7f2cec02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6061b3b9db70fc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:02.424 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.421+0000 7f2cf2ffd640 1 -- 172.21.15.163:0/3511165115 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec007740 con 0x7f2cf40c7e40 2024-05-27T09:27:02.424 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.421+0000 7f2cf2ffd640 1 -- 172.21.15.163:0/3511165115 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cec0078e0 con 0x7f2cf40c7e40 2024-05-27T09:27:02.424 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.421+0000 7f2cf2ffd640 1 -- 172.21.15.163:0/3511165115 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec007cb0 con 0x7f2cf40c7e40 2024-05-27T09:27:02.424 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.422+0000 7f2cfa90d640 1 -- 172.21.15.163:0/3511165115 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf40c7e40 msgr2=0x7f2cf40715d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:02.424 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.422+0000 7f2cfa90d640 1 --2- 172.21.15.163:0/3511165115 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf40c7e40 0x7f2cf40715d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2cec009d90 tx=0x7f2cec02a660 comp rx=0 tx=0).stop 2024-05-27T09:27:02.424 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.422+0000 7f2cfa90d640 1 -- 172.21.15.163:0/3511165115 shutdown_connections 2024-05-27T09:27:02.424 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.422+0000 7f2cfa90d640 1 --2- 172.21.15.163:0/3511165115 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf40c7e40 0x7f2cf40715d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.424 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.422+0000 7f2cfa90d640 1 --2- 172.21.15.163:0/3511165115 >> v2:172.21.15.163:3300/0 conn(0x7f2cf40ca3d0 0x7f2cf41569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.425 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.422+0000 7f2cfa90d640 1 -- 172.21.15.163:0/3511165115 >> 172.21.15.163:0/3511165115 conn(0x7f2cf40cdfd0 msgr2=0x7f2cf40ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:02.425 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.422+0000 7f2cfa90d640 1 -- 172.21.15.163:0/3511165115 shutdown_connections 2024-05-27T09:27:02.425 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.422+0000 7f2cfa90d640 1 -- 172.21.15.163:0/3511165115 wait complete. 2024-05-27T09:27:02.425 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.423+0000 7f2cfa90d640 1 Processor -- start 2024-05-27T09:27:02.425 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.424+0000 7f2cfa90d640 1 -- start start 2024-05-27T09:27:02.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.425+0000 7f2cfa90d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cf40711c0 0x7f2cf4150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:02.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.425+0000 7f2cfa90d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cf40c7e40 0x7f2cf41520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:02.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.425+0000 7f2cfa90d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf40ca3d0 0x7f2cf414e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:02.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.425+0000 7f2cfa90d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf40c5b30 con 0x7f2cf40c7e40 2024-05-27T09:27:02.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.425+0000 7f2cfa90d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf40c6b90 con 0x7f2cf40711c0 2024-05-27T09:27:02.427 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.425+0000 7f2cfa90d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f2cf40c5cb0 con 0x7f2cf40ca3d0 2024-05-27T09:27:02.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.425+0000 7f2cf3fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cf40711c0 0x7f2cf4150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:02.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.425+0000 7f2cf8eaa640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf40ca3d0 0x7f2cf414e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:02.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.425+0000 7f2cf3fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cf40711c0 0x7f2cf4150810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56668/0 (socket says 172.21.15.163:56668) 2024-05-27T09:27:02.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.425+0000 7f2cf3fff640 1 -- 172.21.15.163:0/2050188294 learned_addr learned my addr 172.21.15.163:0/2050188294 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-27T09:27:02.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.425+0000 7f2cf3fff640 1 -- 172.21.15.163:0/2050188294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf40ca3d0 msgr2=0x7f2cf414e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:02.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.425+0000 7f2cf3fff640 1 --2- 172.21.15.163:0/2050188294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf40ca3d0 0x7f2cf414e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.425+0000 7f2cf3fff640 1 -- 172.21.15.163:0/2050188294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cf40c7e40 msgr2=0x7f2cf41520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:02.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.425+0000 7f2cf3fff640 1 --2- 172.21.15.163:0/2050188294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cf40c7e40 0x7f2cf41520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.425+0000 7f2cf3fff640 1 -- 172.21.15.163:0/2050188294 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf4151e50 con 0x7f2cf40711c0 2024-05-27T09:27:02.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.425+0000 7f2cf3fff640 1 --2- 172.21.15.163:0/2050188294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cf40711c0 0x7f2cf4150810 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800e990 tx=0x7f2ce800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:02.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.426+0000 7f2cf0ff9640 1 -- 172.21.15.163:0/2050188294 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8017070 con 0x7f2cf40711c0 2024-05-27T09:27:02.428 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.426+0000 7f2cf0ff9640 1 -- 172.21.15.163:0/2050188294 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ce800c9e0 con 0x7f2cf40711c0 2024-05-27T09:27:02.429 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.426+0000 7f2cfa90d640 1 -- 172.21.15.163:0/2050188294 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf41f3040 con 0x7f2cf40711c0 2024-05-27T09:27:02.429 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.426+0000 7f2cf0ff9640 1 -- 172.21.15.163:0/2050188294 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce800cdb0 con 0x7f2cf40711c0 2024-05-27T09:27:02.429 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.426+0000 7f2cfa90d640 1 -- 172.21.15.163:0/2050188294 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf41f3600 con 0x7f2cf40711c0 2024-05-27T09:27:02.429 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.426+0000 7f2cf0ff9640 1 -- 172.21.15.163:0/2050188294 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2ce801a070 con 0x7f2cf40711c0 2024-05-27T09:27:02.429 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.426+0000 7f2cf0ff9640 1 -- 172.21.15.163:0/2050188294 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f2ce8013470 con 0x7f2cf40711c0 2024-05-27T09:27:02.429 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.427+0000 7f2cfa90d640 1 -- 172.21.15.163:0/2050188294 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf406a380 con 0x7f2cf40711c0 2024-05-27T09:27:02.431 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.430+0000 7f2cf0ff9640 1 -- 172.21.15.163:0/2050188294 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f2ce80064d0 con 0x7f2cf40711c0 2024-05-27T09:27:02.472 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:02.471+0000 7f98ecc38040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T09:27:02.504 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.503+0000 7f2cfa90d640 1 -- 172.21.15.163:0/2050188294 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd new", "uuid": "220a7b32-abcc-492f-bca8-4615cc2ea111", "id": 5} v 0) -- 0x7f2cf406de80 con 0x7f2cf40711c0 2024-05-27T09:27:02.510 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.510+0000 7f2cf0ff9640 1 -- 172.21.15.163:0/2050188294 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "220a7b32-abcc-492f-bca8-4615cc2ea111", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f2ce80066b0 con 0x7f2cf40711c0 2024-05-27T09:27:02.510 INFO:teuthology.orchestra.run.smithi163.stdout:5 2024-05-27T09:27:02.512 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.511+0000 7f2cfa90d640 1 -- 172.21.15.163:0/2050188294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cf40711c0 msgr2=0x7f2cf4150810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:02.512 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.511+0000 7f2cfa90d640 1 --2- 172.21.15.163:0/2050188294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cf40711c0 0x7f2cf4150810 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800e990 tx=0x7f2ce800ee50 comp rx=0 tx=0).stop 2024-05-27T09:27:02.512 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.511+0000 7f2cfa90d640 1 -- 172.21.15.163:0/2050188294 shutdown_connections 2024-05-27T09:27:02.512 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.511+0000 7f2cfa90d640 1 --2- 172.21.15.163:0/2050188294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cf40ca3d0 0x7f2cf414e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.512 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.511+0000 7f2cfa90d640 1 --2- 172.21.15.163:0/2050188294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cf40c7e40 0x7f2cf41520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.512 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.511+0000 7f2cfa90d640 1 --2- 172.21.15.163:0/2050188294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cf40711c0 0x7f2cf4150810 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.512 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.511+0000 7f2cfa90d640 1 -- 172.21.15.163:0/2050188294 >> 172.21.15.163:0/2050188294 conn(0x7f2cf40cdfd0 msgr2=0x7f2cf40715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:02.512 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.511+0000 7f2cfa90d640 1 -- 172.21.15.163:0/2050188294 shutdown_connections 2024-05-27T09:27:02.512 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.511+0000 7f2cfa90d640 1 -- 172.21.15.163:0/2050188294 wait complete. 2024-05-27T09:27:02.522 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 43a74d13-d03f-493a-a6c0-e73b0a056c10 6 2024-05-27T09:27:02.534 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:02.534+0000 7f98ecc38040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T09:27:02.661 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:02.661+0000 7f98ecc38040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T09:27:02.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.672+0000 7fdcf9001640 1 Processor -- start 2024-05-27T09:27:02.672 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.672+0000 7fdcf9001640 1 -- start start 2024-05-27T09:27:02.674 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.673+0000 7fdcf9001640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdcf4156960 0x7fdcf40c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:02.674 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.673+0000 7fdcf9001640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcf40ca830 0x7fdcf40cac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:02.674 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.673+0000 7fdcf9001640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcf40cb180 con 0x7fdcf40c38c0 2024-05-27T09:27:02.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.673+0000 7fdcf9001640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fdcf40ca2d0 con 0x7fdcf4156960 2024-05-27T09:27:02.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.673+0000 7fdcf9001640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf40c99e0 con 0x7fdcf40ca830 2024-05-27T09:27:02.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.673+0000 7fdcf259c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdcf4156960 0x7fdcf40c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:02.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.673+0000 7fdcf359e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcf40ca830 0x7fdcf40cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:02.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.673+0000 7fdcf259c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdcf4156960 0x7fdcf40c9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56678/0 (socket says 172.21.15.163:56678) 2024-05-27T09:27:02.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.673+0000 7fdcf259c640 1 -- 172.21.15.163:0/994836441 learned_addr learned my addr 172.21.15.163:0/994836441 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-27T09:27:02.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.673+0000 7fdcf259c640 1 -- 172.21.15.163:0/994836441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcf40ca830 msgr2=0x7fdcf40cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:02.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.673+0000 7fdcf259c640 1 --2- 172.21.15.163:0/994836441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcf40ca830 0x7fdcf40cac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.673+0000 7fdcf259c640 1 -- 172.21.15.163:0/994836441 >> v1:172.21.15.163:6789/0 conn(0x7fdcf40c38c0 legacy=0x7fdcf4070460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:02.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.673+0000 7fdcf1d9b640 1 -- 172.21.15.163:0/994836441 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3763743145 0 0) 0x7fdce8001110 con 0x7fdcf40c38c0 2024-05-27T09:27:02.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.673+0000 7fdcf259c640 1 -- 172.21.15.163:0/994836441 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf40c9700 con 0x7fdcf4156960 2024-05-27T09:27:02.675 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.673+0000 7fdcf359e640 1 --2- 172.21.15.163:0/994836441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcf40ca830 0x7fdcf40cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:02.677 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.674+0000 7fdcf259c640 1 --2- 172.21.15.163:0/994836441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcf4156960 0x7fdcf40c9d90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdce40098a0 tx=0x7fdce402a310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a26df718393d68bd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:02.677 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.675+0000 7fdcf1d9b640 1 -- 172.21.15.163:0/994836441 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4037070 con 0x7fdcf4156960 2024-05-27T09:27:02.677 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.675+0000 7fdcf1d9b640 1 -- 172.21.15.163:0/994836441 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdce4032640 con 0x7fdcf4156960 2024-05-27T09:27:02.677 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.675+0000 7fdcf1d9b640 1 -- 172.21.15.163:0/994836441 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4032a10 con 0x7fdcf4156960 2024-05-27T09:27:02.677 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.675+0000 7fdcf9001640 1 -- 172.21.15.163:0/994836441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcf4156960 msgr2=0x7fdcf40c9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:02.677 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.675+0000 7fdcf9001640 1 --2- 172.21.15.163:0/994836441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcf4156960 0x7fdcf40c9d90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdce40098a0 tx=0x7fdce402a310 comp rx=0 tx=0).stop 2024-05-27T09:27:02.677 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.675+0000 7fdcf9001640 1 -- 172.21.15.163:0/994836441 shutdown_connections 2024-05-27T09:27:02.677 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.675+0000 7fdcf9001640 1 --2- 172.21.15.163:0/994836441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcf40ca830 0x7fdcf40cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.677 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.675+0000 7fdcf9001640 1 --2- 172.21.15.163:0/994836441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcf4156960 0x7fdcf40c9d90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.677 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.675+0000 7fdcf9001640 1 -- 172.21.15.163:0/994836441 >> 172.21.15.163:0/994836441 conn(0x7fdcf40ce8a0 msgr2=0x7fdcf40cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:02.677 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.675+0000 7fdcf9001640 1 -- 172.21.15.163:0/994836441 shutdown_connections 2024-05-27T09:27:02.677 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.675+0000 7fdcf9001640 1 -- 172.21.15.163:0/994836441 wait complete. 2024-05-27T09:27:02.678 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.677+0000 7fdcf9001640 1 Processor -- start 2024-05-27T09:27:02.678 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.677+0000 7fdcf9001640 1 -- start start 2024-05-27T09:27:02.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.679+0000 7fdcf9001640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcf40c38c0 0x7fdcf4153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:02.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.679+0000 7fdcf9001640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcf40ca830 0x7fdcf4156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:02.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.679+0000 7fdcf9001640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcf4156960 0x7fdcf41f3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:02.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.679+0000 7fdcf9001640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf406cdf0 con 0x7fdcf40ca830 2024-05-27T09:27:02.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.679+0000 7fdcf9001640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf406cb10 con 0x7fdcf4156960 2024-05-27T09:27:02.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.679+0000 7fdcf9001640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf4070b70 con 0x7fdcf40c38c0 2024-05-27T09:27:02.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.679+0000 7fdcf359e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcf4156960 0x7fdcf41f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:02.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.679+0000 7fdcf2d9d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcf40c38c0 0x7fdcf4153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:02.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.679+0000 7fdcf359e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcf4156960 0x7fdcf41f3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56690/0 (socket says 172.21.15.163:56690) 2024-05-27T09:27:02.681 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.679+0000 7fdcf359e640 1 -- 172.21.15.163:0/3720981991 learned_addr learned my addr 172.21.15.163:0/3720981991 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-27T09:27:02.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.679+0000 7fdcf359e640 1 -- 172.21.15.163:0/3720981991 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcf40c38c0 msgr2=0x7fdcf4153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:02.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.679+0000 7fdcf359e640 1 --2- 172.21.15.163:0/3720981991 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcf40c38c0 0x7fdcf4153c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.679+0000 7fdcf359e640 1 -- 172.21.15.163:0/3720981991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcf40ca830 msgr2=0x7fdcf4156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:02.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.679+0000 7fdcf359e640 1 --2- 172.21.15.163:0/3720981991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcf40ca830 0x7fdcf4156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.679+0000 7fdcf359e640 1 -- 172.21.15.163:0/3720981991 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf41f35d0 con 0x7fdcf4156960 2024-05-27T09:27:02.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.679+0000 7fdcf2d9d640 1 --2- 172.21.15.163:0/3720981991 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcf40c38c0 0x7fdcf4153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:02.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.679+0000 7fdcf359e640 1 --2- 172.21.15.163:0/3720981991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcf4156960 0x7fdcf41f3090 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00efc0 tx=0x7fdcec00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:02.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.679+0000 7fdcdb7fe640 1 -- 172.21.15.163:0/3720981991 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec018030 con 0x7fdcf4156960 2024-05-27T09:27:02.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.680+0000 7fdcdb7fe640 1 -- 172.21.15.163:0/3720981991 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcec00cd30 con 0x7fdcf4156960 2024-05-27T09:27:02.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.680+0000 7fdcf9001640 1 -- 172.21.15.163:0/3720981991 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf41560b0 con 0x7fdcf4156960 2024-05-27T09:27:02.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.680+0000 7fdcdb7fe640 1 -- 172.21.15.163:0/3720981991 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec003bb0 con 0x7fdcf4156960 2024-05-27T09:27:02.682 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.680+0000 7fdcf9001640 1 -- 172.21.15.163:0/3720981991 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf41f65c0 con 0x7fdcf4156960 2024-05-27T09:27:02.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.680+0000 7fdcdb7fe640 1 -- 172.21.15.163:0/3720981991 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdcec01b070 con 0x7fdcf4156960 2024-05-27T09:27:02.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.680+0000 7fdcdb7fe640 1 -- 172.21.15.163:0/3720981991 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fdcec01daf0 con 0x7fdcf4156960 2024-05-27T09:27:02.683 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.681+0000 7fdcf9001640 1 -- 172.21.15.163:0/3720981991 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf4154270 con 0x7fdcf4156960 2024-05-27T09:27:02.685 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.685+0000 7fdcdb7fe640 1 -- 172.21.15.163:0/3720981991 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fdcec010040 con 0x7fdcf4156960 2024-05-27T09:27:02.729 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:02.728+0000 7fe8c8e3d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T09:27:02.758 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.758+0000 7fdcf9001640 1 -- 172.21.15.163:0/3720981991 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd new", "uuid": "43a74d13-d03f-493a-a6c0-e73b0a056c10", "id": 6} v 0) -- 0x7fdcf4069c50 con 0x7fdcf4156960 2024-05-27T09:27:02.765 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.765+0000 7fdcdb7fe640 1 -- 172.21.15.163:0/3720981991 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "43a74d13-d03f-493a-a6c0-e73b0a056c10", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fdcf4069c50 con 0x7fdcf4156960 2024-05-27T09:27:02.766 INFO:teuthology.orchestra.run.smithi163.stdout:6 2024-05-27T09:27:02.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.766+0000 7fdcf9001640 1 -- 172.21.15.163:0/3720981991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcf4156960 msgr2=0x7fdcf41f3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:02.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.766+0000 7fdcf9001640 1 --2- 172.21.15.163:0/3720981991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcf4156960 0x7fdcf41f3090 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00efc0 tx=0x7fdcec00c490 comp rx=0 tx=0).stop 2024-05-27T09:27:02.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.766+0000 7fdcf9001640 1 -- 172.21.15.163:0/3720981991 shutdown_connections 2024-05-27T09:27:02.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.766+0000 7fdcf9001640 1 --2- 172.21.15.163:0/3720981991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcf4156960 0x7fdcf41f3090 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.766+0000 7fdcf9001640 1 --2- 172.21.15.163:0/3720981991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcf40ca830 0x7fdcf4156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.766+0000 7fdcf9001640 1 --2- 172.21.15.163:0/3720981991 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcf40c38c0 0x7fdcf4153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.766+0000 7fdcf9001640 1 -- 172.21.15.163:0/3720981991 >> 172.21.15.163:0/3720981991 conn(0x7fdcf40ce8a0 msgr2=0x7fdcf414a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:02.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.766+0000 7fdcf9001640 1 -- 172.21.15.163:0/3720981991 shutdown_connections 2024-05-27T09:27:02.767 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.766+0000 7fdcf9001640 1 -- 172.21.15.163:0/3720981991 wait complete. 2024-05-27T09:27:02.777 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 134629c2-70af-4b84-83a6-7ddf53681baa 7 2024-05-27T09:27:02.879 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:02.878+0000 7fe8c8e3d040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T09:27:02.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.885+0000 7f8b4bf14640 1 Processor -- start 2024-05-27T09:27:02.886 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.885+0000 7f8b4bf14640 1 -- start start 2024-05-27T09:27:02.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.887+0000 7f8b4bf14640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b440c9fe0 0x7f8b440c5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:02.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.887+0000 7f8b4bf14640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8b440c9a90 0x7f8b44073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:02.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.887+0000 7f8b4bf14640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b4415d5e0 con 0x7f8b440c7500 2024-05-27T09:27:02.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.887+0000 7f8b4bf14640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f8b440c9340 con 0x7f8b440c9a90 2024-05-27T09:27:02.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.887+0000 7f8b4bf14640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f8b440c9870 con 0x7f8b440c9fe0 2024-05-27T09:27:02.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.887+0000 7f8b494af640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b440c9fe0 0x7f8b440c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:02.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.887+0000 7f8b4a4b1640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8b440c9a90 0x7f8b44073ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:02.888 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.887+0000 7f8b49cb0640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8b440c7500 0x7f8b441569d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:58828/0 (socket says 172.21.15.163:58828) 2024-05-27T09:27:02.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.887+0000 7f8b49cb0640 1 -- 172.21.15.163:0/3309525130 learned_addr learned my addr 172.21.15.163:0/3309525130 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-05-27T09:27:02.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.887+0000 7f8b494af640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b440c9fe0 0x7f8b440c5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:60466/0 (socket says 172.21.15.163:60466) 2024-05-27T09:27:02.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.887+0000 7f8b494af640 1 -- 172.21.15.163:0/3309525130 >> v2:172.21.15.163:3300/0 conn(0x7f8b440c9a90 msgr2=0x7f8b44073ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:02.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.887+0000 7f8b494af640 1 --2- 172.21.15.163:0/3309525130 >> v2:172.21.15.163:3300/0 conn(0x7f8b440c9a90 0x7f8b44073ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.887+0000 7f8b494af640 1 -- 172.21.15.163:0/3309525130 >> v1:172.21.15.163:6789/0 conn(0x7f8b440c7500 legacy=0x7f8b441569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:02.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.887+0000 7f8b494af640 1 -- 172.21.15.163:0/3309525130 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b440c9590 con 0x7f8b440c9fe0 2024-05-27T09:27:02.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.887+0000 7f8b48cae640 1 -- 172.21.15.163:0/3309525130 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2563654729 0 0) 0x7f8b4415d5e0 con 0x7f8b440c7500 2024-05-27T09:27:02.889 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.887+0000 7f8b4a4b1640 1 --2- 172.21.15.163:0/3309525130 >> v2:172.21.15.163:3300/0 conn(0x7f8b440c9a90 0x7f8b44073ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:02.890 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.888+0000 7f8b494af640 1 --2- 172.21.15.163:0/3309525130 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b440c9fe0 0x7f8b440c5990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8b400097a0 tx=0x7f8b4002a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16b73bfa54a8dedd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:02.890 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.888+0000 7f8b48cae640 1 -- 172.21.15.163:0/3309525130 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40037070 con 0x7f8b440c9fe0 2024-05-27T09:27:02.890 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.888+0000 7f8b48cae640 1 -- 172.21.15.163:0/3309525130 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b4002adb0 con 0x7f8b440c9fe0 2024-05-27T09:27:02.891 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.888+0000 7f8b48cae640 1 -- 172.21.15.163:0/3309525130 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40032660 con 0x7f8b440c9fe0 2024-05-27T09:27:02.891 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.889+0000 7f8b4bf14640 1 -- 172.21.15.163:0/3309525130 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b440c9fe0 msgr2=0x7f8b440c5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:02.891 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.889+0000 7f8b4bf14640 1 --2- 172.21.15.163:0/3309525130 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b440c9fe0 0x7f8b440c5990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8b400097a0 tx=0x7f8b4002a060 comp rx=0 tx=0).stop 2024-05-27T09:27:02.891 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.889+0000 7f8b4bf14640 1 -- 172.21.15.163:0/3309525130 shutdown_connections 2024-05-27T09:27:02.891 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.889+0000 7f8b4bf14640 1 --2- 172.21.15.163:0/3309525130 >> v2:172.21.15.163:3300/0 conn(0x7f8b440c9a90 0x7f8b44073ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.891 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.889+0000 7f8b4bf14640 1 --2- 172.21.15.163:0/3309525130 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b440c9fe0 0x7f8b440c5990 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.891 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.889+0000 7f8b4bf14640 1 -- 172.21.15.163:0/3309525130 >> 172.21.15.163:0/3309525130 conn(0x7f8b440cade0 msgr2=0x7f8b440cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:02.891 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.889+0000 7f8b4bf14640 1 -- 172.21.15.163:0/3309525130 shutdown_connections 2024-05-27T09:27:02.891 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.889+0000 7f8b4bf14640 1 -- 172.21.15.163:0/3309525130 wait complete. 2024-05-27T09:27:02.892 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.891+0000 7f8b4bf14640 1 Processor -- start 2024-05-27T09:27:02.892 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.891+0000 7f8b4bf14640 1 -- start start 2024-05-27T09:27:02.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.892+0000 7f8b4bf14640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b440c7500 0x7f8b44158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:02.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.892+0000 7f8b4bf14640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b440c9a90 0x7f8b4414c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:02.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.892+0000 7f8b4bf14640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b440c9fe0 0x7f8b441f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:02.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.892+0000 7f8b4bf14640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4415d270 con 0x7f8b440c7500 2024-05-27T09:27:02.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.892+0000 7f8b4bf14640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8b440ca9d0 con 0x7f8b440c9fe0 2024-05-27T09:27:02.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.892+0000 7f8b4bf14640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f8b440c5ed0 con 0x7f8b440c9a90 2024-05-27T09:27:02.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.892+0000 7f8b4a4b1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b440c9fe0 0x7f8b441f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:02.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.892+0000 7f8b494af640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b440c9a90 0x7f8b4414c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:02.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.892+0000 7f8b4a4b1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b440c9fe0 0x7f8b441f5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56720/0 (socket says 172.21.15.163:56720) 2024-05-27T09:27:02.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.892+0000 7f8b494af640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b440c9a90 0x7f8b4414c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:60480/0 (socket says 172.21.15.163:60480) 2024-05-27T09:27:02.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.892+0000 7f8b4a4b1640 1 -- 172.21.15.163:0/1931432324 learned_addr learned my addr 172.21.15.163:0/1931432324 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-05-27T09:27:02.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.892+0000 7f8b494af640 1 -- 172.21.15.163:0/1931432324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b440c9fe0 msgr2=0x7f8b441f5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:02.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.892+0000 7f8b494af640 1 --2- 172.21.15.163:0/1931432324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b440c9fe0 0x7f8b441f5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.892+0000 7f8b494af640 1 -- 172.21.15.163:0/1931432324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b440c7500 msgr2=0x7f8b44158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:02.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.892+0000 7f8b494af640 1 --2- 172.21.15.163:0/1931432324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b440c7500 0x7f8b44158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.892+0000 7f8b494af640 1 -- 172.21.15.163:0/1931432324 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b44158870 con 0x7f8b440c9a90 2024-05-27T09:27:02.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.893+0000 7f8b494af640 1 --2- 172.21.15.163:0/1931432324 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b440c9a90 0x7f8b4414c1e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8b40031000 tx=0x7f8b400046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:02.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.893+0000 7f8b3a7fc640 1 -- 172.21.15.163:0/1931432324 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40041070 con 0x7f8b440c9a90 2024-05-27T09:27:02.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.893+0000 7f8b3a7fc640 1 -- 172.21.15.163:0/1931432324 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b40032e20 con 0x7f8b440c9a90 2024-05-27T09:27:02.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.893+0000 7f8b4bf14640 1 -- 172.21.15.163:0/1931432324 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b44157330 con 0x7f8b440c9a90 2024-05-27T09:27:02.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.893+0000 7f8b3a7fc640 1 -- 172.21.15.163:0/1931432324 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40037040 con 0x7f8b440c9a90 2024-05-27T09:27:02.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.893+0000 7f8b4bf14640 1 -- 172.21.15.163:0/1931432324 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b44157b30 con 0x7f8b440c9a90 2024-05-27T09:27:02.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.893+0000 7f8b3a7fc640 1 -- 172.21.15.163:0/1931432324 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8b40044070 con 0x7f8b440c9a90 2024-05-27T09:27:02.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.893+0000 7f8b3a7fc640 1 -- 172.21.15.163:0/1931432324 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f8b40046cd0 con 0x7f8b440c9a90 2024-05-27T09:27:02.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.894+0000 7f8b4bf14640 1 -- 172.21.15.163:0/1931432324 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b4406a380 con 0x7f8b440c9a90 2024-05-27T09:27:02.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.898+0000 7f8b3a7fc640 1 -- 172.21.15.163:0/1931432324 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f8b40046630 con 0x7f8b440c9a90 2024-05-27T09:27:02.971 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.970+0000 7f8b4bf14640 1 -- 172.21.15.163:0/1931432324 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd new", "uuid": "134629c2-70af-4b84-83a6-7ddf53681baa", "id": 7} v 0) -- 0x7f8b440c6ce0 con 0x7f8b440c9a90 2024-05-27T09:27:02.978 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.977+0000 7f8b3a7fc640 1 -- 172.21.15.163:0/1931432324 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "134629c2-70af-4b84-83a6-7ddf53681baa", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f8b40046810 con 0x7f8b440c9a90 2024-05-27T09:27:02.978 INFO:teuthology.orchestra.run.smithi163.stdout:7 2024-05-27T09:27:02.979 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.978+0000 7f8b4bf14640 1 -- 172.21.15.163:0/1931432324 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b440c9a90 msgr2=0x7f8b4414c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:02.979 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.978+0000 7f8b4bf14640 1 --2- 172.21.15.163:0/1931432324 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b440c9a90 0x7f8b4414c1e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8b40031000 tx=0x7f8b400046c0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.979 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.978+0000 7f8b4bf14640 1 -- 172.21.15.163:0/1931432324 shutdown_connections 2024-05-27T09:27:02.979 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.978+0000 7f8b4bf14640 1 --2- 172.21.15.163:0/1931432324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b440c9fe0 0x7f8b441f5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.978+0000 7f8b4bf14640 1 --2- 172.21.15.163:0/1931432324 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b440c9a90 0x7f8b4414c1e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.978+0000 7f8b4bf14640 1 --2- 172.21.15.163:0/1931432324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b440c7500 0x7f8b44158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:02.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.978+0000 7f8b4bf14640 1 -- 172.21.15.163:0/1931432324 >> 172.21.15.163:0/1931432324 conn(0x7f8b440cade0 msgr2=0x7f8b441569d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:02.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.978+0000 7f8b4bf14640 1 -- 172.21.15.163:0/1931432324 shutdown_connections 2024-05-27T09:27:02.980 INFO:teuthology.orchestra.run.smithi163.stderr:2024-05-27T09:27:02.978+0000 7f8b4bf14640 1 -- 172.21.15.163:0/1931432324 wait complete. 2024-05-27T09:27:02.990 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-27T09:27:02.990 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-27T09:27:02.997 INFO:tasks.ceph.osd.0:Started 2024-05-27T09:27:02.997 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-27T09:27:02.997 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-27T09:27:03.004 INFO:tasks.ceph.osd.1:Started 2024-05-27T09:27:03.004 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-27T09:27:03.004 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-27T09:27:03.009 INFO:tasks.ceph.osd.2:Started 2024-05-27T09:27:03.010 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-27T09:27:03.010 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-27T09:27:03.014 INFO:tasks.ceph.osd.3:Started 2024-05-27T09:27:03.015 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-27T09:27:03.015 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-27T09:27:03.019 INFO:tasks.ceph.osd.4:Started 2024-05-27T09:27:03.019 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-27T09:27:03.019 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-27T09:27:03.021 INFO:tasks.ceph.osd.5:Started 2024-05-27T09:27:03.021 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-27T09:27:03.021 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-27T09:27:03.023 INFO:tasks.ceph.osd.6:Started 2024-05-27T09:27:03.023 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-27T09:27:03.024 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-27T09:27:03.025 INFO:tasks.ceph.osd.7:Started 2024-05-27T09:27:03.026 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T09:27:03.168 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:03.168+0000 7f98ecc38040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T09:27:03.173 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:03.172+0000 7fe8c8e3d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T09:27:03.224 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:03.224+0000 7fe8c8e3d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T09:27:03.346 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:03.345+0000 7fe8c8e3d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T09:27:03.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.356+0000 7f9ec8400640 1 Processor -- start 2024-05-27T09:27:03.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.356+0000 7f9ec8400640 1 -- start start 2024-05-27T09:27:03.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.358+0000 7f9ec8400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec0160990 0x7f9ec015d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:03.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.358+0000 7f9ec8400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9ec00c73d0 0x7f9ec00c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:03.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.358+0000 7f9ec8400640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ec00cc6d0 con 0x7f9ec01566a0 2024-05-27T09:27:03.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.358+0000 7f9ec8400640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f9ec015fc60 con 0x7f9ec00c73d0 2024-05-27T09:27:03.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.358+0000 7f9ec8400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f9ec0160500 con 0x7f9ec0160990 2024-05-27T09:27:03.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.358+0000 7f9ec5a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec0160990 0x7f9ec015d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:03.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.358+0000 7f9ec5000640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f9ec01566a0 0x7f9ec00b54d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:44554/0 (socket says 172.21.15.114:44554) 2024-05-27T09:27:03.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.358+0000 7f9ec6400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9ec00c73d0 0x7f9ec00c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:03.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.358+0000 7f9ec5000640 1 -- 172.21.15.114:0/3540392841 learned_addr learned my addr 172.21.15.114:0/3540392841 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:03.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.358+0000 7f9ec6400640 1 -- 172.21.15.114:0/3540392841 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec0160990 msgr2=0x7f9ec015d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:03.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.358+0000 7f9ebfe00640 1 -- 172.21.15.114:0/3540392841 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3029743188 0 0) 0x7f9ec00cc6d0 con 0x7f9ec01566a0 2024-05-27T09:27:03.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.358+0000 7f9ec6400640 1 --2- 172.21.15.114:0/3540392841 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec0160990 0x7f9ec015d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:03.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.358+0000 7f9ec6400640 1 -- 172.21.15.114:0/3540392841 >> v1:172.21.15.114:6789/0 conn(0x7f9ec01566a0 legacy=0x7f9ec00b54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:03.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.358+0000 7f9ec6400640 1 -- 172.21.15.114:0/3540392841 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec00cc970 con 0x7f9ec00c73d0 2024-05-27T09:27:03.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.358+0000 7f9ec5a00640 1 --2- 172.21.15.114:0/3540392841 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec0160990 0x7f9ec015d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:03.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.359+0000 7f9ec6400640 1 --2- 172.21.15.114:0/3540392841 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ec00c73d0 0x7f9ec00c77e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8009d90 tx=0x7f9eb802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf5fa543c33a6326 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:03.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.360+0000 7f9ebfe00640 1 -- 172.21.15.114:0/3540392841 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8007740 con 0x7f9ec00c73d0 2024-05-27T09:27:03.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.360+0000 7f9ebfe00640 1 -- 172.21.15.114:0/3540392841 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eb80078e0 con 0x7f9ec00c73d0 2024-05-27T09:27:03.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.360+0000 7f9ebfe00640 1 -- 172.21.15.114:0/3540392841 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8007cb0 con 0x7f9ec00c73d0 2024-05-27T09:27:03.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.360+0000 7f9ec8400640 1 -- 172.21.15.114:0/3540392841 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ec00c73d0 msgr2=0x7f9ec00c77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:03.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.360+0000 7f9ec8400640 1 --2- 172.21.15.114:0/3540392841 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ec00c73d0 0x7f9ec00c77e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8009d90 tx=0x7f9eb802a5f0 comp rx=0 tx=0).stop 2024-05-27T09:27:03.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.360+0000 7f9ec8400640 1 -- 172.21.15.114:0/3540392841 shutdown_connections 2024-05-27T09:27:03.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.360+0000 7f9ec8400640 1 --2- 172.21.15.114:0/3540392841 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ec00c73d0 0x7f9ec00c77e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:03.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.360+0000 7f9ec8400640 1 --2- 172.21.15.114:0/3540392841 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec0160990 0x7f9ec015d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:03.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.360+0000 7f9ec8400640 1 -- 172.21.15.114:0/3540392841 >> 172.21.15.114:0/3540392841 conn(0x7f9ec006f340 msgr2=0x7f9ec0073dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:03.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.361+0000 7f9ec8400640 1 -- 172.21.15.114:0/3540392841 shutdown_connections 2024-05-27T09:27:03.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.361+0000 7f9ec8400640 1 -- 172.21.15.114:0/3540392841 wait complete. 2024-05-27T09:27:03.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.363+0000 7f9ec8400640 1 Processor -- start 2024-05-27T09:27:03.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.363+0000 7f9ec8400640 1 -- start start 2024-05-27T09:27:03.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.364+0000 7f9ec8400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ec00c73d0 0x7f9ec01e81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:03.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.364+0000 7f9ec8400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ec01566a0 0x7f9ec01f6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:03.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.364+0000 7f9ec8400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec0155b60 0x7f9ec014f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:03.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.364+0000 7f9ec8400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec0160190 con 0x7f9ec01566a0 2024-05-27T09:27:03.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.364+0000 7f9ec8400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec015feb0 con 0x7f9ec00c73d0 2024-05-27T09:27:03.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.364+0000 7f9ec8400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f9ec00c7090 con 0x7f9ec0155b60 2024-05-27T09:27:03.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.364+0000 7f9ec6400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec0155b60 0x7f9ec014f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:03.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.365+0000 7f9ec5a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ec00c73d0 0x7f9ec01e81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:03.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.365+0000 7f9ec6400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec0155b60 0x7f9ec014f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:40620/0 (socket says 172.21.15.114:40620) 2024-05-27T09:27:03.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.365+0000 7f9ec6400640 1 -- 172.21.15.114:0/592601270 learned_addr learned my addr 172.21.15.114:0/592601270 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:03.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.365+0000 7f9ec5000640 1 --2- 172.21.15.114:0/592601270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ec01566a0 0x7f9ec01f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:03.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.365+0000 7f9ec6400640 1 -- 172.21.15.114:0/592601270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ec00c73d0 msgr2=0x7f9ec01e81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:03.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.365+0000 7f9ec6400640 1 --2- 172.21.15.114:0/592601270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ec00c73d0 0x7f9ec01e81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:03.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.365+0000 7f9ec6400640 1 -- 172.21.15.114:0/592601270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ec01566a0 msgr2=0x7f9ec01f6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:03.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.365+0000 7f9ec6400640 1 --2- 172.21.15.114:0/592601270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ec01566a0 0x7f9ec01f6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:03.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.365+0000 7f9ec6400640 1 -- 172.21.15.114:0/592601270 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec01e8700 con 0x7f9ec0155b60 2024-05-27T09:27:03.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.365+0000 7f9ec6400640 1 --2- 172.21.15.114:0/592601270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec0155b60 0x7f9ec014f310 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8009eb0 tx=0x7f9eb8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:03.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.365+0000 7f9ebd600640 1 -- 172.21.15.114:0/592601270 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb803f070 con 0x7f9ec0155b60 2024-05-27T09:27:03.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.366+0000 7f9ebd600640 1 -- 172.21.15.114:0/592601270 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eb8031be0 con 0x7f9ec0155b60 2024-05-27T09:27:03.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.366+0000 7f9ec8400640 1 -- 172.21.15.114:0/592601270 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec01f6740 con 0x7f9ec0155b60 2024-05-27T09:27:03.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.366+0000 7f9ebd600640 1 -- 172.21.15.114:0/592601270 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8009410 con 0x7f9ec0155b60 2024-05-27T09:27:03.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.366+0000 7f9ec8400640 1 -- 172.21.15.114:0/592601270 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec01e6d00 con 0x7f9ec0155b60 2024-05-27T09:27:03.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.366+0000 7f9ebd600640 1 -- 172.21.15.114:0/592601270 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9eb80437a0 con 0x7f9ec0155b60 2024-05-27T09:27:03.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.366+0000 7f9ebd600640 1 -- 172.21.15.114:0/592601270 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f9eb803b410 con 0x7f9ec0155b60 2024-05-27T09:27:03.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.367+0000 7f9ec8400640 1 -- 172.21.15.114:0/592601270 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ec00cf490 con 0x7f9ec0155b60 2024-05-27T09:27:03.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.369+0000 7f9ebd600640 1 -- 172.21.15.114:0/592601270 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f9ec00cf490 con 0x7f9ec0155b60 2024-05-27T09:27:03.386 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:03.386+0000 7f98ecc38040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T09:27:03.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.433+0000 7f9ec8400640 1 -- 172.21.15.114:0/592601270 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ec00cbf10 con 0x7f9ec0155b60 2024-05-27T09:27:03.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.434+0000 7f9ebd600640 1 -- 172.21.15.114:0/592601270 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f9eb8007840 con 0x7f9ec0155b60 2024-05-27T09:27:03.435 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:03.435 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":10,"fsid":"2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48","created":"2024-05-27T09:27:00.307278+0000","modified":"2024-05-27T09:27:02.972880+0000","last_up_change":"0.000000","last_in_change":"2024-05-27T09:27:02.972880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3561c750-1d63-42f6-a6ae-0ee63721e15f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9a407d04-38c0-4676-928f-d04e8116bd19","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"255a1cc6-9357-43f7-9db5-d760f4806ab2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cb4d7692-9a17-41b9-927e-d8eac3e217c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cdef1940-3453-4fbd-b0f1-459f5f5c7142","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"220a7b32-abcc-492f-bca8-4615cc2ea111","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"43a74d13-d03f-493a-a6c0-e73b0a056c10","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"134629c2-70af-4b84-83a6-7ddf53681baa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T09:27:03.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.436+0000 7f9ec8400640 1 -- 172.21.15.114:0/592601270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec0155b60 msgr2=0x7f9ec014f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:03.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.436+0000 7f9ec8400640 1 --2- 172.21.15.114:0/592601270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec0155b60 0x7f9ec014f310 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8009eb0 tx=0x7f9eb8004a40 comp rx=0 tx=0).stop 2024-05-27T09:27:03.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.436+0000 7f9ec8400640 1 -- 172.21.15.114:0/592601270 shutdown_connections 2024-05-27T09:27:03.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.436+0000 7f9ec8400640 1 --2- 172.21.15.114:0/592601270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ec0155b60 0x7f9ec014f310 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:03.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.436+0000 7f9ec8400640 1 --2- 172.21.15.114:0/592601270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ec01566a0 0x7f9ec01f6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:03.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.436+0000 7f9ec8400640 1 --2- 172.21.15.114:0/592601270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ec00c73d0 0x7f9ec01e81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:03.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.436+0000 7f9ec8400640 1 -- 172.21.15.114:0/592601270 >> 172.21.15.114:0/592601270 conn(0x7f9ec006f340 msgr2=0x7f9ec008e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:03.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.436+0000 7f9ec8400640 1 -- 172.21.15.114:0/592601270 shutdown_connections 2024-05-27T09:27:03.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:03.436+0000 7f9ec8400640 1 -- 172.21.15.114:0/592601270 wait complete. 2024-05-27T09:27:03.457 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-27T09:27:03.457 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-27T09:27:03.628 INFO:tasks.ceph.osd.5.smithi163.stderr:2024-05-27T09:27:03.627+0000 7fe64720c640 -1 Falling back to public interface 2024-05-27T09:27:03.628 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:03.628+0000 7fe8c8e3d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T09:27:03.645 INFO:tasks.ceph.osd.6.smithi163.stderr:2024-05-27T09:27:03.644+0000 7fd45f321640 -1 Falling back to public interface 2024-05-27T09:27:03.654 INFO:tasks.ceph.osd.4.smithi163.stderr:2024-05-27T09:27:03.653+0000 7f6b2e5b1640 -1 Falling back to public interface 2024-05-27T09:27:03.663 INFO:tasks.ceph.osd.7.smithi163.stderr:2024-05-27T09:27:03.663+0000 7f8945c9d640 -1 Falling back to public interface 2024-05-27T09:27:03.682 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:03.682+0000 7fe8c8e3d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T09:27:03.691 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:03.690+0000 7f98ecc38040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T09:27:03.751 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:03.751+0000 7f98ecc38040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T09:27:03.793 INFO:tasks.ceph.osd.3.smithi114.stderr:2024-05-27T09:27:03.793+0000 7fd6921c0640 -1 Falling back to public interface 2024-05-27T09:27:03.802 INFO:tasks.ceph.osd.2.smithi114.stderr:2024-05-27T09:27:03.802+0000 7fca89933640 -1 Falling back to public interface 2024-05-27T09:27:03.840 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:03.839+0000 7fe8c8e3d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T09:27:03.844 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-05-27T09:27:03.844+0000 7f9c24114640 -1 Falling back to public interface 2024-05-27T09:27:03.853 INFO:tasks.ceph.osd.1.smithi114.stderr:2024-05-27T09:27:03.853+0000 7fbf7401e640 -1 Falling back to public interface 2024-05-27T09:27:03.889 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:03.889+0000 7f98ecc38040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T09:27:03.979 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:03.979+0000 7fe8c8e3d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T09:27:04.030 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:04.029+0000 7fe8c8e3d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T09:27:04.099 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:04.098+0000 7fe8c8e3d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T09:27:04.144 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:04.143+0000 7fe8c8e3d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T09:27:04.227 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:04.226+0000 7f98ecc38040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T09:27:04.262 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-27T09:27:04.283 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:04.284+0000 7f98ecc38040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T09:27:04.478 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:04.478+0000 7f98ecc38040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T09:27:04.487 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.487+0000 7f111d600640 1 Processor -- start 2024-05-27T09:27:04.487 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.487+0000 7f111d600640 1 -- start start 2024-05-27T09:27:04.489 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.489+0000 7f111d600640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f11180c8490 0x7f111815d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:04.490 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.489+0000 7f111d600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11180c3d30 0x7f111806fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:04.490 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.489+0000 7f111d600640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f111815a580 con 0x7f11180c9d30 2024-05-27T09:27:04.490 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.489+0000 7f111d600640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f11180c8c20 con 0x7f11180c8490 2024-05-27T09:27:04.490 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.489+0000 7f111d600640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f111815dab0 con 0x7f11180c3d30 2024-05-27T09:27:04.490 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.489+0000 7f1116a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f11180c8490 0x7f111815d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:04.490 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.489+0000 7f1116a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f11180c8490 0x7f111815d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41756/0 (socket says 172.21.15.114:41756) 2024-05-27T09:27:04.490 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.489+0000 7f1116a00640 1 -- 172.21.15.114:0/897186138 learned_addr learned my addr 172.21.15.114:0/897186138 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:04.490 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.489+0000 7f1116000640 1 --1- 172.21.15.114:0/897186138 >> v1:172.21.15.163:6789/0 conn(0x7f11180c9d30 0x7f11180c2920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:59170/0 (socket says 172.21.15.114:59170) 2024-05-27T09:27:04.490 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.489+0000 7f1117400640 1 --2- 172.21.15.114:0/897186138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11180c3d30 0x7f111806fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:04.490 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.489+0000 7f1116a00640 1 -- 172.21.15.114:0/897186138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11180c3d30 msgr2=0x7f111806fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:04.490 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.489+0000 7f1116a00640 1 --2- 172.21.15.114:0/897186138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11180c3d30 0x7f111806fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:04.490 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.489+0000 7f1116a00640 1 -- 172.21.15.114:0/897186138 >> v1:172.21.15.163:6789/0 conn(0x7f11180c9d30 legacy=0x7f11180c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:04.491 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.489+0000 7f1115600640 1 -- 172.21.15.114:0/897186138 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 330489644 0 0) 0x7f111815a580 con 0x7f11180c9d30 2024-05-27T09:27:04.491 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.489+0000 7f1116a00640 1 -- 172.21.15.114:0/897186138 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1118159800 con 0x7f11180c8490 2024-05-27T09:27:04.492 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.490+0000 7f1116a00640 1 --2- 172.21.15.114:0/897186138 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11180c8490 0x7f111815d570 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f110800bb10 tx=0x7f110802c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99409b695c3651bd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:04.492 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.491+0000 7f1115600640 1 -- 172.21.15.114:0/897186138 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1108035070 con 0x7f11180c8490 2024-05-27T09:27:04.492 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.491+0000 7f1115600640 1 -- 172.21.15.114:0/897186138 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1108031610 con 0x7f11180c8490 2024-05-27T09:27:04.492 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.491+0000 7f1115600640 1 -- 172.21.15.114:0/897186138 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11080319e0 con 0x7f11180c8490 2024-05-27T09:27:04.492 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.491+0000 7f111d600640 1 -- 172.21.15.114:0/897186138 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11180c8490 msgr2=0x7f111815d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:04.492 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.491+0000 7f111d600640 1 --2- 172.21.15.114:0/897186138 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11180c8490 0x7f111815d570 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f110800bb10 tx=0x7f110802c9b0 comp rx=0 tx=0).stop 2024-05-27T09:27:04.492 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.492+0000 7f111d600640 1 -- 172.21.15.114:0/897186138 shutdown_connections 2024-05-27T09:27:04.492 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.492+0000 7f111d600640 1 --2- 172.21.15.114:0/897186138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11180c3d30 0x7f111806fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:04.492 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.492+0000 7f111d600640 1 --2- 172.21.15.114:0/897186138 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11180c8490 0x7f111815d570 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:04.492 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.492+0000 7f111d600640 1 -- 172.21.15.114:0/897186138 >> 172.21.15.114:0/897186138 conn(0x7f1118070d20 msgr2=0x7f1118071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:04.492 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.492+0000 7f111d600640 1 -- 172.21.15.114:0/897186138 shutdown_connections 2024-05-27T09:27:04.493 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.492+0000 7f111d600640 1 -- 172.21.15.114:0/897186138 wait complete. 2024-05-27T09:27:04.494 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.494+0000 7f111d600640 1 Processor -- start 2024-05-27T09:27:04.494 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.494+0000 7f111d600640 1 -- start start 2024-05-27T09:27:04.497 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.495+0000 7f111d600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11180c3d30 0x7f11181ec1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:04.497 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.495+0000 7f111d600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11180c8490 0x7f1118156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:04.497 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.495+0000 7f111d600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11180c9d30 0x7f1118156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:04.498 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.495+0000 7f111d600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1118159d80 con 0x7f11180c9d30 2024-05-27T09:27:04.498 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.495+0000 7f111d600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1118159aa0 con 0x7f11180c3d30 2024-05-27T09:27:04.498 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.495+0000 7f111d600640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f111815a060 con 0x7f11180c8490 2024-05-27T09:27:04.498 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.496+0000 7f1116000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11180c8490 0x7f1118156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:04.498 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.496+0000 7f1117400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11180c9d30 0x7f1118156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:04.498 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.496+0000 7f1116a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11180c3d30 0x7f11181ec1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:04.498 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.496+0000 7f1116a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11180c3d30 0x7f11181ec1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:51770/0 (socket says 172.21.15.114:51770) 2024-05-27T09:27:04.498 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.496+0000 7f1116000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11180c8490 0x7f1118156400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:40642/0 (socket says 172.21.15.114:40642) 2024-05-27T09:27:04.498 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.496+0000 7f1116a00640 1 -- 172.21.15.114:0/3551907859 learned_addr learned my addr 172.21.15.114:0/3551907859 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:04.498 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.496+0000 7f1117400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11180c9d30 0x7f1118156940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41770/0 (socket says 172.21.15.114:41770) 2024-05-27T09:27:04.498 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.496+0000 7f1116000640 1 -- 172.21.15.114:0/3551907859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11180c3d30 msgr2=0x7f11181ec1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:04.498 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.496+0000 7f1116000640 1 --2- 172.21.15.114:0/3551907859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11180c3d30 0x7f11181ec1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:04.499 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.496+0000 7f1116000640 1 -- 172.21.15.114:0/3551907859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11180c9d30 msgr2=0x7f1118156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:04.499 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.496+0000 7f1116000640 1 --2- 172.21.15.114:0/3551907859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11180c9d30 0x7f1118156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:04.499 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.496+0000 7f1116000640 1 -- 172.21.15.114:0/3551907859 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11181f2540 con 0x7f11180c8490 2024-05-27T09:27:04.499 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.496+0000 7f1116a00640 1 --2- 172.21.15.114:0/3551907859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11180c3d30 0x7f11181ec1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:04.499 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.496+0000 7f1117400640 1 --2- 172.21.15.114:0/3551907859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11180c9d30 0x7f1118156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:04.499 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.496+0000 7f1116000640 1 --2- 172.21.15.114:0/3551907859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11180c8490 0x7f1118156400 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f11040016f0 tx=0x7f1104001bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:04.499 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.497+0000 7f110ea00640 1 -- 172.21.15.114:0/3551907859 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1104001ce0 con 0x7f11180c8490 2024-05-27T09:27:04.499 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.497+0000 7f110ea00640 1 -- 172.21.15.114:0/3551907859 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1104007070 con 0x7f11180c8490 2024-05-27T09:27:04.499 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.497+0000 7f111d600640 1 -- 172.21.15.114:0/3551907859 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1118154bc0 con 0x7f11180c8490 2024-05-27T09:27:04.499 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.497+0000 7f110ea00640 1 -- 172.21.15.114:0/3551907859 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1104007460 con 0x7f11180c8490 2024-05-27T09:27:04.499 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.497+0000 7f111d600640 1 -- 172.21.15.114:0/3551907859 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1118155140 con 0x7f11180c8490 2024-05-27T09:27:04.499 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.497+0000 7f110ea00640 1 -- 172.21.15.114:0/3551907859 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1104004b70 con 0x7f11180c8490 2024-05-27T09:27:04.499 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.497+0000 7f110ea00640 1 -- 172.21.15.114:0/3551907859 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f1104007600 con 0x7f11180c8490 2024-05-27T09:27:04.500 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.498+0000 7f111d600640 1 -- 172.21.15.114:0/3551907859 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f111806a380 con 0x7f11180c8490 2024-05-27T09:27:04.500 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.500+0000 7f110ea00640 1 -- 172.21.15.114:0/3551907859 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f1104010630 con 0x7f11180c8490 2024-05-27T09:27:04.565 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.564+0000 7f111d600640 1 -- 172.21.15.114:0/3551907859 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11180c4ab0 con 0x7f11180c8490 2024-05-27T09:27:04.566 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.566+0000 7f110ea00640 1 -- 172.21.15.114:0/3551907859 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f1104010810 con 0x7f11180c8490 2024-05-27T09:27:04.566 INFO:teuthology.misc.health.smithi114.stdout: 2024-05-27T09:27:04.566 INFO:teuthology.misc.health.smithi114.stdout:{"epoch":10,"fsid":"2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48","created":"2024-05-27T09:27:00.307278+0000","modified":"2024-05-27T09:27:02.972880+0000","last_up_change":"0.000000","last_in_change":"2024-05-27T09:27:02.972880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3561c750-1d63-42f6-a6ae-0ee63721e15f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9a407d04-38c0-4676-928f-d04e8116bd19","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"255a1cc6-9357-43f7-9db5-d760f4806ab2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cb4d7692-9a17-41b9-927e-d8eac3e217c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cdef1940-3453-4fbd-b0f1-459f5f5c7142","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"220a7b32-abcc-492f-bca8-4615cc2ea111","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"43a74d13-d03f-493a-a6c0-e73b0a056c10","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"134629c2-70af-4b84-83a6-7ddf53681baa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T09:27:04.567 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.567+0000 7f111d600640 1 -- 172.21.15.114:0/3551907859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11180c8490 msgr2=0x7f1118156400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:04.567 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.567+0000 7f111d600640 1 --2- 172.21.15.114:0/3551907859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11180c8490 0x7f1118156400 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f11040016f0 tx=0x7f1104001bb0 comp rx=0 tx=0).stop 2024-05-27T09:27:04.567 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.567+0000 7f111d600640 1 -- 172.21.15.114:0/3551907859 shutdown_connections 2024-05-27T09:27:04.567 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.567+0000 7f111d600640 1 --2- 172.21.15.114:0/3551907859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11180c9d30 0x7f1118156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:04.567 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.567+0000 7f111d600640 1 --2- 172.21.15.114:0/3551907859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11180c8490 0x7f1118156400 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:04.567 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.567+0000 7f111d600640 1 --2- 172.21.15.114:0/3551907859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11180c3d30 0x7f11181ec1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:04.568 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.567+0000 7f111d600640 1 -- 172.21.15.114:0/3551907859 >> 172.21.15.114:0/3551907859 conn(0x7f1118070d20 msgr2=0x7f111806e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:04.568 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.567+0000 7f111d600640 1 -- 172.21.15.114:0/3551907859 shutdown_connections 2024-05-27T09:27:04.568 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:04.567+0000 7f111d600640 1 -- 172.21.15.114:0/3551907859 wait complete. 2024-05-27T09:27:04.585 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-27T09:27:04.628 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:04.628+0000 7fe8c8e3d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T09:27:04.654 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:04.654+0000 7f98ecc38040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T09:27:04.714 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:04.714+0000 7f98ecc38040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T09:27:04.747 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:04.747+0000 7fe8c8e3d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T09:27:04.802 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:04.802+0000 7f98ecc38040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T09:27:04.855 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:04.855+0000 7f98ecc38040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T09:27:05.401 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:05.401+0000 7f98ecc38040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T09:27:05.536 INFO:tasks.ceph.mgr.y.smithi114.stderr:2024-05-27T09:27:05.536+0000 7f98ecc38040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T09:27:05.775 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:05.775+0000 7fe868018640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-27T09:27:05.776 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-05-27T09:27:05.776+0000 7fe868018640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-27T09:27:07.078 INFO:tasks.ceph.osd.5.smithi163.stderr:2024-05-27T09:27:07.078+0000 7fe64720c640 -1 osd.5 0 log_to_monitors true 2024-05-27T09:27:07.162 INFO:tasks.ceph.osd.6.smithi163.stderr:2024-05-27T09:27:07.161+0000 7fd45f321640 -1 osd.6 0 log_to_monitors true 2024-05-27T09:27:07.164 INFO:tasks.ceph.osd.7.smithi163.stderr:2024-05-27T09:27:07.163+0000 7f8945c9d640 -1 osd.7 0 log_to_monitors true 2024-05-27T09:27:07.168 INFO:tasks.ceph.osd.4.smithi163.stderr:2024-05-27T09:27:07.168+0000 7f6b2e5b1640 -1 osd.4 0 log_to_monitors true 2024-05-27T09:27:07.380 INFO:tasks.ceph.osd.3.smithi114.stderr:2024-05-27T09:27:07.380+0000 7fd6921c0640 -1 osd.3 0 log_to_monitors true 2024-05-27T09:27:07.424 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-05-27T09:27:07.424+0000 7f9c24114640 -1 osd.0 0 log_to_monitors true 2024-05-27T09:27:07.481 INFO:tasks.ceph.osd.1.smithi114.stderr:2024-05-27T09:27:07.481+0000 7fbf7401e640 -1 osd.1 0 log_to_monitors true 2024-05-27T09:27:07.642 INFO:tasks.ceph.osd.2.smithi114.stderr:2024-05-27T09:27:07.642+0000 7fca89933640 -1 osd.2 0 log_to_monitors true 2024-05-27T09:27:09.434 INFO:tasks.ceph.osd.7.smithi163.stderr:2024-05-27T09:27:09.433+0000 7f8942447640 -1 osd.7 0 waiting for initial osdmap 2024-05-27T09:27:09.449 INFO:tasks.ceph.osd.6.smithi163.stderr:2024-05-27T09:27:09.449+0000 7fd45bacb640 -1 osd.6 0 waiting for initial osdmap 2024-05-27T09:27:09.451 INFO:tasks.ceph.osd.7.smithi163.stderr:2024-05-27T09:27:09.451+0000 7f8939a2e640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T09:27:09.468 INFO:tasks.ceph.osd.6.smithi163.stderr:2024-05-27T09:27:09.468+0000 7fd4530b2640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T09:27:09.476 INFO:tasks.ceph.osd.5.smithi163.stderr:2024-05-27T09:27:09.476+0000 7fe642a8c640 -1 osd.5 0 waiting for initial osdmap 2024-05-27T09:27:09.477 INFO:tasks.ceph.osd.4.smithi163.stderr:2024-05-27T09:27:09.476+0000 7f6b2ad5b640 -1 osd.4 0 waiting for initial osdmap 2024-05-27T09:27:09.488 INFO:tasks.ceph.osd.4.smithi163.stderr:2024-05-27T09:27:09.488+0000 7f6b22342640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T09:27:09.492 INFO:tasks.ceph.osd.5.smithi163.stderr:2024-05-27T09:27:09.492+0000 7fe63a887640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T09:27:10.884 INFO:tasks.ceph.osd.2.smithi114.stderr:2024-05-27T09:27:10.884+0000 7fca85200640 -1 osd.2 0 waiting for initial osdmap 2024-05-27T09:27:10.894 INFO:tasks.ceph.osd.2.smithi114.stderr:2024-05-27T09:27:10.893+0000 7fca7a800640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T09:27:10.909 INFO:tasks.ceph.osd.1.smithi114.stderr:2024-05-27T09:27:10.909+0000 7fbf6ea00640 -1 osd.1 0 waiting for initial osdmap 2024-05-27T09:27:10.915 INFO:tasks.ceph.osd.1.smithi114.stderr:2024-05-27T09:27:10.915+0000 7fbf64a00640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T09:27:10.960 INFO:tasks.ceph.osd.3.smithi114.stderr:2024-05-27T09:27:10.960+0000 7fd68d000640 -1 osd.3 0 waiting for initial osdmap 2024-05-27T09:27:10.968 INFO:tasks.ceph.osd.3.smithi114.stderr:2024-05-27T09:27:10.968+0000 7fd683000640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T09:27:11.000 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-05-27T09:27:11.000+0000 7f9c1fa00640 -1 osd.0 0 waiting for initial osdmap 2024-05-27T09:27:11.010 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-05-27T09:27:11.010+0000 7f9c15000640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T09:27:11.392 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-27T09:27:11.640 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.639+0000 7f0b1a200640 1 Processor -- start 2024-05-27T09:27:11.640 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.640+0000 7f0b1a200640 1 -- start start 2024-05-27T09:27:11.642 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.642+0000 7f0b1a200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0b140cbba0 0x7f0b140ca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:11.642 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.642+0000 7f0b1a200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0b140c6460 0x7f0b14156b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:11.642 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.642+0000 7f0b1a200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b140cf690 0x7f0b140cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:11.642 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.642+0000 7f0b1a200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f0b140cc1b0 con 0x7f0b140cbba0 2024-05-27T09:27:11.643 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.642+0000 7f0b1a200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f0b140cc420 con 0x7f0b140c6460 2024-05-27T09:27:11.643 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.642+0000 7f0b1a200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f0b140cc700 con 0x7f0b140cf690 2024-05-27T09:27:11.643 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.642+0000 7f0b18e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0b140cbba0 0x7f0b140ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:11.643 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.642+0000 7f0b18e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0b140cbba0 0x7f0b140ca680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43468/0 (socket says 172.21.15.114:43468) 2024-05-27T09:27:11.643 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.642+0000 7f0b18e00640 1 -- 172.21.15.114:0/1275869188 learned_addr learned my addr 172.21.15.114:0/1275869188 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:11.643 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.642+0000 7f0b19800640 1 --2- 172.21.15.114:0/1275869188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b140cf690 0x7f0b140cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:11.643 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.642+0000 7f0b13e00640 1 --2- 172.21.15.114:0/1275869188 >> v2:172.21.15.163:3300/0 conn(0x7f0b140c6460 0x7f0b14156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:11.644 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.643+0000 7f0b18e00640 1 -- 172.21.15.114:0/1275869188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b140cf690 msgr2=0x7f0b140cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:11.644 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.643+0000 7f0b18e00640 1 --2- 172.21.15.114:0/1275869188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b140cf690 0x7f0b140cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:11.644 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.643+0000 7f0b18e00640 1 -- 172.21.15.114:0/1275869188 >> v2:172.21.15.163:3300/0 conn(0x7f0b140c6460 msgr2=0x7f0b14156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:11.644 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.643+0000 7f0b18e00640 1 --2- 172.21.15.114:0/1275869188 >> v2:172.21.15.163:3300/0 conn(0x7f0b140c6460 0x7f0b14156b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:11.644 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.643+0000 7f0b18e00640 1 -- 172.21.15.114:0/1275869188 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b140cc9e0 con 0x7f0b140cbba0 2024-05-27T09:27:11.644 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.644+0000 7f0b18e00640 1 --2- 172.21.15.114:0/1275869188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b140cbba0 0x7f0b140ca680 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0b140c6410 tx=0x7f0b0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b3d1633421e135 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:11.645 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.644+0000 7f0b13400640 1 -- 172.21.15.114:0/1275869188 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04037070 con 0x7f0b140cbba0 2024-05-27T09:27:11.645 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.644+0000 7f0b13400640 1 -- 172.21.15.114:0/1275869188 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b04032440 con 0x7f0b140cbba0 2024-05-27T09:27:11.645 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.644+0000 7f0b13400640 1 -- 172.21.15.114:0/1275869188 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04032810 con 0x7f0b140cbba0 2024-05-27T09:27:11.645 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.645+0000 7f0b1a200640 1 -- 172.21.15.114:0/1275869188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b140cbba0 msgr2=0x7f0b140ca680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:11.645 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.645+0000 7f0b1a200640 1 --2- 172.21.15.114:0/1275869188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b140cbba0 0x7f0b140ca680 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0b140c6410 tx=0x7f0b0402a270 comp rx=0 tx=0).stop 2024-05-27T09:27:11.646 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.645+0000 7f0b1a200640 1 -- 172.21.15.114:0/1275869188 shutdown_connections 2024-05-27T09:27:11.646 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.645+0000 7f0b1a200640 1 --2- 172.21.15.114:0/1275869188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b140cf690 0x7f0b140cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:11.646 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.645+0000 7f0b1a200640 1 --2- 172.21.15.114:0/1275869188 >> v2:172.21.15.163:3300/0 conn(0x7f0b140c6460 0x7f0b14156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:11.646 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.645+0000 7f0b1a200640 1 --2- 172.21.15.114:0/1275869188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b140cbba0 0x7f0b140ca680 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:11.646 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.645+0000 7f0b1a200640 1 -- 172.21.15.114:0/1275869188 >> 172.21.15.114:0/1275869188 conn(0x7f0b140716e0 msgr2=0x7f0b14068720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:11.646 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.645+0000 7f0b1a200640 1 -- 172.21.15.114:0/1275869188 shutdown_connections 2024-05-27T09:27:11.646 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.645+0000 7f0b1a200640 1 -- 172.21.15.114:0/1275869188 wait complete. 2024-05-27T09:27:11.647 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.647+0000 7f0b1a200640 1 Processor -- start 2024-05-27T09:27:11.647 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.648+0000 7f0b1a200640 1 -- start start 2024-05-27T09:27:11.650 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.649+0000 7f0b1a200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b140c6460 0x7f0b1415ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:11.650 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.649+0000 7f0b1a200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b140cbba0 0x7f0b140c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:11.650 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.649+0000 7f0b1a200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b140cf690 0x7f0b140cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:11.650 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.649+0000 7f0b1a200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1415fec0 con 0x7f0b140c6460 2024-05-27T09:27:11.650 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.649+0000 7f0b1a200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0b140cacc0 con 0x7f0b140cf690 2024-05-27T09:27:11.650 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.649+0000 7f0b1a200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f0b14160040 con 0x7f0b140cbba0 2024-05-27T09:27:11.650 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.649+0000 7f0b18e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b140c6460 0x7f0b1415ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:11.650 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.649+0000 7f0b18e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b140c6460 0x7f0b1415ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43480/0 (socket says 172.21.15.114:43480) 2024-05-27T09:27:11.650 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.649+0000 7f0b18e00640 1 -- 172.21.15.114:0/2409154745 learned_addr learned my addr 172.21.15.114:0/2409154745 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:11.650 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.649+0000 7f0b13e00640 1 --2- 172.21.15.114:0/2409154745 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b140cbba0 0x7f0b140c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:11.650 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.649+0000 7f0b19800640 1 --2- 172.21.15.114:0/2409154745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b140cf690 0x7f0b140cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:11.651 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.650+0000 7f0b18e00640 1 -- 172.21.15.114:0/2409154745 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b140cbba0 msgr2=0x7f0b140c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:11.651 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.650+0000 7f0b18e00640 1 --2- 172.21.15.114:0/2409154745 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b140cbba0 0x7f0b140c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:11.651 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.650+0000 7f0b18e00640 1 -- 172.21.15.114:0/2409154745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b140cf690 msgr2=0x7f0b140cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:11.651 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.650+0000 7f0b18e00640 1 --2- 172.21.15.114:0/2409154745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b140cf690 0x7f0b140cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:11.651 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.650+0000 7f0b18e00640 1 -- 172.21.15.114:0/2409154745 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b140c7280 con 0x7f0b140c6460 2024-05-27T09:27:11.651 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.650+0000 7f0b18e00640 1 --2- 172.21.15.114:0/2409154745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b140c6460 0x7f0b1415ef90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0b0402a2b0 tx=0x7f0b040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:11.652 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.650+0000 7f0b10c00640 1 -- 172.21.15.114:0/2409154745 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04041020 con 0x7f0b140c6460 2024-05-27T09:27:11.652 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.650+0000 7f0b10c00640 1 -- 172.21.15.114:0/2409154745 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b0403d620 con 0x7f0b140c6460 2024-05-27T09:27:11.652 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.650+0000 7f0b1a200640 1 -- 172.21.15.114:0/2409154745 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b140ceff0 con 0x7f0b140c6460 2024-05-27T09:27:11.652 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.650+0000 7f0b10c00640 1 -- 172.21.15.114:0/2409154745 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04037040 con 0x7f0b140c6460 2024-05-27T09:27:11.652 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.650+0000 7f0b1a200640 1 -- 172.21.15.114:0/2409154745 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b1415a340 con 0x7f0b140c6460 2024-05-27T09:27:11.652 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.651+0000 7f0b1a200640 1 -- 172.21.15.114:0/2409154745 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b14050860 con 0x7f0b140c6460 2024-05-27T09:27:11.652 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.653+0000 7f0b10c00640 1 -- 172.21.15.114:0/2409154745 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 4) ==== 98145+0+0 (secure 0 0 0) 0x7f0b04044020 con 0x7f0b140c6460 2024-05-27T09:27:11.654 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.653+0000 7f0b10c00640 1 --2- 172.21.15.114:0/2409154745 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0af0076240 0x7f0af0078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:11.654 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.654+0000 7f0b10c00640 1 -- 172.21.15.114:0/2409154745 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4846+0+0 (secure 0 0 0) 0x7f0b040b7bb0 con 0x7f0b140c6460 2024-05-27T09:27:11.654 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.654+0000 7f0b13e00640 1 --2- 172.21.15.114:0/2409154745 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0af0076240 0x7f0af0078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:11.654 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.654+0000 7f0b13e00640 1 --2- 172.21.15.114:0/2409154745 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0af0076240 0x7f0af0078750 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0b1415d9a0 tx=0x7f0afc00d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:11.657 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.658+0000 7f0b10c00640 1 -- 172.21.15.114:0/2409154745 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0b0404b030 con 0x7f0b140c6460 2024-05-27T09:27:11.850 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.849+0000 7f0b1a200640 1 -- 172.21.15.114:0/2409154745 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b1406dd60 con 0x7f0b140c6460 2024-05-27T09:27:11.852 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.852+0000 7f0b10c00640 1 -- 172.21.15.114:0/2409154745 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+12832 (secure 0 0 0) 0x7f0b040812e0 con 0x7f0b140c6460 2024-05-27T09:27:11.852 INFO:teuthology.misc.health.smithi114.stdout: 2024-05-27T09:27:11.852 INFO:teuthology.misc.health.smithi114.stdout:{"epoch":15,"fsid":"2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48","created":"2024-05-27T09:27:00.307278+0000","modified":"2024-05-27T09:27:11.463226+0000","last_up_change":"2024-05-27T09:27:11.463226+0000","last_in_change":"2024-05-27T09:27:02.972880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T09:27:10.796859+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3561c750-1d63-42f6-a6ae-0ee63721e15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6817","nonce":1960550710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6819","nonce":1960550710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6824","nonce":1960550710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6821","nonce":1960550710}]},"public_addr":"172.21.15.114:6817/1960550710","cluster_addr":"172.21.15.114:6819/1960550710","heartbeat_back_addr":"172.21.15.114:6824/1960550710","heartbeat_front_addr":"172.21.15.114:6821/1960550710","state":["exists","up"]},{"osd":1,"uuid":"9a407d04-38c0-4676-928f-d04e8116bd19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6825","nonce":326580057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6827","nonce":326580057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6831","nonce":326580057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6829","nonce":326580057}]},"public_addr":"172.21.15.114:6825/326580057","cluster_addr":"172.21.15.114:6827/326580057","heartbeat_back_addr":"172.21.15.114:6831/326580057","heartbeat_front_addr":"172.21.15.114:6829/326580057","state":["exists","up"]},{"osd":2,"uuid":"255a1cc6-9357-43f7-9db5-d760f4806ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6809","nonce":865568619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6811","nonce":865568619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6815","nonce":865568619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6813","nonce":865568619}]},"public_addr":"172.21.15.114:6809/865568619","cluster_addr":"172.21.15.114:6811/865568619","heartbeat_back_addr":"172.21.15.114:6815/865568619","heartbeat_front_addr":"172.21.15.114:6813/865568619","state":["exists","up"]},{"osd":3,"uuid":"cb4d7692-9a17-41b9-927e-d8eac3e217c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6801","nonce":1601792164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6803","nonce":1601792164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6807","nonce":1601792164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6805","nonce":1601792164}]},"public_addr":"172.21.15.114:6801/1601792164","cluster_addr":"172.21.15.114:6803/1601792164","heartbeat_back_addr":"172.21.15.114:6807/1601792164","heartbeat_front_addr":"172.21.15.114:6805/1601792164","state":["exists","up"]},{"osd":4,"uuid":"cdef1940-3453-4fbd-b0f1-459f5f5c7142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6817","nonce":1982246015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6819","nonce":1982246015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6823","nonce":1982246015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6821","nonce":1982246015}]},"public_addr":"172.21.15.163:6817/1982246015","cluster_addr":"172.21.15.163:6819/1982246015","heartbeat_back_addr":"172.21.15.163:6823/1982246015","heartbeat_front_addr":"172.21.15.163:6821/1982246015","state":["exists","up"]},{"osd":5,"uuid":"220a7b32-abcc-492f-bca8-4615cc2ea111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6801","nonce":1695294004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6803","nonce":1695294004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6807","nonce":1695294004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6805","nonce":1695294004}]},"public_addr":"172.21.15.163:6801/1695294004","cluster_addr":"172.21.15.163:6803/1695294004","heartbeat_back_addr":"172.21.15.163:6807/1695294004","heartbeat_front_addr":"172.21.15.163:6805/1695294004","state":["exists","up"]},{"osd":6,"uuid":"43a74d13-d03f-493a-a6c0-e73b0a056c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6809","nonce":3649718505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6811","nonce":3649718505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6815","nonce":3649718505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6813","nonce":3649718505}]},"public_addr":"172.21.15.163:6809/3649718505","cluster_addr":"172.21.15.163:6811/3649718505","heartbeat_back_addr":"172.21.15.163:6815/3649718505","heartbeat_front_addr":"172.21.15.163:6813/3649718505","state":["exists","up"]},{"osd":7,"uuid":"134629c2-70af-4b84-83a6-7ddf53681baa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6825","nonce":2539451311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6827","nonce":2539451311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6831","nonce":2539451311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6829","nonce":2539451311}]},"public_addr":"172.21.15.163:6825/2539451311","cluster_addr":"172.21.15.163:6827/2539451311","heartbeat_back_addr":"172.21.15.163:6831/2539451311","heartbeat_front_addr":"172.21.15.163:6829/2539451311","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.220072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.079531+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.180419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.175008+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T09:27:11.854 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.854+0000 7f0b1a200640 1 -- 172.21.15.114:0/2409154745 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0af0076240 msgr2=0x7f0af0078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:11.854 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.854+0000 7f0b1a200640 1 --2- 172.21.15.114:0/2409154745 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0af0076240 0x7f0af0078750 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0b1415d9a0 tx=0x7f0afc00d040 comp rx=0 tx=0).stop 2024-05-27T09:27:11.854 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.854+0000 7f0b1a200640 1 -- 172.21.15.114:0/2409154745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b140c6460 msgr2=0x7f0b1415ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:11.854 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.854+0000 7f0b1a200640 1 --2- 172.21.15.114:0/2409154745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b140c6460 0x7f0b1415ef90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0b0402a2b0 tx=0x7f0b040046c0 comp rx=0 tx=0).stop 2024-05-27T09:27:11.855 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.854+0000 7f0b1a200640 1 -- 172.21.15.114:0/2409154745 shutdown_connections 2024-05-27T09:27:11.855 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.854+0000 7f0b1a200640 1 --2- 172.21.15.114:0/2409154745 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0af0076240 0x7f0af0078750 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:11.855 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.854+0000 7f0b1a200640 1 --2- 172.21.15.114:0/2409154745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b140cf690 0x7f0b140cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:11.855 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.854+0000 7f0b1a200640 1 --2- 172.21.15.114:0/2409154745 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b140cbba0 0x7f0b140c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:11.855 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.854+0000 7f0b1a200640 1 --2- 172.21.15.114:0/2409154745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b140c6460 0x7f0b1415ef90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:11.855 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.854+0000 7f0b1a200640 1 -- 172.21.15.114:0/2409154745 >> 172.21.15.114:0/2409154745 conn(0x7f0b140716e0 msgr2=0x7f0b140cfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:11.855 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.855+0000 7f0b1a200640 1 -- 172.21.15.114:0/2409154745 shutdown_connections 2024-05-27T09:27:11.855 INFO:teuthology.misc.health.smithi114.stderr:2024-05-27T09:27:11.855+0000 7f0b1a200640 1 -- 172.21.15.114:0/2409154745 wait complete. 2024-05-27T09:27:11.876 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-27T09:27:11.876 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-27T09:27:11.876 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-27T09:27:11.876 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-27T09:27:11.883 INFO:tasks.ceph.mds.a:Started 2024-05-27T09:27:11.883 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-27T09:27:11.883 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-27T09:27:11.889 INFO:tasks.ceph.mds.c:Started 2024-05-27T09:27:11.889 INFO:tasks.ceph.mds.e:Restarting daemon 2024-05-27T09:27:11.890 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-05-27T09:27:11.896 INFO:tasks.ceph.mds.e:Started 2024-05-27T09:27:11.896 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-27T09:27:11.896 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-27T09:27:11.899 INFO:tasks.ceph.mds.b:Started 2024-05-27T09:27:11.899 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-27T09:27:11.899 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-27T09:27:11.901 INFO:tasks.ceph.mds.d:Started 2024-05-27T09:27:11.901 INFO:tasks.ceph.mds.f:Restarting daemon 2024-05-27T09:27:11.902 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-05-27T09:27:11.904 INFO:tasks.ceph.mds.f:Started 2024-05-27T09:27:11.904 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-27T09:27:11.904 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:27:11.997 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-05-27T09:27:11.997 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-05-27T09:27:11.998 INFO:tasks.ceph.mds.f.smithi163.stdout:starting mds.f at 2024-05-27T09:27:12.101 INFO:tasks.ceph.mds.a.smithi114.stdout:starting mds.a at 2024-05-27T09:27:12.101 INFO:tasks.ceph.mds.c.smithi114.stdout:starting mds.c at 2024-05-27T09:27:12.109 INFO:tasks.ceph.mds.e.smithi114.stdout:starting mds.e at 2024-05-27T09:27:12.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.145+0000 7f24fbc00640 1 Processor -- start 2024-05-27T09:27:12.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.145+0000 7f24fbc00640 1 -- start start 2024-05-27T09:27:12.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.146+0000 7f24fbc00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f24f40c9170 0x7f24f4152f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:12.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.146+0000 7f24fbc00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f24f40714b0 0x7f24f40ca400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:12.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.146+0000 7f24fbc00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24f40c42f0 con 0x7f24f40c8370 2024-05-27T09:27:12.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.147+0000 7f24fbc00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f24f40c9b40 con 0x7f24f40714b0 2024-05-27T09:27:12.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.147+0000 7f24fbc00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f24f40c2b20 con 0x7f24f40c9170 2024-05-27T09:27:12.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.147+0000 7f24f3e00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f24f40c8370 0x7f24f415c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:42080/0 (socket says 172.21.15.114:42080) 2024-05-27T09:27:12.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.147+0000 7f24f3e00640 1 -- 172.21.15.114:0/2697714658 learned_addr learned my addr 172.21.15.114:0/2697714658 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:12.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.147+0000 7f24f9c00640 1 --2- 172.21.15.114:0/2697714658 >> v2:172.21.15.114:3300/0 conn(0x7f24f40714b0 0x7f24f40ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:12.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.147+0000 7f24f9200640 1 --2- 172.21.15.114:0/2697714658 >> v2:172.21.15.163:3300/0 conn(0x7f24f40c9170 0x7f24f4152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.147+0000 7f24f3400640 1 -- 172.21.15.114:0/2697714658 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4029862229 0 0) 0x7f24f40c42f0 con 0x7f24f40c8370 2024-05-27T09:27:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.147+0000 7f24f3400640 1 -- 172.21.15.114:0/2697714658 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24dc003520 con 0x7f24f40c8370 2024-05-27T09:27:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.147+0000 7f24f9c00640 1 -- 172.21.15.114:0/2697714658 >> v2:172.21.15.163:3300/0 conn(0x7f24f40c9170 msgr2=0x7f24f4152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.147+0000 7f24f9c00640 1 --2- 172.21.15.114:0/2697714658 >> v2:172.21.15.163:3300/0 conn(0x7f24f40c9170 0x7f24f4152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.147+0000 7f24f9c00640 1 -- 172.21.15.114:0/2697714658 >> v1:172.21.15.114:6789/0 conn(0x7f24f40c8370 legacy=0x7f24f415c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.147+0000 7f24f9c00640 1 -- 172.21.15.114:0/2697714658 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f40c45d0 con 0x7f24f40714b0 2024-05-27T09:27:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.148+0000 7f24f9c00640 1 --2- 172.21.15.114:0/2697714658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24f40714b0 0x7f24f40ca400 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f24f40ca3d0 tx=0x7f24ec02c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73465259ce799a81 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.149+0000 7f24f3400640 1 -- 172.21.15.114:0/2697714658 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec009540 con 0x7f24f40714b0 2024-05-27T09:27:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.149+0000 7f24f3400640 1 -- 172.21.15.114:0/2697714658 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24ec037070 con 0x7f24f40714b0 2024-05-27T09:27:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.149+0000 7f24f3400640 1 -- 172.21.15.114:0/2697714658 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec009900 con 0x7f24f40714b0 2024-05-27T09:27:12.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.149+0000 7f24fbc00640 1 -- 172.21.15.114:0/2697714658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24f40714b0 msgr2=0x7f24f40ca400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:12.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.149+0000 7f24fbc00640 1 --2- 172.21.15.114:0/2697714658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24f40714b0 0x7f24f40ca400 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f24f40ca3d0 tx=0x7f24ec02c910 comp rx=0 tx=0).stop 2024-05-27T09:27:12.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.149+0000 7f24fbc00640 1 -- 172.21.15.114:0/2697714658 shutdown_connections 2024-05-27T09:27:12.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.149+0000 7f24fbc00640 1 --2- 172.21.15.114:0/2697714658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24f40714b0 0x7f24f40ca400 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:12.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.149+0000 7f24fbc00640 1 --2- 172.21.15.114:0/2697714658 >> v2:172.21.15.163:3300/0 conn(0x7f24f40c9170 0x7f24f4152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:12.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.149+0000 7f24fbc00640 1 -- 172.21.15.114:0/2697714658 >> 172.21.15.114:0/2697714658 conn(0x7f24f406f6e0 msgr2=0x7f24f4070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:12.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.150+0000 7f24fbc00640 1 -- 172.21.15.114:0/2697714658 shutdown_connections 2024-05-27T09:27:12.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.150+0000 7f24fbc00640 1 -- 172.21.15.114:0/2697714658 wait complete. 2024-05-27T09:27:12.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.152+0000 7f24fbc00640 1 Processor -- start 2024-05-27T09:27:12.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.152+0000 7f24fbc00640 1 -- start start 2024-05-27T09:27:12.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.153+0000 7f24fbc00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24f40714b0 0x7f24f41f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:12.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.153+0000 7f24fbc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24f40c8370 0x7f24f414de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:12.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.153+0000 7f24fbc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24f40c9170 0x7f24f4151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:12.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.153+0000 7f24fbc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f24f40c9fd0 con 0x7f24f40c8370 2024-05-27T09:27:12.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.153+0000 7f24fbc00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f24f40c97d0 con 0x7f24f40c9170 2024-05-27T09:27:12.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.153+0000 7f24fbc00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f24f40ca150 con 0x7f24f40714b0 2024-05-27T09:27:12.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.153+0000 7f24f3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24f40c8370 0x7f24f414de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:12.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.154+0000 7f24f9200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24f40714b0 0x7f24f41f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:12.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.154+0000 7f24f3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24f40c8370 0x7f24f414de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43522/0 (socket says 172.21.15.114:43522) 2024-05-27T09:27:12.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.154+0000 7f24f3e00640 1 -- 172.21.15.114:0/3602331525 learned_addr learned my addr 172.21.15.114:0/3602331525 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:12.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.154+0000 7f24f9c00640 1 --2- 172.21.15.114:0/3602331525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24f40c9170 0x7f24f4151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:12.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.154+0000 7f24f9c00640 1 -- 172.21.15.114:0/3602331525 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24f40714b0 msgr2=0x7f24f41f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:12.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.154+0000 7f24f9c00640 1 --2- 172.21.15.114:0/3602331525 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24f40714b0 0x7f24f41f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:12.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.154+0000 7f24f9c00640 1 -- 172.21.15.114:0/3602331525 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24f40c8370 msgr2=0x7f24f414de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:12.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.154+0000 7f24f9c00640 1 --2- 172.21.15.114:0/3602331525 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24f40c8370 0x7f24f414de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:12.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.154+0000 7f24f9c00640 1 -- 172.21.15.114:0/3602331525 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f414a070 con 0x7f24f40c9170 2024-05-27T09:27:12.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.154+0000 7f24f9200640 1 --2- 172.21.15.114:0/3602331525 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24f40714b0 0x7f24f41f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:12.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.154+0000 7f24f9c00640 1 --2- 172.21.15.114:0/3602331525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24f40c9170 0x7f24f4151bf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f24f40cb130 tx=0x7f24ec004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:12.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.154+0000 7f24f0c00640 1 -- 172.21.15.114:0/3602331525 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec009d00 con 0x7f24f40c9170 2024-05-27T09:27:12.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.154+0000 7f24f0c00640 1 -- 172.21.15.114:0/3602331525 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24ec032400 con 0x7f24f40c9170 2024-05-27T09:27:12.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.155+0000 7f24f0c00640 1 -- 172.21.15.114:0/3602331525 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec032830 con 0x7f24f40c9170 2024-05-27T09:27:12.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.155+0000 7f24fbc00640 1 -- 172.21.15.114:0/3602331525 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f414e380 con 0x7f24f40c9170 2024-05-27T09:27:12.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.155+0000 7f24fbc00640 1 -- 172.21.15.114:0/3602331525 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f414d360 con 0x7f24f40c9170 2024-05-27T09:27:12.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.156+0000 7f24fbc00640 1 -- 172.21.15.114:0/3602331525 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24f40c4350 con 0x7f24f40c9170 2024-05-27T09:27:12.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.157+0000 7f24f0c00640 1 -- 172.21.15.114:0/3602331525 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 4) ==== 98145+0+0 (secure 0 0 0) 0x7f24ec031070 con 0x7f24f40c9170 2024-05-27T09:27:12.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.158+0000 7f24f0c00640 1 --2- 172.21.15.114:0/3602331525 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f24d4076240 0x7f24d4078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:12.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.159+0000 7f24f0c00640 1 -- 172.21.15.114:0/3602331525 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4846+0+0 (secure 0 0 0) 0x7f24ec0c0620 con 0x7f24f40c9170 2024-05-27T09:27:12.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.159+0000 7f24f9200640 1 --2- 172.21.15.114:0/3602331525 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f24d4076240 0x7f24d4078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:12.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.159+0000 7f24f9200640 1 --2- 172.21.15.114:0/3602331525 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f24d4076240 0x7f24d4078750 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009e90 tx=0x7f24e8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:12.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.162+0000 7f24f0c00640 1 -- 172.21.15.114:0/3602331525 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f24f40c4350 con 0x7f24f40c9170 2024-05-27T09:27:12.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.395+0000 7f24fbc00640 1 -- 172.21.15.114:0/3602331525 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24f406a380 con 0x7f24f40c9170 2024-05-27T09:27:12.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.395+0000 7f24f0c00640 1 -- 172.21.15.114:0/3602331525 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7f24f414a070 con 0x7f24f40c9170 2024-05-27T09:27:12.396 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:12.396 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-27T09:27:12.396 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 2 2024-05-27T09:27:12.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.398+0000 7f24fbc00640 1 -- 172.21.15.114:0/3602331525 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f24d4076240 msgr2=0x7f24d4078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:12.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.398+0000 7f24fbc00640 1 --2- 172.21.15.114:0/3602331525 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f24d4076240 0x7f24d4078750 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009e90 tx=0x7f24e8009190 comp rx=0 tx=0).stop 2024-05-27T09:27:12.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.398+0000 7f24fbc00640 1 -- 172.21.15.114:0/3602331525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24f40c9170 msgr2=0x7f24f4151bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:12.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.398+0000 7f24fbc00640 1 --2- 172.21.15.114:0/3602331525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24f40c9170 0x7f24f4151bf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f24f40cb130 tx=0x7f24ec004a40 comp rx=0 tx=0).stop 2024-05-27T09:27:12.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.398+0000 7f24fbc00640 1 -- 172.21.15.114:0/3602331525 shutdown_connections 2024-05-27T09:27:12.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.398+0000 7f24fbc00640 1 --2- 172.21.15.114:0/3602331525 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f24d4076240 0x7f24d4078750 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:12.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.398+0000 7f24fbc00640 1 --2- 172.21.15.114:0/3602331525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24f40c9170 0x7f24f4151bf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:12.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.398+0000 7f24fbc00640 1 --2- 172.21.15.114:0/3602331525 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24f40c8370 0x7f24f414de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:12.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.398+0000 7f24fbc00640 1 --2- 172.21.15.114:0/3602331525 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24f40714b0 0x7f24f41f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:12.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.398+0000 7f24fbc00640 1 -- 172.21.15.114:0/3602331525 >> 172.21.15.114:0/3602331525 conn(0x7f24f406f6e0 msgr2=0x7f24f41491d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:12.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.398+0000 7f24fbc00640 1 -- 172.21.15.114:0/3602331525 shutdown_connections 2024-05-27T09:27:12.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:12.398+0000 7f24fbc00640 1 -- 172.21.15.114:0/3602331525 wait complete. 2024-05-27T09:27:14.421 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:27:14.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.705+0000 7fc98a600640 1 Processor -- start 2024-05-27T09:27:14.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.706+0000 7fc98a600640 1 -- start start 2024-05-27T09:27:14.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.708+0000 7fc98a600640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc984070060 0x7fc984167df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:14.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.708+0000 7fc98a600640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9840c8470 con 0x7fc984156880 2024-05-27T09:27:14.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.708+0000 7fc98a600640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9840c8dd0 con 0x7fc984074a80 2024-05-27T09:27:14.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.708+0000 7fc98a600640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc98415d480 con 0x7fc984070060 2024-05-27T09:27:14.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.708+0000 7fc989c00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc984070060 0x7fc984167df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:14.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.708+0000 7fc989c00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc984070060 0x7fc984167df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43530/0 (socket says 172.21.15.114:43530) 2024-05-27T09:27:14.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.708+0000 7fc989c00640 1 -- 172.21.15.114:0/956585262 learned_addr learned my addr 172.21.15.114:0/956585262 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:14.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.708+0000 7fc989200640 1 --1- 172.21.15.114:0/956585262 >> v1:172.21.15.163:6789/0 conn(0x7fc984074a80 0x7fc98415cd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:33584/0 (socket says 172.21.15.114:33584) 2024-05-27T09:27:14.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.709+0000 7fc989c00640 1 -- 172.21.15.114:0/956585262 >> v1:172.21.15.163:6789/0 conn(0x7fc984074a80 legacy=0x7fc98415cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:14.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.709+0000 7fc989c00640 1 -- 172.21.15.114:0/956585262 >> v1:172.21.15.114:6789/0 conn(0x7fc984156880 legacy=0x7fc98406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:14.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.709+0000 7fc989c00640 1 -- 172.21.15.114:0/956585262 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98415c570 con 0x7fc984070060 2024-05-27T09:27:14.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.710+0000 7fc989c00640 1 --2- 172.21.15.114:0/956585262 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc984070060 0x7fc984167df0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009e50 tx=0x7fc97c02a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747adc030a1ae197 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:14.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.710+0000 7fc983e00640 1 -- 172.21.15.114:0/956585262 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c036070 con 0x7fc984070060 2024-05-27T09:27:14.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.710+0000 7fc983e00640 1 -- 172.21.15.114:0/956585262 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc97c031600 con 0x7fc984070060 2024-05-27T09:27:14.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.710+0000 7fc983e00640 1 -- 172.21.15.114:0/956585262 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c0319d0 con 0x7fc984070060 2024-05-27T09:27:14.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.711+0000 7fc98a600640 1 -- 172.21.15.114:0/956585262 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc984070060 msgr2=0x7fc984167df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:14.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.711+0000 7fc98a600640 1 --2- 172.21.15.114:0/956585262 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc984070060 0x7fc984167df0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009e50 tx=0x7fc97c02a7b0 comp rx=0 tx=0).stop 2024-05-27T09:27:14.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.711+0000 7fc98a600640 1 -- 172.21.15.114:0/956585262 shutdown_connections 2024-05-27T09:27:14.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.711+0000 7fc98a600640 1 --2- 172.21.15.114:0/956585262 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc984070060 0x7fc984167df0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:14.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.711+0000 7fc98a600640 1 -- 172.21.15.114:0/956585262 >> 172.21.15.114:0/956585262 conn(0x7fc9840ca320 msgr2=0x7fc9840ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:14.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.711+0000 7fc98a600640 1 -- 172.21.15.114:0/956585262 shutdown_connections 2024-05-27T09:27:14.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.711+0000 7fc98a600640 1 -- 172.21.15.114:0/956585262 wait complete. 2024-05-27T09:27:14.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.713+0000 7fc98a600640 1 Processor -- start 2024-05-27T09:27:14.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.713+0000 7fc98a600640 1 -- start start 2024-05-27T09:27:14.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.715+0000 7fc98a600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc984070060 0x7fc98415bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:14.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.715+0000 7fc98a600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc984074a80 0x7fc984158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:14.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.715+0000 7fc98a600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc984156880 0x7fc9840cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:14.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.715+0000 7fc98a600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc9840c8750 con 0x7fc984156880 2024-05-27T09:27:14.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.715+0000 7fc98a600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc9840c8220 con 0x7fc984074a80 2024-05-27T09:27:14.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.715+0000 7fc98a600640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fc98415c8e0 con 0x7fc984070060 2024-05-27T09:27:14.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.715+0000 7fc989c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc984156880 0x7fc9840cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:14.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.715+0000 7fc989c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc984156880 0x7fc9840cbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43542/0 (socket says 172.21.15.114:43542) 2024-05-27T09:27:14.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.715+0000 7fc989c00640 1 -- 172.21.15.114:0/1407816059 learned_addr learned my addr 172.21.15.114:0/1407816059 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:14.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.715+0000 7fc989200640 1 --2- 172.21.15.114:0/1407816059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc984070060 0x7fc98415bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:14.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.715+0000 7fc97be00640 1 --2- 172.21.15.114:0/1407816059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc984074a80 0x7fc984158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:14.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.716+0000 7fc989c00640 1 -- 172.21.15.114:0/1407816059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc984070060 msgr2=0x7fc98415bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:14.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.716+0000 7fc989c00640 1 --2- 172.21.15.114:0/1407816059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc984070060 0x7fc98415bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:14.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.716+0000 7fc989c00640 1 -- 172.21.15.114:0/1407816059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc984074a80 msgr2=0x7fc984158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:14.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.716+0000 7fc989c00640 1 --2- 172.21.15.114:0/1407816059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc984074a80 0x7fc984158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:14.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.716+0000 7fc989c00640 1 -- 172.21.15.114:0/1407816059 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc984158f60 con 0x7fc984156880 2024-05-27T09:27:14.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.716+0000 7fc97be00640 1 --2- 172.21.15.114:0/1407816059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc984074a80 0x7fc984158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:27:14.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.716+0000 7fc989c00640 1 --2- 172.21.15.114:0/1407816059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc984156880 0x7fc9840cbf70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009f70 tx=0x7fc97c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:14.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.716+0000 7fc981600640 1 -- 172.21.15.114:0/1407816059 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c040020 con 0x7fc984156880 2024-05-27T09:27:14.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.716+0000 7fc981600640 1 -- 172.21.15.114:0/1407816059 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc97c0077a0 con 0x7fc984156880 2024-05-27T09:27:14.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.716+0000 7fc98a600640 1 -- 172.21.15.114:0/1407816059 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc98415f540 con 0x7fc984156880 2024-05-27T09:27:14.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.716+0000 7fc981600640 1 -- 172.21.15.114:0/1407816059 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c036040 con 0x7fc984156880 2024-05-27T09:27:14.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.716+0000 7fc98a600640 1 -- 172.21.15.114:0/1407816059 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc984158220 con 0x7fc984156880 2024-05-27T09:27:14.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.718+0000 7fc98a600640 1 -- 172.21.15.114:0/1407816059 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc984050860 con 0x7fc984156880 2024-05-27T09:27:14.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.719+0000 7fc981600640 1 -- 172.21.15.114:0/1407816059 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 4) ==== 98145+0+0 (secure 0 0 0) 0x7fc97c043020 con 0x7fc984156880 2024-05-27T09:27:14.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.720+0000 7fc981600640 1 --2- 172.21.15.114:0/1407816059 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc960076270 0x7fc960078780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:14.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.720+0000 7fc981600640 1 -- 172.21.15.114:0/1407816059 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fc97c0b7b70 con 0x7fc984156880 2024-05-27T09:27:14.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.720+0000 7fc989200640 1 --2- 172.21.15.114:0/1407816059 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc960076270 0x7fc960078780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:14.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.721+0000 7fc989200640 1 --2- 172.21.15.114:0/1407816059 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc960076270 0x7fc960078780 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc974000f60 tx=0x7fc974007e20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:14.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.726+0000 7fc981600640 1 -- 172.21.15.114:0/1407816059 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc97c04a030 con 0x7fc984156880 2024-05-27T09:27:14.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.972+0000 7fc98a600640 1 -- 172.21.15.114:0/1407816059 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc984074860 con 0x7fc984156880 2024-05-27T09:27:14.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.973+0000 7fc981600640 1 -- 172.21.15.114:0/1407816059 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+5524 (secure 0 0 0) 0x7fc97c081860 con 0x7fc984156880 2024-05-27T09:27:14.974 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:14.974 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/444766663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":444766663},{"type":"v1","addr":"172.21.15.163:6839","nonce":444766663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4300,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-27T09:27:14.974 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 3 2024-05-27T09:27:14.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.976+0000 7fc98a600640 1 -- 172.21.15.114:0/1407816059 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc960076270 msgr2=0x7fc960078780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:14.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.976+0000 7fc98a600640 1 --2- 172.21.15.114:0/1407816059 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc960076270 0x7fc960078780 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc974000f60 tx=0x7fc974007e20 comp rx=0 tx=0).stop 2024-05-27T09:27:14.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.976+0000 7fc98a600640 1 -- 172.21.15.114:0/1407816059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc984156880 msgr2=0x7fc9840cbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:14.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.976+0000 7fc98a600640 1 --2- 172.21.15.114:0/1407816059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc984156880 0x7fc9840cbf70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009f70 tx=0x7fc97c004a40 comp rx=0 tx=0).stop 2024-05-27T09:27:14.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.976+0000 7fc98a600640 1 -- 172.21.15.114:0/1407816059 shutdown_connections 2024-05-27T09:27:14.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.976+0000 7fc98a600640 1 --2- 172.21.15.114:0/1407816059 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc960076270 0x7fc960078780 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:14.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.976+0000 7fc98a600640 1 --2- 172.21.15.114:0/1407816059 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc984156880 0x7fc9840cbf70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:14.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.976+0000 7fc98a600640 1 --2- 172.21.15.114:0/1407816059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc984074a80 0x7fc984158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:14.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.976+0000 7fc98a600640 1 --2- 172.21.15.114:0/1407816059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc984070060 0x7fc98415bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:14.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.976+0000 7fc98a600640 1 -- 172.21.15.114:0/1407816059 >> 172.21.15.114:0/1407816059 conn(0x7fc9840ca320 msgr2=0x7fc984152cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:14.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.976+0000 7fc98a600640 1 -- 172.21.15.114:0/1407816059 shutdown_connections 2024-05-27T09:27:14.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:14.976+0000 7fc98a600640 1 -- 172.21.15.114:0/1407816059 wait complete. 2024-05-27T09:27:15.001 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-27T09:27:15.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.270+0000 7f9609c00640 1 Processor -- start 2024-05-27T09:27:15.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.270+0000 7f9609c00640 1 -- start start 2024-05-27T09:27:15.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.272+0000 7f9609c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96040cad40 0x7f9604070f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:15.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.272+0000 7f9609c00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f960406caa0 con 0x7f960406c690 2024-05-27T09:27:15.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.272+0000 7f9609c00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f960415b1c0 con 0x7f9604154600 2024-05-27T09:27:15.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.272+0000 7f9609c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f9604070ae0 con 0x7f96040cad40 2024-05-27T09:27:15.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.272+0000 7f9603400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f960406c690 0x7f9604164d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:41924/0 (socket says 172.21.15.114:41924) 2024-05-27T09:27:15.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.272+0000 7f9603400640 1 -- 172.21.15.114:0/1881191635 learned_addr learned my addr 172.21.15.114:0/1881191635 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:15.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.272+0000 7f9602000640 1 --2- 172.21.15.114:0/1881191635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96040cad40 0x7f9604070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:15.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.273+0000 7f9601600640 1 -- 172.21.15.114:0/1881191635 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3918757430 0 0) 0x7f960415b1c0 con 0x7f9604154600 2024-05-27T09:27:15.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.273+0000 7f9602000640 1 -- 172.21.15.114:0/1881191635 >> v1:172.21.15.163:6789/0 conn(0x7f9604154600 legacy=0x7f96040cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:15.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.273+0000 7f9602000640 1 -- 172.21.15.114:0/1881191635 >> v1:172.21.15.114:6789/0 conn(0x7f960406c690 legacy=0x7f9604164d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:15.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.273+0000 7f9602000640 1 -- 172.21.15.114:0/1881191635 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96040701b0 con 0x7f96040cad40 2024-05-27T09:27:15.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.274+0000 7f9602000640 1 --2- 172.21.15.114:0/1881191635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96040cad40 0x7f9604070f70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f95f00097d0 tx=0x7f95f002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16f0d95560d5ec66 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:15.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.274+0000 7f9601600640 1 -- 172.21.15.114:0/1881191635 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f0037070 con 0x7f96040cad40 2024-05-27T09:27:15.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.274+0000 7f9601600640 1 -- 172.21.15.114:0/1881191635 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f002ae30 con 0x7f96040cad40 2024-05-27T09:27:15.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.274+0000 7f9601600640 1 -- 172.21.15.114:0/1881191635 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f0032640 con 0x7f96040cad40 2024-05-27T09:27:15.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.275+0000 7f9609c00640 1 -- 172.21.15.114:0/1881191635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96040cad40 msgr2=0x7f9604070f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:15.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.275+0000 7f9609c00640 1 --2- 172.21.15.114:0/1881191635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96040cad40 0x7f9604070f70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f95f00097d0 tx=0x7f95f002a1a0 comp rx=0 tx=0).stop 2024-05-27T09:27:15.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.275+0000 7f9609c00640 1 -- 172.21.15.114:0/1881191635 shutdown_connections 2024-05-27T09:27:15.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.275+0000 7f9609c00640 1 --2- 172.21.15.114:0/1881191635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96040cad40 0x7f9604070f70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:15.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.275+0000 7f9609c00640 1 -- 172.21.15.114:0/1881191635 >> 172.21.15.114:0/1881191635 conn(0x7f9604072090 msgr2=0x7f96040724d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:15.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.275+0000 7f9609c00640 1 -- 172.21.15.114:0/1881191635 shutdown_connections 2024-05-27T09:27:15.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.275+0000 7f9609c00640 1 -- 172.21.15.114:0/1881191635 wait complete. 2024-05-27T09:27:15.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.277+0000 7f9609c00640 1 Processor -- start 2024-05-27T09:27:15.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.278+0000 7f9609c00640 1 -- start start 2024-05-27T09:27:15.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.279+0000 7f9609c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f960406c690 0x7f96040c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:15.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.279+0000 7f9609c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96040cad40 0x7f96040c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:15.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.279+0000 7f9609c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9604154600 0x7f960414a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:15.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.279+0000 7f9609c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f960415ae50 con 0x7f960406c690 2024-05-27T09:27:15.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.279+0000 7f9609c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9604070770 con 0x7f9604154600 2024-05-27T09:27:15.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.279+0000 7f9609c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f96040cbe40 con 0x7f96040cad40 2024-05-27T09:27:15.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.279+0000 7f9602a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f960406c690 0x7f96040c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:15.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.279+0000 7f9602000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96040cad40 0x7f96040c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:15.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.279+0000 7f9602a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f960406c690 0x7f96040c4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44256/0 (socket says 172.21.15.114:44256) 2024-05-27T09:27:15.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.280+0000 7f9602a00640 1 -- 172.21.15.114:0/4072277195 learned_addr learned my addr 172.21.15.114:0/4072277195 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:15.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.280+0000 7f9602000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96040cad40 0x7f96040c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:34370/0 (socket says 172.21.15.114:34370) 2024-05-27T09:27:15.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.280+0000 7f9603400640 1 --2- 172.21.15.114:0/4072277195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9604154600 0x7f960414a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:15.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.280+0000 7f9602a00640 1 -- 172.21.15.114:0/4072277195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96040cad40 msgr2=0x7f96040c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:15.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.280+0000 7f9602a00640 1 --2- 172.21.15.114:0/4072277195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96040cad40 0x7f96040c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:15.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.280+0000 7f9602a00640 1 -- 172.21.15.114:0/4072277195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9604154600 msgr2=0x7f960414a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:15.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.280+0000 7f9602a00640 1 --2- 172.21.15.114:0/4072277195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9604154600 0x7f960414a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:15.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.280+0000 7f9602a00640 1 -- 172.21.15.114:0/4072277195 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960414ac30 con 0x7f960406c690 2024-05-27T09:27:15.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.280+0000 7f9602000640 1 --2- 172.21.15.114:0/4072277195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96040cad40 0x7f96040c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:15.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.280+0000 7f9602a00640 1 --2- 172.21.15.114:0/4072277195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f960406c690 0x7f96040c4710 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f95f400c3a0 tx=0x7f95f400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:15.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.281+0000 7f95faa00640 1 -- 172.21.15.114:0/4072277195 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4017070 con 0x7f960406c690 2024-05-27T09:27:15.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.281+0000 7f95faa00640 1 -- 172.21.15.114:0/4072277195 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f40035f0 con 0x7f960406c690 2024-05-27T09:27:15.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.281+0000 7f9609c00640 1 -- 172.21.15.114:0/4072277195 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96041e7220 con 0x7f960406c690 2024-05-27T09:27:15.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.281+0000 7f95faa00640 1 -- 172.21.15.114:0/4072277195 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f40039e0 con 0x7f960406c690 2024-05-27T09:27:15.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.281+0000 7f9609c00640 1 -- 172.21.15.114:0/4072277195 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96040c85f0 con 0x7f960406c690 2024-05-27T09:27:15.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.282+0000 7f9609c00640 1 -- 172.21.15.114:0/4072277195 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9604159750 con 0x7f960406c690 2024-05-27T09:27:15.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.283+0000 7f95faa00640 1 -- 172.21.15.114:0/4072277195 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 4) ==== 98145+0+0 (secure 0 0 0) 0x7f95f401a070 con 0x7f960406c690 2024-05-27T09:27:15.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.284+0000 7f95faa00640 1 --2- 172.21.15.114:0/4072277195 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f95e4076270 0x7f95e4078780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:15.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.284+0000 7f95faa00640 1 -- 172.21.15.114:0/4072277195 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f95f40963d0 con 0x7f960406c690 2024-05-27T09:27:15.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.285+0000 7f9602000640 1 --2- 172.21.15.114:0/4072277195 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f95e4076270 0x7f95e4078780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:15.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.285+0000 7f9602000640 1 --2- 172.21.15.114:0/4072277195 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f95e4076270 0x7f95e4078780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f95f0034000 tx=0x7f95f002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:15.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.290+0000 7f95faa00640 1 -- 172.21.15.114:0/4072277195 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9604159750 con 0x7f960406c690 2024-05-27T09:27:15.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.500+0000 7f9609c00640 1 -- 172.21.15.114:0/4072277195 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f960406a380 con 0x7f960406c690 2024-05-27T09:27:15.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.506+0000 7f95faa00640 1 -- 172.21.15.114:0/4072277195 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f95f40960c0 con 0x7f960406c690 2024-05-27T09:27:15.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.510+0000 7f95faa00640 1 -- 172.21.15.114:0/4072277195 <== mon.0 v2:172.21.15.114:3300/0 8 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f960414ac30 con 0x7f960406c690 2024-05-27T09:27:15.511 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:15.511 INFO:teuthology.orchestra.run.smithi114.stdout:[ 2024-05-27T09:27:15.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-05-27T09:27:15.511 INFO:teuthology.orchestra.run.smithi114.stdout: "poolnum": 1, 2024-05-27T09:27:15.511 INFO:teuthology.orchestra.run.smithi114.stdout: "poolname": ".mgr" 2024-05-27T09:27:15.511 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-05-27T09:27:15.511 INFO:teuthology.orchestra.run.smithi114.stdout:] 2024-05-27T09:27:15.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.513+0000 7f9609c00640 1 -- 172.21.15.114:0/4072277195 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f95e4076270 msgr2=0x7f95e4078780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:15.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.513+0000 7f9609c00640 1 --2- 172.21.15.114:0/4072277195 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f95e4076270 0x7f95e4078780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f95f0034000 tx=0x7f95f002e000 comp rx=0 tx=0).stop 2024-05-27T09:27:15.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.513+0000 7f9609c00640 1 -- 172.21.15.114:0/4072277195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f960406c690 msgr2=0x7f96040c4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:15.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.513+0000 7f9609c00640 1 --2- 172.21.15.114:0/4072277195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f960406c690 0x7f96040c4710 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f95f400c3a0 tx=0x7f95f400c860 comp rx=0 tx=0).stop 2024-05-27T09:27:15.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.513+0000 7f9609c00640 1 -- 172.21.15.114:0/4072277195 shutdown_connections 2024-05-27T09:27:15.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.513+0000 7f9609c00640 1 --2- 172.21.15.114:0/4072277195 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f95e4076270 0x7f95e4078780 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:15.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.513+0000 7f9609c00640 1 --2- 172.21.15.114:0/4072277195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9604154600 0x7f960414a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:15.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.513+0000 7f9609c00640 1 --2- 172.21.15.114:0/4072277195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96040cad40 0x7f96040c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:15.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.513+0000 7f9609c00640 1 --2- 172.21.15.114:0/4072277195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f960406c690 0x7f96040c4710 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:15.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.513+0000 7f9609c00640 1 -- 172.21.15.114:0/4072277195 >> 172.21.15.114:0/4072277195 conn(0x7f9604072090 msgr2=0x7f9604158270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:15.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.514+0000 7f9609c00640 1 -- 172.21.15.114:0/4072277195 shutdown_connections 2024-05-27T09:27:15.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.514+0000 7f9609c00640 1 -- 172.21.15.114:0/4072277195 wait complete. 2024-05-27T09:27:15.539 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-27T09:27:15.539 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-27T09:27:15.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.813+0000 7f19aee00640 1 Processor -- start 2024-05-27T09:27:15.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.814+0000 7f19aee00640 1 -- start start 2024-05-27T09:27:15.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.816+0000 7f19aee00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19a80c9830 0x7f19a806c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:15.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.816+0000 7f19aee00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f19a815af40 0x7f19a806cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:15.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.816+0000 7f19aee00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19a8074cb0 con 0x7f19a8073260 2024-05-27T09:27:15.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.816+0000 7f19aee00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f19a8073ee0 con 0x7f19a815af40 2024-05-27T09:27:15.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.816+0000 7f19aee00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f19a8074130 con 0x7f19a80c9830 2024-05-27T09:27:15.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.816+0000 7f19ace00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f19a815af40 0x7f19a806cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:15.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.816+0000 7f19a7e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19a80c9830 0x7f19a806c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:15.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.816+0000 7f19ace00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f19a815af40 0x7f19a806cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44270/0 (socket says 172.21.15.114:44270) 2024-05-27T09:27:15.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.816+0000 7f19a7e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19a80c9830 0x7f19a806c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:34380/0 (socket says 172.21.15.114:34380) 2024-05-27T09:27:15.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.816+0000 7f19ace00640 1 -- 172.21.15.114:0/2341003609 learned_addr learned my addr 172.21.15.114:0/2341003609 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:15.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.816+0000 7f19a7400640 1 --1- 172.21.15.114:0/2341003609 >> v1:172.21.15.163:6789/0 conn(0x7f19a8073260 0x7f19a815a830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:60478/0 (socket says 172.21.15.114:60478) 2024-05-27T09:27:15.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.817+0000 7f19a7e00640 1 -- 172.21.15.114:0/2341003609 >> v2:172.21.15.114:3300/0 conn(0x7f19a815af40 msgr2=0x7f19a806cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:15.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.817+0000 7f19a7e00640 1 --2- 172.21.15.114:0/2341003609 >> v2:172.21.15.114:3300/0 conn(0x7f19a815af40 0x7f19a806cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:15.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.817+0000 7f19a7e00640 1 -- 172.21.15.114:0/2341003609 >> v1:172.21.15.163:6789/0 conn(0x7f19a8073260 legacy=0x7f19a815a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:15.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.817+0000 7f19a7e00640 1 -- 172.21.15.114:0/2341003609 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a8074410 con 0x7f19a80c9830 2024-05-27T09:27:15.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.817+0000 7f19a6a00640 1 -- 172.21.15.114:0/2341003609 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3680261950 0 0) 0x7f19a8074cb0 con 0x7f19a8073260 2024-05-27T09:27:15.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.818+0000 7f19a7e00640 1 --2- 172.21.15.114:0/2341003609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19a80c9830 0x7f19a806c690 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f19980097a0 tx=0x7f199802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c31658ead414ff9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:15.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.818+0000 7f19a6a00640 1 -- 172.21.15.114:0/2341003609 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998037070 con 0x7f19a80c9830 2024-05-27T09:27:15.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.818+0000 7f19a6a00640 1 -- 172.21.15.114:0/2341003609 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1998032440 con 0x7f19a80c9830 2024-05-27T09:27:15.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.818+0000 7f19a6a00640 1 -- 172.21.15.114:0/2341003609 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998032810 con 0x7f19a80c9830 2024-05-27T09:27:15.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.819+0000 7f19aee00640 1 -- 172.21.15.114:0/2341003609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19a80c9830 msgr2=0x7f19a806c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:15.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.819+0000 7f19aee00640 1 --2- 172.21.15.114:0/2341003609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19a80c9830 0x7f19a806c690 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f19980097a0 tx=0x7f199802a210 comp rx=0 tx=0).stop 2024-05-27T09:27:15.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.819+0000 7f19aee00640 1 -- 172.21.15.114:0/2341003609 shutdown_connections 2024-05-27T09:27:15.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.819+0000 7f19aee00640 1 --2- 172.21.15.114:0/2341003609 >> v2:172.21.15.114:3300/0 conn(0x7f19a815af40 0x7f19a806cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:15.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.819+0000 7f19aee00640 1 --2- 172.21.15.114:0/2341003609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19a80c9830 0x7f19a806c690 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:15.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.819+0000 7f19aee00640 1 -- 172.21.15.114:0/2341003609 >> 172.21.15.114:0/2341003609 conn(0x7f19a80cd9e0 msgr2=0x7f19a80cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:15.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.819+0000 7f19aee00640 1 -- 172.21.15.114:0/2341003609 shutdown_connections 2024-05-27T09:27:15.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.819+0000 7f19aee00640 1 -- 172.21.15.114:0/2341003609 wait complete. 2024-05-27T09:27:15.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.822+0000 7f19aee00640 1 Processor -- start 2024-05-27T09:27:15.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.822+0000 7f19aee00640 1 -- start start 2024-05-27T09:27:15.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.824+0000 7f19aee00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19a8073260 0x7f19a8158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:15.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.824+0000 7f19aee00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19a80c9830 0x7f19a8149d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:15.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.824+0000 7f19aee00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19a815af40 0x7f19a80c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:15.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.824+0000 7f19aee00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f19a80706c0 con 0x7f19a80c9830 2024-05-27T09:27:15.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.824+0000 7f19aee00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f19a80749d0 con 0x7f19a815af40 2024-05-27T09:27:15.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.824+0000 7f19aee00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f19a8073940 con 0x7f19a8073260 2024-05-27T09:27:15.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.824+0000 7f19a7e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19a8073260 0x7f19a8158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:15.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.824+0000 7f19a7e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19a8073260 0x7f19a8158b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:34396/0 (socket says 172.21.15.114:34396) 2024-05-27T09:27:15.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.824+0000 7f19a7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19a80c9830 0x7f19a8149d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:15.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.824+0000 7f19a7e00640 1 -- 172.21.15.114:0/3612341016 learned_addr learned my addr 172.21.15.114:0/3612341016 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:15.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.824+0000 7f19ace00640 1 --2- 172.21.15.114:0/3612341016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19a815af40 0x7f19a80c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:15.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.825+0000 7f19a7e00640 1 -- 172.21.15.114:0/3612341016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19a815af40 msgr2=0x7f19a80c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:15.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.825+0000 7f19a7e00640 1 --2- 172.21.15.114:0/3612341016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19a815af40 0x7f19a80c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:15.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.825+0000 7f19a7e00640 1 -- 172.21.15.114:0/3612341016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19a80c9830 msgr2=0x7f19a8149d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:15.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.825+0000 7f19a7e00640 1 --2- 172.21.15.114:0/3612341016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19a80c9830 0x7f19a8149d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:15.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.825+0000 7f19a7e00640 1 -- 172.21.15.114:0/3612341016 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a8159070 con 0x7f19a8073260 2024-05-27T09:27:15.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.825+0000 7f19a7e00640 1 --2- 172.21.15.114:0/3612341016 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19a8073260 0x7f19a8158b30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1998035040 tx=0x7f19980046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:15.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.825+0000 7f199fe00640 1 -- 172.21.15.114:0/3612341016 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998037040 con 0x7f19a8073260 2024-05-27T09:27:15.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.825+0000 7f199fe00640 1 -- 172.21.15.114:0/3612341016 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1998030500 con 0x7f19a8073260 2024-05-27T09:27:15.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.825+0000 7f19aee00640 1 -- 172.21.15.114:0/3612341016 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a80c3bb0 con 0x7f19a8073260 2024-05-27T09:27:15.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.825+0000 7f199fe00640 1 -- 172.21.15.114:0/3612341016 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19980308d0 con 0x7f19a8073260 2024-05-27T09:27:15.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.825+0000 7f19aee00640 1 -- 172.21.15.114:0/3612341016 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a80c4170 con 0x7f19a8073260 2024-05-27T09:27:15.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.827+0000 7f19aee00640 1 -- 172.21.15.114:0/3612341016 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19a8152b50 con 0x7f19a8073260 2024-05-27T09:27:15.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.828+0000 7f199fe00640 1 -- 172.21.15.114:0/3612341016 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1998044020 con 0x7f19a8073260 2024-05-27T09:27:15.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.829+0000 7f199fe00640 1 --2- 172.21.15.114:0/3612341016 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f19840762d0 0x7f19840787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:15.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.829+0000 7f199fe00640 1 -- 172.21.15.114:0/3612341016 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f19980b7400 con 0x7f19a8073260 2024-05-27T09:27:15.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.829+0000 7f19a7400640 1 --2- 172.21.15.114:0/3612341016 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f19840762d0 0x7f19840787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:15.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.830+0000 7f19a7400640 1 --2- 172.21.15.114:0/3612341016 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f19840762d0 0x7f19840787e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f19a8150500 tx=0x7f1990001630 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:15.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:15.834+0000 7f199fe00640 1 -- 172.21.15.114:0/3612341016 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f19a8152b50 con 0x7f19a8073260 2024-05-27T09:27:16.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.042+0000 7f19aee00640 1 -- 172.21.15.114:0/3612341016 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f19a806a380 con 0x7f19a8073260 2024-05-27T09:27:16.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.517+0000 7f199fe00640 1 -- 172.21.15.114:0/3612341016 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7f199804b020 con 0x7f19a8073260 2024-05-27T09:27:16.521 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_metadata' created 2024-05-27T09:27:16.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.528+0000 7f19aee00640 1 -- 172.21.15.114:0/3612341016 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f19840762d0 msgr2=0x7f19840787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:16.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.528+0000 7f19aee00640 1 --2- 172.21.15.114:0/3612341016 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f19840762d0 0x7f19840787e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f19a8150500 tx=0x7f1990001630 comp rx=0 tx=0).stop 2024-05-27T09:27:16.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.528+0000 7f19aee00640 1 -- 172.21.15.114:0/3612341016 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19a8073260 msgr2=0x7f19a8158b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:16.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.528+0000 7f19aee00640 1 --2- 172.21.15.114:0/3612341016 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19a8073260 0x7f19a8158b30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1998035040 tx=0x7f19980046c0 comp rx=0 tx=0).stop 2024-05-27T09:27:16.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.529+0000 7f19aee00640 1 -- 172.21.15.114:0/3612341016 shutdown_connections 2024-05-27T09:27:16.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.529+0000 7f19aee00640 1 --2- 172.21.15.114:0/3612341016 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f19840762d0 0x7f19840787e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:16.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.529+0000 7f19aee00640 1 --2- 172.21.15.114:0/3612341016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19a815af40 0x7f19a80c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:16.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.529+0000 7f19aee00640 1 --2- 172.21.15.114:0/3612341016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19a80c9830 0x7f19a8149d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:16.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.529+0000 7f19aee00640 1 --2- 172.21.15.114:0/3612341016 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19a8073260 0x7f19a8158b30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:16.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.529+0000 7f19aee00640 1 -- 172.21.15.114:0/3612341016 >> 172.21.15.114:0/3612341016 conn(0x7f19a80cd9e0 msgr2=0x7f19a806efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:16.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.529+0000 7f19aee00640 1 -- 172.21.15.114:0/3612341016 shutdown_connections 2024-05-27T09:27:16.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.529+0000 7f19aee00640 1 -- 172.21.15.114:0/3612341016 wait complete. 2024-05-27T09:27:16.559 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-27T09:27:16.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.819+0000 7fcf60800640 1 Processor -- start 2024-05-27T09:27:16.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.819+0000 7fcf60800640 1 -- start start 2024-05-27T09:27:16.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.822+0000 7fcf60800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf581548b0 0x7fcf580ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:16.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.822+0000 7fcf60800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fcf580ca920 0x7fcf5815a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:16.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.822+0000 7fcf60800640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf580c8dc0 con 0x7fcf580748f0 2024-05-27T09:27:16.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.822+0000 7fcf60800640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fcf580c9be0 con 0x7fcf580ca920 2024-05-27T09:27:16.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.822+0000 7fcf60800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fcf580c9060 con 0x7fcf581548b0 2024-05-27T09:27:16.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.822+0000 7fcf5e800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fcf580ca920 0x7fcf5815a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:16.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.822+0000 7fcf5d400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf581548b0 0x7fcf580ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:16.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.822+0000 7fcf5e800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fcf580ca920 0x7fcf5815a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44286/0 (socket says 172.21.15.114:44286) 2024-05-27T09:27:16.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.822+0000 7fcf5de00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fcf580748f0 0x7fcf58073260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:41940/0 (socket says 172.21.15.114:41940) 2024-05-27T09:27:16.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.822+0000 7fcf5e800640 1 -- 172.21.15.114:0/580186811 learned_addr learned my addr 172.21.15.114:0/580186811 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:16.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.823+0000 7fcf5d400640 1 -- 172.21.15.114:0/580186811 >> v2:172.21.15.114:3300/0 conn(0x7fcf580ca920 msgr2=0x7fcf5815a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:16.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.823+0000 7fcf5d400640 1 --2- 172.21.15.114:0/580186811 >> v2:172.21.15.114:3300/0 conn(0x7fcf580ca920 0x7fcf5815a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:16.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.823+0000 7fcf5d400640 1 -- 172.21.15.114:0/580186811 >> v1:172.21.15.114:6789/0 conn(0x7fcf580748f0 legacy=0x7fcf58073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:16.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.823+0000 7fcf5d400640 1 -- 172.21.15.114:0/580186811 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf580c9340 con 0x7fcf581548b0 2024-05-27T09:27:16.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.824+0000 7fcf5d400640 1 --2- 172.21.15.114:0/580186811 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf581548b0 0x7fcf580ca3e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcf480097a0 tx=0x7fcf4802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83f45b56af61fd66 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:16.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.824+0000 7fcf5ca00640 1 -- 172.21.15.114:0/580186811 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48037070 con 0x7fcf581548b0 2024-05-27T09:27:16.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.824+0000 7fcf5ca00640 1 -- 172.21.15.114:0/580186811 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf48032440 con 0x7fcf581548b0 2024-05-27T09:27:16.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.824+0000 7fcf5ca00640 1 -- 172.21.15.114:0/580186811 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48032810 con 0x7fcf581548b0 2024-05-27T09:27:16.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.825+0000 7fcf60800640 1 -- 172.21.15.114:0/580186811 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf581548b0 msgr2=0x7fcf580ca3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:16.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.825+0000 7fcf60800640 1 --2- 172.21.15.114:0/580186811 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf581548b0 0x7fcf580ca3e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcf480097a0 tx=0x7fcf4802a180 comp rx=0 tx=0).stop 2024-05-27T09:27:16.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.825+0000 7fcf60800640 1 -- 172.21.15.114:0/580186811 shutdown_connections 2024-05-27T09:27:16.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.825+0000 7fcf60800640 1 --2- 172.21.15.114:0/580186811 >> v2:172.21.15.114:3300/0 conn(0x7fcf580ca920 0x7fcf5815a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:16.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.825+0000 7fcf60800640 1 --2- 172.21.15.114:0/580186811 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf581548b0 0x7fcf580ca3e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:16.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.825+0000 7fcf60800640 1 -- 172.21.15.114:0/580186811 >> 172.21.15.114:0/580186811 conn(0x7fcf5806f6e0 msgr2=0x7fcf58070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:16.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.825+0000 7fcf60800640 1 -- 172.21.15.114:0/580186811 shutdown_connections 2024-05-27T09:27:16.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.825+0000 7fcf60800640 1 -- 172.21.15.114:0/580186811 wait complete. 2024-05-27T09:27:16.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.828+0000 7fcf60800640 1 Processor -- start 2024-05-27T09:27:16.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.828+0000 7fcf60800640 1 -- start start 2024-05-27T09:27:16.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.829+0000 7fcf60800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf580748f0 0x7fcf58158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:16.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.829+0000 7fcf60800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf580ca920 0x7fcf580c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:16.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.829+0000 7fcf60800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf581548b0 0x7fcf580cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:16.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.829+0000 7fcf60800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcf580c5780 con 0x7fcf580748f0 2024-05-27T09:27:16.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.829+0000 7fcf60800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcf580c9900 con 0x7fcf580ca920 2024-05-27T09:27:16.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.829+0000 7fcf60800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fcf58073970 con 0x7fcf581548b0 2024-05-27T09:27:16.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.829+0000 7fcf5de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf580748f0 0x7fcf58158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:16.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.830+0000 7fcf5de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf580748f0 0x7fcf58158dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44288/0 (socket says 172.21.15.114:44288) 2024-05-27T09:27:16.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.830+0000 7fcf5de00640 1 -- 172.21.15.114:0/1605787779 learned_addr learned my addr 172.21.15.114:0/1605787779 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:16.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.830+0000 7fcf5e800640 1 --2- 172.21.15.114:0/1605787779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf581548b0 0x7fcf580cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:16.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.830+0000 7fcf5d400640 1 --2- 172.21.15.114:0/1605787779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf580ca920 0x7fcf580c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:16.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.830+0000 7fcf5de00640 1 -- 172.21.15.114:0/1605787779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf581548b0 msgr2=0x7fcf580cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:16.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.830+0000 7fcf5de00640 1 --2- 172.21.15.114:0/1605787779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf581548b0 0x7fcf580cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:16.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.830+0000 7fcf5de00640 1 -- 172.21.15.114:0/1605787779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf580ca920 msgr2=0x7fcf580c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:16.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.830+0000 7fcf5de00640 1 --2- 172.21.15.114:0/1605787779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf580ca920 0x7fcf580c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:16.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.830+0000 7fcf5de00640 1 -- 172.21.15.114:0/1605787779 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf58159300 con 0x7fcf580748f0 2024-05-27T09:27:16.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.830+0000 7fcf5e800640 1 --2- 172.21.15.114:0/1605787779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf581548b0 0x7fcf580cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:16.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.830+0000 7fcf5de00640 1 --2- 172.21.15.114:0/1605787779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf580748f0 0x7fcf58158dc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00c3a0 tx=0x7fcf4c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:16.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.831+0000 7fcf52000640 1 -- 172.21.15.114:0/1605787779 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c0074e0 con 0x7fcf580748f0 2024-05-27T09:27:16.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.831+0000 7fcf52000640 1 -- 172.21.15.114:0/1605787779 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf4c007680 con 0x7fcf580748f0 2024-05-27T09:27:16.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.831+0000 7fcf52000640 1 -- 172.21.15.114:0/1605787779 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c004a20 con 0x7fcf580748f0 2024-05-27T09:27:16.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.831+0000 7fcf60800640 1 -- 172.21.15.114:0/1605787779 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf5815a1c0 con 0x7fcf580748f0 2024-05-27T09:27:16.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.831+0000 7fcf60800640 1 -- 172.21.15.114:0/1605787779 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf580c7710 con 0x7fcf580748f0 2024-05-27T09:27:16.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.832+0000 7fcf60800640 1 -- 172.21.15.114:0/1605787779 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf5806a380 con 0x7fcf580748f0 2024-05-27T09:27:16.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.833+0000 7fcf52000640 1 -- 172.21.15.114:0/1605787779 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcf4c01a020 con 0x7fcf580748f0 2024-05-27T09:27:16.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.834+0000 7fcf52000640 1 --2- 172.21.15.114:0/1605787779 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcf3c076300 0x7fcf3c078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:16.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.835+0000 7fcf52000640 1 -- 172.21.15.114:0/1605787779 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7fcf4c096970 con 0x7fcf580748f0 2024-05-27T09:27:16.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.835+0000 7fcf5d400640 1 --2- 172.21.15.114:0/1605787779 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcf3c076300 0x7fcf3c078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:16.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.835+0000 7fcf5d400640 1 --2- 172.21.15.114:0/1605787779 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcf3c076300 0x7fcf3c078810 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf48034000 tx=0x7fcf4802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:16.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:16.840+0000 7fcf52000640 1 -- 172.21.15.114:0/1605787779 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcf4c05fef0 con 0x7fcf580748f0 2024-05-27T09:27:17.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.046+0000 7fcf60800640 1 -- 172.21.15.114:0/1605787779 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcf58153460 con 0x7fcf580748f0 2024-05-27T09:27:17.264 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:27:17.272 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:27:17.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.525+0000 7fcf52000640 1 -- 172.21.15.114:0/1605787779 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7fcf4c0641f0 con 0x7fcf580748f0 2024-05-27T09:27:17.527 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_data' created 2024-05-27T09:27:17.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.529+0000 7fcf60800640 1 -- 172.21.15.114:0/1605787779 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcf3c076300 msgr2=0x7fcf3c078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:17.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.529+0000 7fcf60800640 1 --2- 172.21.15.114:0/1605787779 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcf3c076300 0x7fcf3c078810 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf48034000 tx=0x7fcf4802e000 comp rx=0 tx=0).stop 2024-05-27T09:27:17.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.529+0000 7fcf60800640 1 -- 172.21.15.114:0/1605787779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf580748f0 msgr2=0x7fcf58158dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:17.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.529+0000 7fcf60800640 1 --2- 172.21.15.114:0/1605787779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf580748f0 0x7fcf58158dc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00c3a0 tx=0x7fcf4c00c860 comp rx=0 tx=0).stop 2024-05-27T09:27:17.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.530+0000 7fcf60800640 1 -- 172.21.15.114:0/1605787779 shutdown_connections 2024-05-27T09:27:17.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.530+0000 7fcf60800640 1 --2- 172.21.15.114:0/1605787779 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcf3c076300 0x7fcf3c078810 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:17.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.530+0000 7fcf60800640 1 --2- 172.21.15.114:0/1605787779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf581548b0 0x7fcf580cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:17.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.530+0000 7fcf60800640 1 --2- 172.21.15.114:0/1605787779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf580ca920 0x7fcf580c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:17.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.530+0000 7fcf60800640 1 --2- 172.21.15.114:0/1605787779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf580748f0 0x7fcf58158dc0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:17.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.530+0000 7fcf60800640 1 -- 172.21.15.114:0/1605787779 >> 172.21.15.114:0/1605787779 conn(0x7fcf5806f6e0 msgr2=0x7fcf58152940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:17.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.533+0000 7fcf60800640 1 -- 172.21.15.114:0/1605787779 shutdown_connections 2024-05-27T09:27:17.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.534+0000 7fcf60800640 1 -- 172.21.15.114:0/1605787779 wait complete. 2024-05-27T09:27:17.580 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-27T09:27:17.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.829+0000 7f9f58a00640 1 Processor -- start 2024-05-27T09:27:17.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.830+0000 7f9f58a00640 1 -- start start 2024-05-27T09:27:17.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.832+0000 7f9f58a00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9f54073b90 0x7f9f540ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:17.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.832+0000 7f9f58a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f540ca810 0x7f9f5415cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:17.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.832+0000 7f9f58a00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f540cac20 con 0x7f9f541569d0 2024-05-27T09:27:17.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.832+0000 7f9f58a00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f9f54074d70 con 0x7f9f54073b90 2024-05-27T09:27:17.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.832+0000 7f9f58a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f9f540c90a0 con 0x7f9f540ca810 2024-05-27T09:27:17.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.832+0000 7f9f51e00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f9f541569d0 0x7f9f54073260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:41944/0 (socket says 172.21.15.114:41944) 2024-05-27T09:27:17.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.832+0000 7f9f51e00640 1 -- 172.21.15.114:0/4055610036 learned_addr learned my addr 172.21.15.114:0/4055610036 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:17.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.832+0000 7f9f51400640 1 --2- 172.21.15.114:0/4055610036 >> v2:172.21.15.163:3300/0 conn(0x7f9f54073b90 0x7f9f540ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:17.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.832+0000 7f9f52800640 1 --2- 172.21.15.114:0/4055610036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f540ca810 0x7f9f5415cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:17.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.832+0000 7f9f50a00640 1 -- 172.21.15.114:0/4055610036 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 942722117 0 0) 0x7f9f540cac20 con 0x7f9f541569d0 2024-05-27T09:27:17.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.833+0000 7f9f50a00640 1 -- 172.21.15.114:0/4055610036 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f34003520 con 0x7f9f541569d0 2024-05-27T09:27:17.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.833+0000 7f9f51400640 1 -- 172.21.15.114:0/4055610036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f540ca810 msgr2=0x7f9f5415cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:17.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.833+0000 7f9f51400640 1 --2- 172.21.15.114:0/4055610036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f540ca810 0x7f9f5415cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:17.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.833+0000 7f9f51400640 1 -- 172.21.15.114:0/4055610036 >> v1:172.21.15.114:6789/0 conn(0x7f9f541569d0 legacy=0x7f9f54073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:17.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.833+0000 7f9f51400640 1 -- 172.21.15.114:0/4055610036 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f540c92f0 con 0x7f9f54073b90 2024-05-27T09:27:17.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.833+0000 7f9f52800640 1 --2- 172.21.15.114:0/4055610036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f540ca810 0x7f9f5415cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:17.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.834+0000 7f9f51400640 1 --2- 172.21.15.114:0/4055610036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f54073b90 0x7f9f540ca2d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009d90 tx=0x7f9f3c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3efc79a367687ec server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:17.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.834+0000 7f9f50a00640 1 -- 172.21.15.114:0/4055610036 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c007740 con 0x7f9f54073b90 2024-05-27T09:27:17.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.834+0000 7f9f50a00640 1 -- 172.21.15.114:0/4055610036 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f3c0078e0 con 0x7f9f54073b90 2024-05-27T09:27:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.834+0000 7f9f50a00640 1 -- 172.21.15.114:0/4055610036 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c007cb0 con 0x7f9f54073b90 2024-05-27T09:27:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.835+0000 7f9f58a00640 1 -- 172.21.15.114:0/4055610036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f54073b90 msgr2=0x7f9f540ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.835+0000 7f9f58a00640 1 --2- 172.21.15.114:0/4055610036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f54073b90 0x7f9f540ca2d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009d90 tx=0x7f9f3c02a660 comp rx=0 tx=0).stop 2024-05-27T09:27:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.835+0000 7f9f58a00640 1 -- 172.21.15.114:0/4055610036 shutdown_connections 2024-05-27T09:27:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.835+0000 7f9f58a00640 1 --2- 172.21.15.114:0/4055610036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f540ca810 0x7f9f5415cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.835+0000 7f9f58a00640 1 --2- 172.21.15.114:0/4055610036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f54073b90 0x7f9f540ca2d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.835+0000 7f9f58a00640 1 -- 172.21.15.114:0/4055610036 >> 172.21.15.114:0/4055610036 conn(0x7f9f5406f6e0 msgr2=0x7f9f54070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.835+0000 7f9f58a00640 1 -- 172.21.15.114:0/4055610036 shutdown_connections 2024-05-27T09:27:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.835+0000 7f9f58a00640 1 -- 172.21.15.114:0/4055610036 wait complete. 2024-05-27T09:27:17.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.837+0000 7f9f58a00640 1 Processor -- start 2024-05-27T09:27:17.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.838+0000 7f9f58a00640 1 -- start start 2024-05-27T09:27:17.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.839+0000 7f9f58a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f54073b90 0x7f9f540c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:17.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.839+0000 7f9f58a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f540ca810 0x7f9f540c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:17.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.839+0000 7f9f58a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f541569d0 0x7f9f54155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:17.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.839+0000 7f9f58a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9f540c98b0 con 0x7f9f54073b90 2024-05-27T09:27:17.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.839+0000 7f9f58a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9f540c9f00 con 0x7f9f541569d0 2024-05-27T09:27:17.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.839+0000 7f9f58a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f9f540c9b90 con 0x7f9f540ca810 2024-05-27T09:27:17.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.839+0000 7f9f51e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f54073b90 0x7f9f540c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:17.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.839+0000 7f9f51400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f540ca810 0x7f9f540c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:17.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.839+0000 7f9f52800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f541569d0 0x7f9f54155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:17.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.839+0000 7f9f51400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f540ca810 0x7f9f540c8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:34426/0 (socket says 172.21.15.114:34426) 2024-05-27T09:27:17.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.839+0000 7f9f51400640 1 -- 172.21.15.114:0/1893922608 learned_addr learned my addr 172.21.15.114:0/1893922608 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:17.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.840+0000 7f9f52800640 1 -- 172.21.15.114:0/1893922608 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f540ca810 msgr2=0x7f9f540c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:17.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.840+0000 7f9f52800640 1 --2- 172.21.15.114:0/1893922608 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f540ca810 0x7f9f540c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:17.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.840+0000 7f9f52800640 1 -- 172.21.15.114:0/1893922608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f54073b90 msgr2=0x7f9f540c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:17.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.840+0000 7f9f52800640 1 --2- 172.21.15.114:0/1893922608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f54073b90 0x7f9f540c53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:17.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.840+0000 7f9f52800640 1 -- 172.21.15.114:0/1893922608 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f540b2bc0 con 0x7f9f541569d0 2024-05-27T09:27:17.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.840+0000 7f9f51e00640 1 --2- 172.21.15.114:0/1893922608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f54073b90 0x7f9f540c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:17.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.840+0000 7f9f52800640 1 --2- 172.21.15.114:0/1893922608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f541569d0 0x7f9f54155210 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00e7c0 tx=0x7f9f4c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:17.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.840+0000 7f9f4a000640 1 -- 172.21.15.114:0/1893922608 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c017070 con 0x7f9f541569d0 2024-05-27T09:27:17.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.840+0000 7f9f4a000640 1 -- 172.21.15.114:0/1893922608 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f4c00c940 con 0x7f9f541569d0 2024-05-27T09:27:17.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.840+0000 7f9f4a000640 1 -- 172.21.15.114:0/1893922608 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c00cd10 con 0x7f9f541569d0 2024-05-27T09:27:17.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.840+0000 7f9f58a00640 1 -- 172.21.15.114:0/1893922608 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f541f5d80 con 0x7f9f541569d0 2024-05-27T09:27:17.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.841+0000 7f9f58a00640 1 -- 172.21.15.114:0/1893922608 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f541f6320 con 0x7f9f541569d0 2024-05-27T09:27:17.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.842+0000 7f9f58a00640 1 -- 172.21.15.114:0/1893922608 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f5406a380 con 0x7f9f541569d0 2024-05-27T09:27:17.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.843+0000 7f9f4a000640 1 -- 172.21.15.114:0/1893922608 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9f4c01a070 con 0x7f9f541569d0 2024-05-27T09:27:17.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.844+0000 7f9f4a000640 1 --2- 172.21.15.114:0/1893922608 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9f300762d0 0x7f9f300787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:17.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.844+0000 7f9f4a000640 1 -- 172.21.15.114:0/1893922608 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f9f4c097a40 con 0x7f9f541569d0 2024-05-27T09:27:17.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.844+0000 7f9f51e00640 1 --2- 172.21.15.114:0/1893922608 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9f300762d0 0x7f9f300787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:17.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.845+0000 7f9f51e00640 1 --2- 172.21.15.114:0/1893922608 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9f300762d0 0x7f9f300787e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9f44007a60 tx=0x7f9f440079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:17.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:17.848+0000 7f9f4a000640 1 -- 172.21.15.114:0/1893922608 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9f4c060d90 con 0x7f9f541569d0 2024-05-27T09:27:18.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.076+0000 7f9f58a00640 1 -- 172.21.15.114:0/1893922608 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9f540c40a0 con 0x7f9f541569d0 2024-05-27T09:27:18.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.533+0000 7f9f4a000640 1 -- 172.21.15.114:0/1893922608 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f9f4c065090 con 0x7f9f541569d0 2024-05-27T09:27:18.535 INFO:teuthology.orchestra.run.smithi114.stderr:new fs with metadata pool 2 and data pool 3 2024-05-27T09:27:18.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.538+0000 7f9f58a00640 1 -- 172.21.15.114:0/1893922608 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9f300762d0 msgr2=0x7f9f300787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:18.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.538+0000 7f9f58a00640 1 --2- 172.21.15.114:0/1893922608 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9f300762d0 0x7f9f300787e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9f44007a60 tx=0x7f9f440079b0 comp rx=0 tx=0).stop 2024-05-27T09:27:18.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.538+0000 7f9f58a00640 1 -- 172.21.15.114:0/1893922608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f541569d0 msgr2=0x7f9f54155210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:18.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.538+0000 7f9f58a00640 1 --2- 172.21.15.114:0/1893922608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f541569d0 0x7f9f54155210 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00e7c0 tx=0x7f9f4c00ec80 comp rx=0 tx=0).stop 2024-05-27T09:27:18.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.539+0000 7f9f58a00640 1 -- 172.21.15.114:0/1893922608 shutdown_connections 2024-05-27T09:27:18.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.539+0000 7f9f58a00640 1 --2- 172.21.15.114:0/1893922608 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9f300762d0 0x7f9f300787e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:18.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.539+0000 7f9f58a00640 1 --2- 172.21.15.114:0/1893922608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f541569d0 0x7f9f54155210 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:18.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.539+0000 7f9f58a00640 1 --2- 172.21.15.114:0/1893922608 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f540ca810 0x7f9f540c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:18.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.539+0000 7f9f58a00640 1 --2- 172.21.15.114:0/1893922608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f54073b90 0x7f9f540c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:18.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.539+0000 7f9f58a00640 1 -- 172.21.15.114:0/1893922608 >> 172.21.15.114:0/1893922608 conn(0x7f9f5406f6e0 msgr2=0x7f9f540cd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:18.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.540+0000 7f9f58a00640 1 -- 172.21.15.114:0/1893922608 shutdown_connections 2024-05-27T09:27:18.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.540+0000 7f9f58a00640 1 -- 172.21.15.114:0/1893922608 wait complete. 2024-05-27T09:27:18.584 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T09:27:18.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.818+0000 7fb362a00640 1 Processor -- start 2024-05-27T09:27:18.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.819+0000 7fb362a00640 1 -- start start 2024-05-27T09:27:18.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.820+0000 7fb362a00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb35c15ad10 0x7fb35c0c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:18.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.821+0000 7fb362a00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb35c0c8470 con 0x7fb35c070060 2024-05-27T09:27:18.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.821+0000 7fb362a00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb35c06ef90 con 0x7fb35c074a80 2024-05-27T09:27:18.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.821+0000 7fb362a00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fb35c0c8220 con 0x7fb35c15ad10 2024-05-27T09:27:18.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.821+0000 7fb362000640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fb35c070060 0x7fb35c16b1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:41950/0 (socket says 172.21.15.114:41950) 2024-05-27T09:27:18.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.821+0000 7fb362000640 1 -- 172.21.15.114:0/1098186777 learned_addr learned my addr 172.21.15.114:0/1098186777 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:18.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.821+0000 7fb361600640 1 --2- 172.21.15.114:0/1098186777 >> v2:172.21.15.163:3300/0 conn(0x7fb35c15ad10 0x7fb35c0c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:18.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.821+0000 7fb35be00640 1 -- 172.21.15.114:0/1098186777 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3441031879 0 0) 0x7fb35c0c8470 con 0x7fb35c070060 2024-05-27T09:27:18.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.821+0000 7fb35be00640 1 -- 172.21.15.114:0/1098186777 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb344003520 con 0x7fb35c070060 2024-05-27T09:27:18.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.822+0000 7fb361600640 1 -- 172.21.15.114:0/1098186777 >> v1:172.21.15.163:6789/0 conn(0x7fb35c074a80 legacy=0x7fb35c0c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:18.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.822+0000 7fb361600640 1 -- 172.21.15.114:0/1098186777 >> v1:172.21.15.114:6789/0 conn(0x7fb35c070060 legacy=0x7fb35c16b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:18.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.822+0000 7fb361600640 1 -- 172.21.15.114:0/1098186777 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35c06f270 con 0x7fb35c15ad10 2024-05-27T09:27:18.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.823+0000 7fb361600640 1 --2- 172.21.15.114:0/1098186777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb35c15ad10 0x7fb35c0c96c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb35c0c83a0 tx=0x7fb35002a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0b293b5f5389da4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:18.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.823+0000 7fb35be00640 1 -- 172.21.15.114:0/1098186777 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350037070 con 0x7fb35c15ad10 2024-05-27T09:27:18.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.823+0000 7fb35be00640 1 -- 172.21.15.114:0/1098186777 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb350032640 con 0x7fb35c15ad10 2024-05-27T09:27:18.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.823+0000 7fb35be00640 1 -- 172.21.15.114:0/1098186777 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3500329f0 con 0x7fb35c15ad10 2024-05-27T09:27:18.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.824+0000 7fb362a00640 1 -- 172.21.15.114:0/1098186777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb35c15ad10 msgr2=0x7fb35c0c96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:18.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.824+0000 7fb362a00640 1 --2- 172.21.15.114:0/1098186777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb35c15ad10 0x7fb35c0c96c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb35c0c83a0 tx=0x7fb35002a2c0 comp rx=0 tx=0).stop 2024-05-27T09:27:18.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.824+0000 7fb362a00640 1 -- 172.21.15.114:0/1098186777 shutdown_connections 2024-05-27T09:27:18.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.824+0000 7fb362a00640 1 --2- 172.21.15.114:0/1098186777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb35c15ad10 0x7fb35c0c96c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:18.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.824+0000 7fb362a00640 1 -- 172.21.15.114:0/1098186777 >> 172.21.15.114:0/1098186777 conn(0x7fb35c0ca320 msgr2=0x7fb35c0ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:18.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.824+0000 7fb362a00640 1 -- 172.21.15.114:0/1098186777 shutdown_connections 2024-05-27T09:27:18.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.824+0000 7fb362a00640 1 -- 172.21.15.114:0/1098186777 wait complete. 2024-05-27T09:27:18.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.826+0000 7fb362a00640 1 Processor -- start 2024-05-27T09:27:18.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.826+0000 7fb362a00640 1 -- start start 2024-05-27T09:27:18.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.827+0000 7fb362a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb35c070060 0x7fb35c15cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:18.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.827+0000 7fb362a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb35c074a80 0x7fb35c0cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:18.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.827+0000 7fb362a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb35c15ad10 0x7fb35c15d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:18.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.827+0000 7fb362a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c0c2840 con 0x7fb35c070060 2024-05-27T09:27:18.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.827+0000 7fb362a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c06f830 con 0x7fb35c074a80 2024-05-27T09:27:18.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.827+0000 7fb362a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fb35c0c29c0 con 0x7fb35c15ad10 2024-05-27T09:27:18.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.828+0000 7fb361600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb35c070060 0x7fb35c15cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:18.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.828+0000 7fb361600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb35c070060 0x7fb35c15cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44300/0 (socket says 172.21.15.114:44300) 2024-05-27T09:27:18.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.828+0000 7fb361600640 1 -- 172.21.15.114:0/3471132952 learned_addr learned my addr 172.21.15.114:0/3471132952 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:18.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.828+0000 7fb360c00640 1 --2- 172.21.15.114:0/3471132952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb35c074a80 0x7fb35c0cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:18.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.828+0000 7fb362000640 1 --2- 172.21.15.114:0/3471132952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb35c15ad10 0x7fb35c15d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:18.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.828+0000 7fb361600640 1 -- 172.21.15.114:0/3471132952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb35c15ad10 msgr2=0x7fb35c15d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:18.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.828+0000 7fb361600640 1 --2- 172.21.15.114:0/3471132952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb35c15ad10 0x7fb35c15d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:18.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.828+0000 7fb361600640 1 -- 172.21.15.114:0/3471132952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb35c074a80 msgr2=0x7fb35c0cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:18.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.828+0000 7fb361600640 1 --2- 172.21.15.114:0/3471132952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb35c074a80 0x7fb35c0cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:18.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.828+0000 7fb361600640 1 -- 172.21.15.114:0/3471132952 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35c0cc120 con 0x7fb35c070060 2024-05-27T09:27:18.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.828+0000 7fb361600640 1 --2- 172.21.15.114:0/3471132952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb35c070060 0x7fb35c15cf70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb350035040 tx=0x7fb3500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:18.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.829+0000 7fb359600640 1 -- 172.21.15.114:0/3471132952 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350040020 con 0x7fb35c070060 2024-05-27T09:27:18.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.829+0000 7fb359600640 1 -- 172.21.15.114:0/3471132952 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb350038040 con 0x7fb35c070060 2024-05-27T09:27:18.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.829+0000 7fb362a00640 1 -- 172.21.15.114:0/3471132952 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35c15b7a0 con 0x7fb35c070060 2024-05-27T09:27:18.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.829+0000 7fb359600640 1 -- 172.21.15.114:0/3471132952 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350037070 con 0x7fb35c070060 2024-05-27T09:27:18.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.829+0000 7fb362a00640 1 -- 172.21.15.114:0/3471132952 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb35c15bd60 con 0x7fb35c070060 2024-05-27T09:27:18.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.830+0000 7fb362a00640 1 -- 172.21.15.114:0/3471132952 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb35c050860 con 0x7fb35c070060 2024-05-27T09:27:18.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.831+0000 7fb359600640 1 -- 172.21.15.114:0/3471132952 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb350043020 con 0x7fb35c070060 2024-05-27T09:27:18.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.832+0000 7fb359600640 1 --2- 172.21.15.114:0/3471132952 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb33c0762d0 0x7fb33c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:18.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.833+0000 7fb359600640 1 -- 172.21.15.114:0/3471132952 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb3500b78e0 con 0x7fb35c070060 2024-05-27T09:27:18.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.833+0000 7fb360c00640 1 --2- 172.21.15.114:0/3471132952 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb33c0762d0 0x7fb33c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:18.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.834+0000 7fb360c00640 1 --2- 172.21.15.114:0/3471132952 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb33c0762d0 0x7fb33c0787e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb35c0c9c60 tx=0x7fb348009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:18.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:18.838+0000 7fb359600640 1 -- 172.21.15.114:0/3471132952 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb35004a030 con 0x7fb35c070060 2024-05-27T09:27:19.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.046+0000 7fb362a00640 1 -- 172.21.15.114:0/3471132952 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb35c1566a0 con 0x7fb35c070060 2024-05-27T09:27:19.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.048+0000 7fb359600640 1 -- 172.21.15.114:0/3471132952 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16658 (secure 0 0 0) 0x7fb350080be0 con 0x7fb35c070060 2024-05-27T09:27:19.049 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:19.050 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":20,"fsid":"2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48","created":"2024-05-27T09:27:00.307278+0000","modified":"2024-05-27T09:27:18.079313+0000","last_up_change":"2024-05-27T09:27:11.463226+0000","last_in_change":"2024-05-27T09:27:02.972880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T09:27:10.796859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T09:27:16.063199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T09:27:17.064857+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3561c750-1d63-42f6-a6ae-0ee63721e15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6817","nonce":1960550710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6819","nonce":1960550710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6824","nonce":1960550710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6821","nonce":1960550710}]},"public_addr":"172.21.15.114:6817/1960550710","cluster_addr":"172.21.15.114:6819/1960550710","heartbeat_back_addr":"172.21.15.114:6824/1960550710","heartbeat_front_addr":"172.21.15.114:6821/1960550710","state":["exists","up"]},{"osd":1,"uuid":"9a407d04-38c0-4676-928f-d04e8116bd19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6825","nonce":326580057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6827","nonce":326580057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6831","nonce":326580057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6829","nonce":326580057}]},"public_addr":"172.21.15.114:6825/326580057","cluster_addr":"172.21.15.114:6827/326580057","heartbeat_back_addr":"172.21.15.114:6831/326580057","heartbeat_front_addr":"172.21.15.114:6829/326580057","state":["exists","up"]},{"osd":2,"uuid":"255a1cc6-9357-43f7-9db5-d760f4806ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6809","nonce":865568619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6811","nonce":865568619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6815","nonce":865568619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6813","nonce":865568619}]},"public_addr":"172.21.15.114:6809/865568619","cluster_addr":"172.21.15.114:6811/865568619","heartbeat_back_addr":"172.21.15.114:6815/865568619","heartbeat_front_addr":"172.21.15.114:6813/865568619","state":["exists","up"]},{"osd":3,"uuid":"cb4d7692-9a17-41b9-927e-d8eac3e217c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6801","nonce":1601792164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6803","nonce":1601792164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6807","nonce":1601792164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6805","nonce":1601792164}]},"public_addr":"172.21.15.114:6801/1601792164","cluster_addr":"172.21.15.114:6803/1601792164","heartbeat_back_addr":"172.21.15.114:6807/1601792164","heartbeat_front_addr":"172.21.15.114:6805/1601792164","state":["exists","up"]},{"osd":4,"uuid":"cdef1940-3453-4fbd-b0f1-459f5f5c7142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6817","nonce":1982246015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6819","nonce":1982246015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6823","nonce":1982246015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6821","nonce":1982246015}]},"public_addr":"172.21.15.163:6817/1982246015","cluster_addr":"172.21.15.163:6819/1982246015","heartbeat_back_addr":"172.21.15.163:6823/1982246015","heartbeat_front_addr":"172.21.15.163:6821/1982246015","state":["exists","up"]},{"osd":5,"uuid":"220a7b32-abcc-492f-bca8-4615cc2ea111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6801","nonce":1695294004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6803","nonce":1695294004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6807","nonce":1695294004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6805","nonce":1695294004}]},"public_addr":"172.21.15.163:6801/1695294004","cluster_addr":"172.21.15.163:6803/1695294004","heartbeat_back_addr":"172.21.15.163:6807/1695294004","heartbeat_front_addr":"172.21.15.163:6805/1695294004","state":["exists","up"]},{"osd":6,"uuid":"43a74d13-d03f-493a-a6c0-e73b0a056c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6809","nonce":3649718505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6811","nonce":3649718505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6815","nonce":3649718505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6813","nonce":3649718505}]},"public_addr":"172.21.15.163:6809/3649718505","cluster_addr":"172.21.15.163:6811/3649718505","heartbeat_back_addr":"172.21.15.163:6815/3649718505","heartbeat_front_addr":"172.21.15.163:6813/3649718505","state":["exists","up"]},{"osd":7,"uuid":"134629c2-70af-4b84-83a6-7ddf53681baa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6825","nonce":2539451311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6827","nonce":2539451311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6831","nonce":2539451311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6829","nonce":2539451311}]},"public_addr":"172.21.15.163:6825/2539451311","cluster_addr":"172.21.15.163:6827/2539451311","heartbeat_back_addr":"172.21.15.163:6831/2539451311","heartbeat_front_addr":"172.21.15.163:6829/2539451311","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.388942+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.515675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.646761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.400212+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.220072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.079531+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.180419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.175008+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T09:27:19.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.050+0000 7fb362a00640 1 -- 172.21.15.114:0/3471132952 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb33c0762d0 msgr2=0x7fb33c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:19.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.050+0000 7fb362a00640 1 --2- 172.21.15.114:0/3471132952 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb33c0762d0 0x7fb33c0787e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb35c0c9c60 tx=0x7fb348009040 comp rx=0 tx=0).stop 2024-05-27T09:27:19.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.050+0000 7fb362a00640 1 -- 172.21.15.114:0/3471132952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb35c070060 msgr2=0x7fb35c15cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:19.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.050+0000 7fb362a00640 1 --2- 172.21.15.114:0/3471132952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb35c070060 0x7fb35c15cf70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb350035040 tx=0x7fb3500046c0 comp rx=0 tx=0).stop 2024-05-27T09:27:19.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.051+0000 7fb362a00640 1 -- 172.21.15.114:0/3471132952 shutdown_connections 2024-05-27T09:27:19.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.051+0000 7fb362a00640 1 --2- 172.21.15.114:0/3471132952 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb33c0762d0 0x7fb33c0787e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:19.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.051+0000 7fb362a00640 1 --2- 172.21.15.114:0/3471132952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb35c15ad10 0x7fb35c15d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:19.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.051+0000 7fb362a00640 1 --2- 172.21.15.114:0/3471132952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb35c074a80 0x7fb35c0cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:19.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.051+0000 7fb362a00640 1 --2- 172.21.15.114:0/3471132952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb35c070060 0x7fb35c15cf70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:19.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.051+0000 7fb362a00640 1 -- 172.21.15.114:0/3471132952 >> 172.21.15.114:0/3471132952 conn(0x7fb35c0ca320 msgr2=0x7fb35c155040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:19.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.051+0000 7fb362a00640 1 -- 172.21.15.114:0/3471132952 shutdown_connections 2024-05-27T09:27:19.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.051+0000 7fb362a00640 1 -- 172.21.15.114:0/3471132952 wait complete. 2024-05-27T09:27:19.078 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T09:27:19.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.325+0000 7f96b5c00640 1 Processor -- start 2024-05-27T09:27:19.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.326+0000 7f96b5c00640 1 -- start start 2024-05-27T09:27:19.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.327+0000 7f96b5c00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f96b00741f0 0x7f96b00c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:19.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.327+0000 7f96b5c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96b01547d0 0x7f96b015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:19.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.327+0000 7f96b5c00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f96b0074840 0x7f96b006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:19.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.327+0000 7f96b5c00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f96b00703d0 con 0x7f96b0074840 2024-05-27T09:27:19.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.327+0000 7f96b5c00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f96b015b260 con 0x7f96b00741f0 2024-05-27T09:27:19.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.327+0000 7f96b5c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f96b006fe50 con 0x7f96b01547d0 2024-05-27T09:27:19.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.328+0000 7f96afe00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f96b00741f0 0x7f96b00c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:19.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.328+0000 7f96b5200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f96b0074840 0x7f96b006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:19.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.328+0000 7f96a7400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96b01547d0 0x7f96b015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:19.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.328+0000 7f96afe00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f96b00741f0 0x7f96b00c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:55112/0 (socket says 172.21.15.114:55112) 2024-05-27T09:27:19.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.328+0000 7f96afe00640 1 -- 172.21.15.114:0/2722244499 learned_addr learned my addr 172.21.15.114:0/2722244499 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:19.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.328+0000 7f96afe00640 1 -- 172.21.15.114:0/2722244499 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96b01547d0 msgr2=0x7f96b015ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:19.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.328+0000 7f96afe00640 1 --2- 172.21.15.114:0/2722244499 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96b01547d0 0x7f96b015ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:19.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.328+0000 7f96afe00640 1 -- 172.21.15.114:0/2722244499 >> v2:172.21.15.114:3300/0 conn(0x7f96b0074840 msgr2=0x7f96b006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:19.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.328+0000 7f96afe00640 1 --2- 172.21.15.114:0/2722244499 >> v2:172.21.15.114:3300/0 conn(0x7f96b0074840 0x7f96b006edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:19.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.328+0000 7f96afe00640 1 -- 172.21.15.114:0/2722244499 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b00700f0 con 0x7f96b00741f0 2024-05-27T09:27:19.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.329+0000 7f96afe00640 1 --2- 172.21.15.114:0/2722244499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96b00741f0 0x7f96b00c63a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f96b00caf30 tx=0x7f96a002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9570582fafd20c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:19.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.329+0000 7f96af400640 1 -- 172.21.15.114:0/2722244499 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0037070 con 0x7f96b00741f0 2024-05-27T09:27:19.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.329+0000 7f96af400640 1 -- 172.21.15.114:0/2722244499 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a0032440 con 0x7f96b00741f0 2024-05-27T09:27:19.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.329+0000 7f96af400640 1 -- 172.21.15.114:0/2722244499 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0032810 con 0x7f96b00741f0 2024-05-27T09:27:19.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.330+0000 7f96b5c00640 1 -- 172.21.15.114:0/2722244499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96b00741f0 msgr2=0x7f96b00c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:19.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.330+0000 7f96b5c00640 1 --2- 172.21.15.114:0/2722244499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96b00741f0 0x7f96b00c63a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f96b00caf30 tx=0x7f96a002a270 comp rx=0 tx=0).stop 2024-05-27T09:27:19.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.330+0000 7f96b5c00640 1 -- 172.21.15.114:0/2722244499 shutdown_connections 2024-05-27T09:27:19.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.330+0000 7f96b5c00640 1 --2- 172.21.15.114:0/2722244499 >> v2:172.21.15.114:3300/0 conn(0x7f96b0074840 0x7f96b006edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:19.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.330+0000 7f96b5c00640 1 --2- 172.21.15.114:0/2722244499 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96b01547d0 0x7f96b015ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:19.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.330+0000 7f96b5c00640 1 --2- 172.21.15.114:0/2722244499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96b00741f0 0x7f96b00c63a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:19.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.330+0000 7f96b5c00640 1 -- 172.21.15.114:0/2722244499 >> 172.21.15.114:0/2722244499 conn(0x7f96b00ca320 msgr2=0x7f96b00ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:19.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.330+0000 7f96b5c00640 1 -- 172.21.15.114:0/2722244499 shutdown_connections 2024-05-27T09:27:19.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.330+0000 7f96b5c00640 1 -- 172.21.15.114:0/2722244499 wait complete. 2024-05-27T09:27:19.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.332+0000 7f96b5c00640 1 Processor -- start 2024-05-27T09:27:19.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.332+0000 7f96b5c00640 1 -- start start 2024-05-27T09:27:19.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.334+0000 7f96b5c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96b00741f0 0x7f96b00cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:19.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.334+0000 7f96b5c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96b0074840 0x7f96b00cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:19.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.334+0000 7f96b5c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96b01547d0 0x7f96b0147d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:19.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.334+0000 7f96b5c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f96b0073790 con 0x7f96b00741f0 2024-05-27T09:27:19.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.334+0000 7f96b5c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f96b00734b0 con 0x7f96b0074840 2024-05-27T09:27:19.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.334+0000 7f96b5c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f96b0073cb0 con 0x7f96b01547d0 2024-05-27T09:27:19.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.334+0000 7f96afe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96b00741f0 0x7f96b00cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:19.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.334+0000 7f96afe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96b00741f0 0x7f96b00cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44316/0 (socket says 172.21.15.114:44316) 2024-05-27T09:27:19.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.334+0000 7f96afe00640 1 -- 172.21.15.114:0/391008019 learned_addr learned my addr 172.21.15.114:0/391008019 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:19.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.334+0000 7f96a7400640 1 --2- 172.21.15.114:0/391008019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96b0074840 0x7f96b00cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:19.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.334+0000 7f96b5200640 1 --2- 172.21.15.114:0/391008019 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96b01547d0 0x7f96b0147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:19.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.334+0000 7f96afe00640 1 -- 172.21.15.114:0/391008019 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96b01547d0 msgr2=0x7f96b0147d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:19.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.334+0000 7f96afe00640 1 --2- 172.21.15.114:0/391008019 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96b01547d0 0x7f96b0147d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:19.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.334+0000 7f96afe00640 1 -- 172.21.15.114:0/391008019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96b0074840 msgr2=0x7f96b00cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:19.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.334+0000 7f96afe00640 1 --2- 172.21.15.114:0/391008019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96b0074840 0x7f96b00cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:19.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.334+0000 7f96afe00640 1 -- 172.21.15.114:0/391008019 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b00cd5a0 con 0x7f96b00741f0 2024-05-27T09:27:19.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.335+0000 7f96afe00640 1 --2- 172.21.15.114:0/391008019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96b00741f0 0x7f96b00cc4c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f96a0000c00 tx=0x7f96a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:19.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.335+0000 7f96acc00640 1 -- 172.21.15.114:0/391008019 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0041020 con 0x7f96b00741f0 2024-05-27T09:27:19.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.335+0000 7f96acc00640 1 -- 172.21.15.114:0/391008019 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a003d620 con 0x7f96b00741f0 2024-05-27T09:27:19.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.335+0000 7f96acc00640 1 -- 172.21.15.114:0/391008019 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0037040 con 0x7f96b00741f0 2024-05-27T09:27:19.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.335+0000 7f96b5c00640 1 -- 172.21.15.114:0/391008019 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b0148240 con 0x7f96b00741f0 2024-05-27T09:27:19.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.335+0000 7f96b5c00640 1 -- 172.21.15.114:0/391008019 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b00c8570 con 0x7f96b00741f0 2024-05-27T09:27:19.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.336+0000 7f96b5c00640 1 -- 172.21.15.114:0/391008019 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b01514f0 con 0x7f96b00741f0 2024-05-27T09:27:19.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.338+0000 7f96acc00640 1 -- 172.21.15.114:0/391008019 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f96a0044020 con 0x7f96b00741f0 2024-05-27T09:27:19.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.338+0000 7f96acc00640 1 --2- 172.21.15.114:0/391008019 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f968c076300 0x7f968c078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:19.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.339+0000 7f96acc00640 1 -- 172.21.15.114:0/391008019 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f96a00b7c40 con 0x7f96b00741f0 2024-05-27T09:27:19.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.339+0000 7f96a7400640 1 --2- 172.21.15.114:0/391008019 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f968c076300 0x7f968c078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:19.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.339+0000 7f96a7400640 1 --2- 172.21.15.114:0/391008019 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f968c076300 0x7f968c078810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f96b01ead10 tx=0x7f9698009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:19.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.343+0000 7f96acc00640 1 -- 172.21.15.114:0/391008019 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f96a004b030 con 0x7f96b00741f0 2024-05-27T09:27:19.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.555+0000 7f96b5c00640 1 -- 172.21.15.114:0/391008019 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96b0151210 con 0x7f96b00741f0 2024-05-27T09:27:19.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.559+0000 7f96acc00640 1 -- 172.21.15.114:0/391008019 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16645 (secure 0 0 0) 0x7f96a0080e30 con 0x7f96b00741f0 2024-05-27T09:27:19.561 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:19.561 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"fsid":"2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48","created":"2024-05-27T09:27:00.307278+0000","modified":"2024-05-27T09:27:19.530054+0000","last_up_change":"2024-05-27T09:27:11.463226+0000","last_in_change":"2024-05-27T09:27:02.972880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T09:27:10.796859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T09:27:16.063199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T09:27:17.064857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3561c750-1d63-42f6-a6ae-0ee63721e15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6817","nonce":1960550710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6819","nonce":1960550710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6824","nonce":1960550710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6821","nonce":1960550710}]},"public_addr":"172.21.15.114:6817/1960550710","cluster_addr":"172.21.15.114:6819/1960550710","heartbeat_back_addr":"172.21.15.114:6824/1960550710","heartbeat_front_addr":"172.21.15.114:6821/1960550710","state":["exists","up"]},{"osd":1,"uuid":"9a407d04-38c0-4676-928f-d04e8116bd19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6825","nonce":326580057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6827","nonce":326580057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6831","nonce":326580057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6829","nonce":326580057}]},"public_addr":"172.21.15.114:6825/326580057","cluster_addr":"172.21.15.114:6827/326580057","heartbeat_back_addr":"172.21.15.114:6831/326580057","heartbeat_front_addr":"172.21.15.114:6829/326580057","state":["exists","up"]},{"osd":2,"uuid":"255a1cc6-9357-43f7-9db5-d760f4806ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6809","nonce":865568619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6811","nonce":865568619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6815","nonce":865568619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6813","nonce":865568619}]},"public_addr":"172.21.15.114:6809/865568619","cluster_addr":"172.21.15.114:6811/865568619","heartbeat_back_addr":"172.21.15.114:6815/865568619","heartbeat_front_addr":"172.21.15.114:6813/865568619","state":["exists","up"]},{"osd":3,"uuid":"cb4d7692-9a17-41b9-927e-d8eac3e217c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6801","nonce":1601792164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6803","nonce":1601792164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6807","nonce":1601792164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6805","nonce":1601792164}]},"public_addr":"172.21.15.114:6801/1601792164","cluster_addr":"172.21.15.114:6803/1601792164","heartbeat_back_addr":"172.21.15.114:6807/1601792164","heartbeat_front_addr":"172.21.15.114:6805/1601792164","state":["exists","up"]},{"osd":4,"uuid":"cdef1940-3453-4fbd-b0f1-459f5f5c7142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6817","nonce":1982246015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6819","nonce":1982246015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6823","nonce":1982246015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6821","nonce":1982246015}]},"public_addr":"172.21.15.163:6817/1982246015","cluster_addr":"172.21.15.163:6819/1982246015","heartbeat_back_addr":"172.21.15.163:6823/1982246015","heartbeat_front_addr":"172.21.15.163:6821/1982246015","state":["exists","up"]},{"osd":5,"uuid":"220a7b32-abcc-492f-bca8-4615cc2ea111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6801","nonce":1695294004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6803","nonce":1695294004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6807","nonce":1695294004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6805","nonce":1695294004}]},"public_addr":"172.21.15.163:6801/1695294004","cluster_addr":"172.21.15.163:6803/1695294004","heartbeat_back_addr":"172.21.15.163:6807/1695294004","heartbeat_front_addr":"172.21.15.163:6805/1695294004","state":["exists","up"]},{"osd":6,"uuid":"43a74d13-d03f-493a-a6c0-e73b0a056c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6809","nonce":3649718505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6811","nonce":3649718505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6815","nonce":3649718505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6813","nonce":3649718505}]},"public_addr":"172.21.15.163:6809/3649718505","cluster_addr":"172.21.15.163:6811/3649718505","heartbeat_back_addr":"172.21.15.163:6815/3649718505","heartbeat_front_addr":"172.21.15.163:6813/3649718505","state":["exists","up"]},{"osd":7,"uuid":"134629c2-70af-4b84-83a6-7ddf53681baa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6825","nonce":2539451311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6827","nonce":2539451311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6831","nonce":2539451311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6829","nonce":2539451311}]},"public_addr":"172.21.15.163:6825/2539451311","cluster_addr":"172.21.15.163:6827/2539451311","heartbeat_back_addr":"172.21.15.163:6831/2539451311","heartbeat_front_addr":"172.21.15.163:6829/2539451311","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.388942+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.515675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.646761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.400212+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.220072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.079531+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.180419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.175008+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T09:27:19.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.562+0000 7f96b5c00640 1 -- 172.21.15.114:0/391008019 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f968c076300 msgr2=0x7f968c078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:19.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.562+0000 7f96b5c00640 1 --2- 172.21.15.114:0/391008019 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f968c076300 0x7f968c078810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f96b01ead10 tx=0x7f9698009190 comp rx=0 tx=0).stop 2024-05-27T09:27:19.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.562+0000 7f96b5c00640 1 -- 172.21.15.114:0/391008019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96b00741f0 msgr2=0x7f96b00cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:19.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.562+0000 7f96b5c00640 1 --2- 172.21.15.114:0/391008019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96b00741f0 0x7f96b00cc4c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f96a0000c00 tx=0x7f96a00046c0 comp rx=0 tx=0).stop 2024-05-27T09:27:19.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.562+0000 7f96b5c00640 1 -- 172.21.15.114:0/391008019 shutdown_connections 2024-05-27T09:27:19.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.562+0000 7f96b5c00640 1 --2- 172.21.15.114:0/391008019 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f968c076300 0x7f968c078810 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:19.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.562+0000 7f96b5c00640 1 --2- 172.21.15.114:0/391008019 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96b01547d0 0x7f96b0147d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:19.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.562+0000 7f96b5c00640 1 --2- 172.21.15.114:0/391008019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96b0074840 0x7f96b00cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:19.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.562+0000 7f96b5c00640 1 --2- 172.21.15.114:0/391008019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96b00741f0 0x7f96b00cc4c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:19.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.562+0000 7f96b5c00640 1 -- 172.21.15.114:0/391008019 >> 172.21.15.114:0/391008019 conn(0x7f96b00ca320 msgr2=0x7f96b0153130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:19.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.563+0000 7f96b5c00640 1 -- 172.21.15.114:0/391008019 shutdown_connections 2024-05-27T09:27:19.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.563+0000 7f96b5c00640 1 -- 172.21.15.114:0/391008019 wait complete. 2024-05-27T09:27:19.589 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-27T09:27:19.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.844+0000 7efc95c00640 1 Processor -- start 2024-05-27T09:27:19.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.844+0000 7efc95c00640 1 -- start start 2024-05-27T09:27:19.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.847+0000 7efc95c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc9015ad10 0x7efc900709d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:19.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.847+0000 7efc95c00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efc90070f10 0x7efc9016b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:19.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.847+0000 7efc95c00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc900c9240 con 0x7efc900ce440 2024-05-27T09:27:19.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.847+0000 7efc95c00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7efc900c9800 con 0x7efc90070f10 2024-05-27T09:27:19.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.847+0000 7efc95c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7efc900c9ae0 con 0x7efc9015ad10 2024-05-27T09:27:19.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.847+0000 7efc8fe00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7efc900ce440 0x7efc900c8290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:41966/0 (socket says 172.21.15.114:41966) 2024-05-27T09:27:19.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.847+0000 7efc87400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc9015ad10 0x7efc900709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:19.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.847+0000 7efc8fe00640 1 -- 172.21.15.114:0/678983142 learned_addr learned my addr 172.21.15.114:0/678983142 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:19.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.847+0000 7efc95200640 1 --2- 172.21.15.114:0/678983142 >> v2:172.21.15.163:3300/0 conn(0x7efc90070f10 0x7efc9016b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:19.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.847+0000 7efc87400640 1 -- 172.21.15.114:0/678983142 >> v2:172.21.15.163:3300/0 conn(0x7efc90070f10 msgr2=0x7efc9016b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:19.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.847+0000 7efc87400640 1 --2- 172.21.15.114:0/678983142 >> v2:172.21.15.163:3300/0 conn(0x7efc90070f10 0x7efc9016b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:19.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.847+0000 7efc87400640 1 -- 172.21.15.114:0/678983142 >> v1:172.21.15.114:6789/0 conn(0x7efc900ce440 legacy=0x7efc900c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:19.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.847+0000 7efc8f400640 1 -- 172.21.15.114:0/678983142 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1439804547 0 0) 0x7efc900c9240 con 0x7efc900ce440 2024-05-27T09:27:19.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.847+0000 7efc87400640 1 -- 172.21.15.114:0/678983142 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc900c9dc0 con 0x7efc9015ad10 2024-05-27T09:27:19.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.849+0000 7efc87400640 1 --2- 172.21.15.114:0/678983142 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc9015ad10 0x7efc900709d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efc780097a0 tx=0x7efc7802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a0dd83f1daa63d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:19.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.849+0000 7efc8f400640 1 -- 172.21.15.114:0/678983142 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc78037070 con 0x7efc9015ad10 2024-05-27T09:27:19.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.849+0000 7efc8f400640 1 -- 172.21.15.114:0/678983142 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc78032440 con 0x7efc9015ad10 2024-05-27T09:27:19.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.849+0000 7efc8f400640 1 -- 172.21.15.114:0/678983142 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc78032810 con 0x7efc9015ad10 2024-05-27T09:27:19.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.850+0000 7efc95c00640 1 -- 172.21.15.114:0/678983142 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc9015ad10 msgr2=0x7efc900709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:19.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.850+0000 7efc95c00640 1 --2- 172.21.15.114:0/678983142 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc9015ad10 0x7efc900709d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efc780097a0 tx=0x7efc7802a180 comp rx=0 tx=0).stop 2024-05-27T09:27:19.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.850+0000 7efc95c00640 1 -- 172.21.15.114:0/678983142 shutdown_connections 2024-05-27T09:27:19.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.850+0000 7efc95c00640 1 --2- 172.21.15.114:0/678983142 >> v2:172.21.15.163:3300/0 conn(0x7efc90070f10 0x7efc9016b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:19.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.850+0000 7efc95c00640 1 --2- 172.21.15.114:0/678983142 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc9015ad10 0x7efc900709d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:19.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.850+0000 7efc95c00640 1 -- 172.21.15.114:0/678983142 >> 172.21.15.114:0/678983142 conn(0x7efc900cd890 msgr2=0x7efc900cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:19.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.850+0000 7efc95c00640 1 -- 172.21.15.114:0/678983142 shutdown_connections 2024-05-27T09:27:19.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.850+0000 7efc95c00640 1 -- 172.21.15.114:0/678983142 wait complete. 2024-05-27T09:27:19.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.852+0000 7efc95c00640 1 Processor -- start 2024-05-27T09:27:19.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.852+0000 7efc95c00640 1 -- start start 2024-05-27T09:27:19.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.854+0000 7efc95c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc90070f10 0x7efc900ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:19.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.854+0000 7efc95c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc900ce440 0x7efc901faab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:19.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.854+0000 7efc95c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc9015ad10 0x7efc900cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:19.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.854+0000 7efc95c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efc90074720 con 0x7efc90070f10 2024-05-27T09:27:19.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.854+0000 7efc95c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7efc900745a0 con 0x7efc900ce440 2024-05-27T09:27:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.854+0000 7efc95c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7efc900c28b0 con 0x7efc9015ad10 2024-05-27T09:27:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.854+0000 7efc8fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc90070f10 0x7efc900ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.854+0000 7efc8fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc90070f10 0x7efc900ccf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44318/0 (socket says 172.21.15.114:44318) 2024-05-27T09:27:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.854+0000 7efc8fe00640 1 -- 172.21.15.114:0/4194114617 learned_addr learned my addr 172.21.15.114:0/4194114617 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.854+0000 7efc95200640 1 --2- 172.21.15.114:0/4194114617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc9015ad10 0x7efc900cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.854+0000 7efc87400640 1 --2- 172.21.15.114:0/4194114617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc900ce440 0x7efc901faab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.854+0000 7efc8fe00640 1 -- 172.21.15.114:0/4194114617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc9015ad10 msgr2=0x7efc900cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.854+0000 7efc8fe00640 1 --2- 172.21.15.114:0/4194114617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc9015ad10 0x7efc900cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.854+0000 7efc8fe00640 1 -- 172.21.15.114:0/4194114617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc900ce440 msgr2=0x7efc901faab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.854+0000 7efc8fe00640 1 --2- 172.21.15.114:0/4194114617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc900ce440 0x7efc901faab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.854+0000 7efc8fe00640 1 -- 172.21.15.114:0/4194114617 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc900cbbd0 con 0x7efc90070f10 2024-05-27T09:27:19.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.855+0000 7efc8fe00640 1 --2- 172.21.15.114:0/4194114617 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc90070f10 0x7efc900ccf10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efc8000c3e0 tx=0x7efc8000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:19.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.855+0000 7efc8cc00640 1 -- 172.21.15.114:0/4194114617 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc800074e0 con 0x7efc90070f10 2024-05-27T09:27:19.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.855+0000 7efc8cc00640 1 -- 172.21.15.114:0/4194114617 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc800047e0 con 0x7efc90070f10 2024-05-27T09:27:19.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.855+0000 7efc95c00640 1 -- 172.21.15.114:0/4194114617 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc901631a0 con 0x7efc90070f10 2024-05-27T09:27:19.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.855+0000 7efc8cc00640 1 -- 172.21.15.114:0/4194114617 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc80004bd0 con 0x7efc90070f10 2024-05-27T09:27:19.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.855+0000 7efc95c00640 1 -- 172.21.15.114:0/4194114617 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc90163760 con 0x7efc90070f10 2024-05-27T09:27:19.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.856+0000 7efc95c00640 1 -- 172.21.15.114:0/4194114617 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc900c31b0 con 0x7efc90070f10 2024-05-27T09:27:19.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.857+0000 7efc8cc00640 1 -- 172.21.15.114:0/4194114617 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efc80011070 con 0x7efc90070f10 2024-05-27T09:27:19.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.859+0000 7efc8cc00640 1 --2- 172.21.15.114:0/4194114617 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7efc6c076300 0x7efc6c078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:19.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.859+0000 7efc8cc00640 1 -- 172.21.15.114:0/4194114617 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efc80097180 con 0x7efc90070f10 2024-05-27T09:27:19.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.859+0000 7efc87400640 1 --2- 172.21.15.114:0/4194114617 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7efc6c076300 0x7efc6c078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:19.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.860+0000 7efc87400640 1 --2- 172.21.15.114:0/4194114617 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7efc6c076300 0x7efc6c078810 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efc78034000 tx=0x7efc7802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:19.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:19.864+0000 7efc8cc00640 1 -- 172.21.15.114:0/4194114617 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efc900c31b0 con 0x7efc90070f10 2024-05-27T09:27:20.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.091+0000 7efc95c00640 1 -- 172.21.15.114:0/4194114617 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7efc90050860 con 0x7efc90070f10 2024-05-27T09:27:20.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.563+0000 7efc8cc00640 1 -- 172.21.15.114:0/4194114617 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7efc80060360 con 0x7efc90070f10 2024-05-27T09:27:20.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.567+0000 7efc95c00640 1 -- 172.21.15.114:0/4194114617 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7efc6c076300 msgr2=0x7efc6c078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:20.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.567+0000 7efc95c00640 1 --2- 172.21.15.114:0/4194114617 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7efc6c076300 0x7efc6c078810 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efc78034000 tx=0x7efc7802e000 comp rx=0 tx=0).stop 2024-05-27T09:27:20.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.567+0000 7efc95c00640 1 -- 172.21.15.114:0/4194114617 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc90070f10 msgr2=0x7efc900ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:20.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.567+0000 7efc95c00640 1 --2- 172.21.15.114:0/4194114617 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc90070f10 0x7efc900ccf10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efc8000c3e0 tx=0x7efc8000c8a0 comp rx=0 tx=0).stop 2024-05-27T09:27:20.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.567+0000 7efc95c00640 1 -- 172.21.15.114:0/4194114617 shutdown_connections 2024-05-27T09:27:20.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.567+0000 7efc95c00640 1 --2- 172.21.15.114:0/4194114617 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7efc6c076300 0x7efc6c078810 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:20.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.567+0000 7efc95c00640 1 --2- 172.21.15.114:0/4194114617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc9015ad10 0x7efc900cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:20.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.567+0000 7efc95c00640 1 --2- 172.21.15.114:0/4194114617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc900ce440 0x7efc901faab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:20.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.567+0000 7efc95c00640 1 --2- 172.21.15.114:0/4194114617 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc90070f10 0x7efc900ccf10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:20.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.567+0000 7efc95c00640 1 -- 172.21.15.114:0/4194114617 >> 172.21.15.114:0/4194114617 conn(0x7efc900cd890 msgr2=0x7efc90158dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:20.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.567+0000 7efc95c00640 1 -- 172.21.15.114:0/4194114617 shutdown_connections 2024-05-27T09:27:20.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.567+0000 7efc95c00640 1 -- 172.21.15.114:0/4194114617 wait complete. 2024-05-27T09:27:20.594 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300} 2024-05-27T09:27:20.594 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-05-27T09:27:20.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.859+0000 7f7edd000640 1 Processor -- start 2024-05-27T09:27:20.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.860+0000 7f7edd000640 1 -- start start 2024-05-27T09:27:20.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.862+0000 7f7edd000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ed80ca9e0 0x7f7ed815cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:20.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.862+0000 7f7edd000640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ed80c92f0 con 0x7f7ed81569d0 2024-05-27T09:27:20.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.862+0000 7f7edd000640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ed80c95d0 con 0x7f7ed8073b90 2024-05-27T09:27:20.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.862+0000 7f7edd000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed8074d70 con 0x7f7ed80ca9e0 2024-05-27T09:27:20.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.862+0000 7f7ed5e00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f7ed81569d0 0x7f7ed8073260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:41974/0 (socket says 172.21.15.114:41974) 2024-05-27T09:27:20.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.862+0000 7f7ed5e00640 1 -- 172.21.15.114:0/3588721483 learned_addr learned my addr 172.21.15.114:0/3588721483 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:20.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.862+0000 7f7ed6800640 1 --2- 172.21.15.114:0/3588721483 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ed80ca9e0 0x7f7ed815cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:20.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.863+0000 7f7ed4a00640 1 -- 172.21.15.114:0/3588721483 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1261712104 0 0) 0x7f7ed80c92f0 con 0x7f7ed81569d0 2024-05-27T09:27:20.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.863+0000 7f7ed4a00640 1 -- 172.21.15.114:0/3588721483 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ebc003520 con 0x7f7ed81569d0 2024-05-27T09:27:20.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.863+0000 7f7ed4a00640 1 -- 172.21.15.114:0/3588721483 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2404262687 0 0) 0x7f7ed80c95d0 con 0x7f7ed8073b90 2024-05-27T09:27:20.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.863+0000 7f7ed4a00640 1 -- 172.21.15.114:0/3588721483 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ed80c92f0 con 0x7f7ed8073b90 2024-05-27T09:27:20.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.863+0000 7f7ed6800640 1 -- 172.21.15.114:0/3588721483 >> v1:172.21.15.163:6789/0 conn(0x7f7ed8073b90 legacy=0x7f7ed80ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:20.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.863+0000 7f7ed4a00640 1 -- 172.21.15.114:0/3588721483 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 624532664 0 0) 0x7f7ebc003520 con 0x7f7ed81569d0 2024-05-27T09:27:20.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.863+0000 7f7ed6800640 1 -- 172.21.15.114:0/3588721483 >> v1:172.21.15.114:6789/0 conn(0x7f7ed81569d0 legacy=0x7f7ed8073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:20.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.863+0000 7f7ed6800640 1 -- 172.21.15.114:0/3588721483 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed80c98b0 con 0x7f7ed80ca9e0 2024-05-27T09:27:20.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.864+0000 7f7ed6800640 1 --2- 172.21.15.114:0/3588721483 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ed80ca9e0 0x7f7ed815cb50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00097d0 tx=0x7f7ed002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91aba10bef08126f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:20.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.864+0000 7f7ed4a00640 1 -- 172.21.15.114:0/3588721483 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0037070 con 0x7f7ed80ca9e0 2024-05-27T09:27:20.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.864+0000 7f7ed4a00640 1 -- 172.21.15.114:0/3588721483 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed002ac30 con 0x7f7ed80ca9e0 2024-05-27T09:27:20.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.864+0000 7f7ed4a00640 1 -- 172.21.15.114:0/3588721483 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8001a70 con 0x7f7ed80ca9e0 2024-05-27T09:27:20.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.865+0000 7f7edd000640 1 -- 172.21.15.114:0/3588721483 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ed80ca9e0 msgr2=0x7f7ed815cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:20.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.865+0000 7f7edd000640 1 --2- 172.21.15.114:0/3588721483 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ed80ca9e0 0x7f7ed815cb50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00097d0 tx=0x7f7ed002a1a0 comp rx=0 tx=0).stop 2024-05-27T09:27:20.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.865+0000 7f7edd000640 1 -- 172.21.15.114:0/3588721483 shutdown_connections 2024-05-27T09:27:20.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.865+0000 7f7edd000640 1 --2- 172.21.15.114:0/3588721483 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ed80ca9e0 0x7f7ed815cb50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:20.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.865+0000 7f7edd000640 1 -- 172.21.15.114:0/3588721483 >> 172.21.15.114:0/3588721483 conn(0x7f7ed806f6e0 msgr2=0x7f7ed8070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:20.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.865+0000 7f7edd000640 1 -- 172.21.15.114:0/3588721483 shutdown_connections 2024-05-27T09:27:20.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.865+0000 7f7edd000640 1 -- 172.21.15.114:0/3588721483 wait complete. 2024-05-27T09:27:20.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.867+0000 7f7edd000640 1 Processor -- start 2024-05-27T09:27:20.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.867+0000 7f7edd000640 1 -- start start 2024-05-27T09:27:20.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.869+0000 7f7edd000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ed8073b90 0x7f7ed81e8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:20.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.869+0000 7f7edd000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ed80ca9e0 0x7f7ed80c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:20.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.869+0000 7f7edd000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ed81569d0 0x7f7ed814fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:20.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.869+0000 7f7edd000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed80c9f00 con 0x7f7ed8073b90 2024-05-27T09:27:20.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.869+0000 7f7edd000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed80c90a0 con 0x7f7ed81569d0 2024-05-27T09:27:20.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.869+0000 7f7edd000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed8156de0 con 0x7f7ed80ca9e0 2024-05-27T09:27:20.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.869+0000 7f7ed5e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ed8073b90 0x7f7ed81e8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:20.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.869+0000 7f7ed5e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ed8073b90 0x7f7ed81e8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44324/0 (socket says 172.21.15.114:44324) 2024-05-27T09:27:20.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.869+0000 7f7ed5e00640 1 -- 172.21.15.114:0/4025241720 learned_addr learned my addr 172.21.15.114:0/4025241720 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:20.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.869+0000 7f7ed5400640 1 --2- 172.21.15.114:0/4025241720 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ed80ca9e0 0x7f7ed80c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:20.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.869+0000 7f7ed6800640 1 --2- 172.21.15.114:0/4025241720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ed81569d0 0x7f7ed814fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:20.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.869+0000 7f7ed5e00640 1 -- 172.21.15.114:0/4025241720 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ed80ca9e0 msgr2=0x7f7ed80c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:20.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.869+0000 7f7ed5e00640 1 --2- 172.21.15.114:0/4025241720 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ed80ca9e0 0x7f7ed80c65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:20.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.869+0000 7f7ed5e00640 1 -- 172.21.15.114:0/4025241720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ed81569d0 msgr2=0x7f7ed814fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:20.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.870+0000 7f7ed5e00640 1 --2- 172.21.15.114:0/4025241720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ed81569d0 0x7f7ed814fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:20.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.870+0000 7f7ed5e00640 1 -- 172.21.15.114:0/4025241720 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed80c4eb0 con 0x7f7ed8073b90 2024-05-27T09:27:20.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.870+0000 7f7ed5e00640 1 --2- 172.21.15.114:0/4025241720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ed8073b90 0x7f7ed81e8db0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800cbd0 tx=0x7f7ec800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:20.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.870+0000 7f7ece000640 1 -- 172.21.15.114:0/4025241720 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8017070 con 0x7f7ed8073b90 2024-05-27T09:27:20.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.870+0000 7f7ece000640 1 -- 172.21.15.114:0/4025241720 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec800ad00 con 0x7f7ed8073b90 2024-05-27T09:27:20.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.870+0000 7f7ece000640 1 -- 172.21.15.114:0/4025241720 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8012640 con 0x7f7ed8073b90 2024-05-27T09:27:20.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.870+0000 7f7edd000640 1 -- 172.21.15.114:0/4025241720 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed80c7020 con 0x7f7ed8073b90 2024-05-27T09:27:20.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.870+0000 7f7edd000640 1 -- 172.21.15.114:0/4025241720 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed81e7320 con 0x7f7ed8073b90 2024-05-27T09:27:20.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.871+0000 7f7edd000640 1 -- 172.21.15.114:0/4025241720 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed80b3d80 con 0x7f7ed8073b90 2024-05-27T09:27:20.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.872+0000 7f7ece000640 1 -- 172.21.15.114:0/4025241720 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7ec801a070 con 0x7f7ed8073b90 2024-05-27T09:27:20.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.873+0000 7f7ece000640 1 --2- 172.21.15.114:0/4025241720 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7eb4076300 0x7f7eb4078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:20.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.874+0000 7f7ece000640 1 -- 172.21.15.114:0/4025241720 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7ec8097640 con 0x7f7ed8073b90 2024-05-27T09:27:20.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.874+0000 7f7ed5400640 1 --2- 172.21.15.114:0/4025241720 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7eb4076300 0x7f7eb4078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:20.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.874+0000 7f7ed5400640 1 --2- 172.21.15.114:0/4025241720 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7eb4076300 0x7f7eb4078810 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7ed81e9b40 tx=0x7f7ec0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:20.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:20.878+0000 7f7ece000640 1 -- 172.21.15.114:0/4025241720 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7ed80b3d80 con 0x7f7ed8073b90 2024-05-27T09:27:21.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.125+0000 7f7edd000640 1 -- 172.21.15.114:0/4025241720 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f7ed80cadf0 con 0x7f7ed8073b90 2024-05-27T09:27:21.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.573+0000 7f7ece000640 1 -- 172.21.15.114:0/4025241720 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v6) ==== 105+0+0 (secure 0 0 0) 0x7f7ed80cadf0 con 0x7f7ed8073b90 2024-05-27T09:27:21.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.576+0000 7f7edd000640 1 -- 172.21.15.114:0/4025241720 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7eb4076300 msgr2=0x7f7eb4078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:21.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.576+0000 7f7edd000640 1 --2- 172.21.15.114:0/4025241720 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7eb4076300 0x7f7eb4078810 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7ed81e9b40 tx=0x7f7ec0009040 comp rx=0 tx=0).stop 2024-05-27T09:27:21.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.576+0000 7f7edd000640 1 -- 172.21.15.114:0/4025241720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ed8073b90 msgr2=0x7f7ed81e8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:21.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.576+0000 7f7edd000640 1 --2- 172.21.15.114:0/4025241720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ed8073b90 0x7f7ed81e8db0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800cbd0 tx=0x7f7ec800a430 comp rx=0 tx=0).stop 2024-05-27T09:27:21.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.576+0000 7f7edd000640 1 -- 172.21.15.114:0/4025241720 shutdown_connections 2024-05-27T09:27:21.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.576+0000 7f7edd000640 1 --2- 172.21.15.114:0/4025241720 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7eb4076300 0x7f7eb4078810 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:21.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.577+0000 7f7edd000640 1 --2- 172.21.15.114:0/4025241720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ed81569d0 0x7f7ed814fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:21.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.577+0000 7f7edd000640 1 --2- 172.21.15.114:0/4025241720 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ed80ca9e0 0x7f7ed80c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:21.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.577+0000 7f7edd000640 1 --2- 172.21.15.114:0/4025241720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ed8073b90 0x7f7ed81e8db0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:21.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.577+0000 7f7edd000640 1 -- 172.21.15.114:0/4025241720 >> 172.21.15.114:0/4025241720 conn(0x7f7ed806f6e0 msgr2=0x7f7ed80cd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:21.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.577+0000 7f7edd000640 1 -- 172.21.15.114:0/4025241720 shutdown_connections 2024-05-27T09:27:21.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.577+0000 7f7edd000640 1 -- 172.21.15.114:0/4025241720 wait complete. 2024-05-27T09:27:21.604 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-27T09:27:21.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.872+0000 7f68d2a00640 1 Processor -- start 2024-05-27T09:27:21.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.872+0000 7f68d2a00640 1 -- start start 2024-05-27T09:27:21.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.875+0000 7f68d2a00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f68cc0c9750 0x7f68cc0c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:21.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.875+0000 7f68d2a00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68cc070940 con 0x7f68cc071780 2024-05-27T09:27:21.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.875+0000 7f68d2a00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68cc070c10 con 0x7f68cc15afa0 2024-05-27T09:27:21.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.875+0000 7f68d2a00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f68cc0c6530 con 0x7f68cc0c9750 2024-05-27T09:27:21.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.875+0000 7f68d1600640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f68cc071780 0x7f68cc1546d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:41978/0 (socket says 172.21.15.114:41978) 2024-05-27T09:27:21.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.875+0000 7f68d1600640 1 -- 172.21.15.114:0/1290976889 learned_addr learned my addr 172.21.15.114:0/1290976889 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:21.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.875+0000 7f68d2000640 1 --2- 172.21.15.114:0/1290976889 >> v2:172.21.15.163:3300/0 conn(0x7f68cc0c9750 0x7f68cc0c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:21.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.875+0000 7f68cbe00640 1 -- 172.21.15.114:0/1290976889 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3212495923 0 0) 0x7f68cc070940 con 0x7f68cc071780 2024-05-27T09:27:21.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.875+0000 7f68cbe00640 1 -- 172.21.15.114:0/1290976889 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68b4003520 con 0x7f68cc071780 2024-05-27T09:27:21.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.876+0000 7f68d2000640 1 -- 172.21.15.114:0/1290976889 >> v1:172.21.15.163:6789/0 conn(0x7f68cc15afa0 legacy=0x7f68cc0c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:21.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.876+0000 7f68cbe00640 1 -- 172.21.15.114:0/1290976889 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3180998228 0 0) 0x7f68cc070c10 con 0x7f68cc15afa0 2024-05-27T09:27:21.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.876+0000 7f68d2000640 1 -- 172.21.15.114:0/1290976889 >> v1:172.21.15.114:6789/0 conn(0x7f68cc071780 legacy=0x7f68cc1546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:21.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.876+0000 7f68d2000640 1 -- 172.21.15.114:0/1290976889 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68cc070ef0 con 0x7f68cc0c9750 2024-05-27T09:27:21.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.877+0000 7f68d2000640 1 --2- 172.21.15.114:0/1290976889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68cc0c9750 0x7f68cc0c9b60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f68c40097a0 tx=0x7f68c402a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4167a62af1ff1178 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:21.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.877+0000 7f68cbe00640 1 -- 172.21.15.114:0/1290976889 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c4037070 con 0x7f68cc0c9750 2024-05-27T09:27:21.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.877+0000 7f68cbe00640 1 -- 172.21.15.114:0/1290976889 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68c4032640 con 0x7f68cc0c9750 2024-05-27T09:27:21.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.877+0000 7f68cbe00640 1 -- 172.21.15.114:0/1290976889 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c40329f0 con 0x7f68cc0c9750 2024-05-27T09:27:21.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.878+0000 7f68d2a00640 1 -- 172.21.15.114:0/1290976889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68cc0c9750 msgr2=0x7f68cc0c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:21.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.878+0000 7f68d2a00640 1 --2- 172.21.15.114:0/1290976889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68cc0c9750 0x7f68cc0c9b60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f68c40097a0 tx=0x7f68c402a2c0 comp rx=0 tx=0).stop 2024-05-27T09:27:21.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.878+0000 7f68d2a00640 1 -- 172.21.15.114:0/1290976889 shutdown_connections 2024-05-27T09:27:21.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.878+0000 7f68d2a00640 1 --2- 172.21.15.114:0/1290976889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68cc0c9750 0x7f68cc0c9b60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:21.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.878+0000 7f68d2a00640 1 -- 172.21.15.114:0/1290976889 >> 172.21.15.114:0/1290976889 conn(0x7f68cc0cd890 msgr2=0x7f68cc0cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:21.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.878+0000 7f68d2a00640 1 -- 172.21.15.114:0/1290976889 shutdown_connections 2024-05-27T09:27:21.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.878+0000 7f68d2a00640 1 -- 172.21.15.114:0/1290976889 wait complete. 2024-05-27T09:27:21.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.881+0000 7f68d2a00640 1 Processor -- start 2024-05-27T09:27:21.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.881+0000 7f68d2a00640 1 -- start start 2024-05-27T09:27:21.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.883+0000 7f68d2a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68cc071780 0x7f68cc1507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:21.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.883+0000 7f68d2a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68cc0c9750 0x7f68cc075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:21.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.883+0000 7f68d2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68cc15afa0 0x7f68cc1f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:21.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.883+0000 7f68d2a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc0c8430 con 0x7f68cc15afa0 2024-05-27T09:27:21.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.883+0000 7f68d2a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc0714b0 con 0x7f68cc0c9750 2024-05-27T09:27:21.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.883+0000 7f68d2a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f68cc0c8710 con 0x7f68cc071780 2024-05-27T09:27:21.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.883+0000 7f68d2000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68cc15afa0 0x7f68cc1f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:21.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.883+0000 7f68d2000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68cc15afa0 0x7f68cc1f57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44334/0 (socket says 172.21.15.114:44334) 2024-05-27T09:27:21.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.883+0000 7f68d2000640 1 -- 172.21.15.114:0/3442702825 learned_addr learned my addr 172.21.15.114:0/3442702825 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:21.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.883+0000 7f68d1600640 1 --2- 172.21.15.114:0/3442702825 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68cc071780 0x7f68cc1507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:21.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.883+0000 7f68d0c00640 1 --2- 172.21.15.114:0/3442702825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68cc0c9750 0x7f68cc075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:21.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.884+0000 7f68d2000640 1 -- 172.21.15.114:0/3442702825 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68cc071780 msgr2=0x7f68cc1507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:21.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.884+0000 7f68d2000640 1 --2- 172.21.15.114:0/3442702825 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68cc071780 0x7f68cc1507f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:21.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.884+0000 7f68d2000640 1 -- 172.21.15.114:0/3442702825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68cc0c9750 msgr2=0x7f68cc075f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:21.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.884+0000 7f68d2000640 1 --2- 172.21.15.114:0/3442702825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68cc0c9750 0x7f68cc075f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:21.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.884+0000 7f68d2000640 1 -- 172.21.15.114:0/3442702825 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68cc150d30 con 0x7f68cc15afa0 2024-05-27T09:27:21.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.884+0000 7f68d2000640 1 --2- 172.21.15.114:0/3442702825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68cc15afa0 0x7f68cc1f57f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f68c402a300 tx=0x7f68c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:21.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.884+0000 7f68c9600640 1 -- 172.21.15.114:0/3442702825 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c4041020 con 0x7f68cc15afa0 2024-05-27T09:27:21.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.884+0000 7f68c9600640 1 -- 172.21.15.114:0/3442702825 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68c4003610 con 0x7f68cc15afa0 2024-05-27T09:27:21.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.884+0000 7f68d2a00640 1 -- 172.21.15.114:0/3442702825 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68cc1e1c40 con 0x7f68cc15afa0 2024-05-27T09:27:21.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.884+0000 7f68c9600640 1 -- 172.21.15.114:0/3442702825 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c4037040 con 0x7f68cc15afa0 2024-05-27T09:27:21.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.885+0000 7f68d2a00640 1 -- 172.21.15.114:0/3442702825 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68cc1e2200 con 0x7f68cc15afa0 2024-05-27T09:27:21.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.886+0000 7f68d2a00640 1 -- 172.21.15.114:0/3442702825 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68cc150f00 con 0x7f68cc15afa0 2024-05-27T09:27:21.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.887+0000 7f68c9600640 1 -- 172.21.15.114:0/3442702825 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f68c4044020 con 0x7f68cc15afa0 2024-05-27T09:27:21.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.888+0000 7f68c9600640 1 --2- 172.21.15.114:0/3442702825 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f68b00762d0 0x7f68b00787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:21.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.888+0000 7f68c9600640 1 -- 172.21.15.114:0/3442702825 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f68c40b7840 con 0x7f68cc15afa0 2024-05-27T09:27:21.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.889+0000 7f68d1600640 1 --2- 172.21.15.114:0/3442702825 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f68b00762d0 0x7f68b00787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:21.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.889+0000 7f68d1600640 1 --2- 172.21.15.114:0/3442702825 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f68b00762d0 0x7f68b00787e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f68c0005880 tx=0x7f68c0006aa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:21.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:21.893+0000 7f68c9600640 1 -- 172.21.15.114:0/3442702825 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f68c404b020 con 0x7f68cc15afa0 2024-05-27T09:27:22.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.139+0000 7f68d2a00640 1 -- 172.21.15.114:0/3442702825 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f68cc050860 con 0x7f68cc15afa0 2024-05-27T09:27:22.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.575+0000 7f68c9600640 1 -- 172.21.15.114:0/3442702825 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7f68c4080ab0 con 0x7f68cc15afa0 2024-05-27T09:27:22.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.579+0000 7f68d2a00640 1 -- 172.21.15.114:0/3442702825 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f68b00762d0 msgr2=0x7f68b00787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:22.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.579+0000 7f68d2a00640 1 --2- 172.21.15.114:0/3442702825 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f68b00762d0 0x7f68b00787e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f68c0005880 tx=0x7f68c0006aa0 comp rx=0 tx=0).stop 2024-05-27T09:27:22.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.579+0000 7f68d2a00640 1 -- 172.21.15.114:0/3442702825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68cc15afa0 msgr2=0x7f68cc1f57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:22.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.579+0000 7f68d2a00640 1 --2- 172.21.15.114:0/3442702825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68cc15afa0 0x7f68cc1f57f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f68c402a300 tx=0x7f68c40046c0 comp rx=0 tx=0).stop 2024-05-27T09:27:22.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.579+0000 7f68d2a00640 1 -- 172.21.15.114:0/3442702825 shutdown_connections 2024-05-27T09:27:22.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.579+0000 7f68d2a00640 1 --2- 172.21.15.114:0/3442702825 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f68b00762d0 0x7f68b00787e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:22.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.579+0000 7f68d2a00640 1 --2- 172.21.15.114:0/3442702825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68cc15afa0 0x7f68cc1f57f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:22.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.579+0000 7f68d2a00640 1 --2- 172.21.15.114:0/3442702825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68cc0c9750 0x7f68cc075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:22.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.579+0000 7f68d2a00640 1 --2- 172.21.15.114:0/3442702825 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68cc071780 0x7f68cc1507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:22.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.579+0000 7f68d2a00640 1 -- 172.21.15.114:0/3442702825 >> 172.21.15.114:0/3442702825 conn(0x7f68cc0cd890 msgr2=0x7f68cc0c66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:22.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.580+0000 7f68d2a00640 1 -- 172.21.15.114:0/3442702825 shutdown_connections 2024-05-27T09:27:22.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.580+0000 7f68d2a00640 1 -- 172.21.15.114:0/3442702825 wait complete. 2024-05-27T09:27:22.608 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-05-27T09:27:22.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.886+0000 7f511ec00640 1 Processor -- start 2024-05-27T09:27:22.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.886+0000 7f511ec00640 1 -- start start 2024-05-27T09:27:22.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.888+0000 7f511ec00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f51180c8740 0x7f51181421b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:22.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.888+0000 7f511ec00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5118142810 0x7f51180c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:22.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.888+0000 7f511ec00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51180c9280 con 0x7f5118159fe0 2024-05-27T09:27:22.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.888+0000 7f511ec00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f511815a4b0 con 0x7f51180c8740 2024-05-27T09:27:22.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.888+0000 7f511ec00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f5118074610 con 0x7f5118142810 2024-05-27T09:27:22.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.888+0000 7f5117e00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f5118159fe0 0x7f5118073b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:41986/0 (socket says 172.21.15.114:41986) 2024-05-27T09:27:22.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.888+0000 7f5117e00640 1 -- 172.21.15.114:0/1415329610 learned_addr learned my addr 172.21.15.114:0/1415329610 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:22.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.888+0000 7f511cc00640 1 --2- 172.21.15.114:0/1415329610 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5118142810 0x7f51180c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:22.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.888+0000 7f5117400640 1 --2- 172.21.15.114:0/1415329610 >> v2:172.21.15.163:3300/0 conn(0x7f51180c8740 0x7f51181421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:22.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.889+0000 7f5116a00640 1 -- 172.21.15.114:0/1415329610 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4227838306 0 0) 0x7f51180c9280 con 0x7f5118159fe0 2024-05-27T09:27:22.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.889+0000 7f5116a00640 1 -- 172.21.15.114:0/1415329610 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50fc003520 con 0x7f5118159fe0 2024-05-27T09:27:22.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.889+0000 7f511cc00640 1 -- 172.21.15.114:0/1415329610 >> v2:172.21.15.163:3300/0 conn(0x7f51180c8740 msgr2=0x7f51181421b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:22.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.889+0000 7f511cc00640 1 --2- 172.21.15.114:0/1415329610 >> v2:172.21.15.163:3300/0 conn(0x7f51180c8740 0x7f51181421b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:22.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.889+0000 7f511cc00640 1 -- 172.21.15.114:0/1415329610 >> v1:172.21.15.114:6789/0 conn(0x7f5118159fe0 legacy=0x7f5118073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:22.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.889+0000 7f511cc00640 1 -- 172.21.15.114:0/1415329610 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51180748f0 con 0x7f5118142810 2024-05-27T09:27:22.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.889+0000 7f5117400640 1 --2- 172.21.15.114:0/1415329610 >> v2:172.21.15.163:3300/0 conn(0x7f51180c8740 0x7f51181421b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:22.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.890+0000 7f511cc00640 1 --2- 172.21.15.114:0/1415329610 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5118142810 0x7f51180c2a00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f511815a630 tx=0x7f511002a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fcfab8cf6d226a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:22.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.890+0000 7f5116a00640 1 -- 172.21.15.114:0/1415329610 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511002ae60 con 0x7f5118142810 2024-05-27T09:27:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.890+0000 7f5116a00640 1 -- 172.21.15.114:0/1415329610 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5110007740 con 0x7f5118142810 2024-05-27T09:27:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.890+0000 7f5116a00640 1 -- 172.21.15.114:0/1415329610 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110007b10 con 0x7f5118142810 2024-05-27T09:27:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.891+0000 7f511ec00640 1 -- 172.21.15.114:0/1415329610 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5118142810 msgr2=0x7f51180c2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.891+0000 7f511ec00640 1 --2- 172.21.15.114:0/1415329610 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5118142810 0x7f51180c2a00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f511815a630 tx=0x7f511002a640 comp rx=0 tx=0).stop 2024-05-27T09:27:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.891+0000 7f511ec00640 1 -- 172.21.15.114:0/1415329610 shutdown_connections 2024-05-27T09:27:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.891+0000 7f511ec00640 1 --2- 172.21.15.114:0/1415329610 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5118142810 0x7f51180c2a00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.891+0000 7f511ec00640 1 --2- 172.21.15.114:0/1415329610 >> v2:172.21.15.163:3300/0 conn(0x7f51180c8740 0x7f51181421b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.891+0000 7f511ec00640 1 -- 172.21.15.114:0/1415329610 >> 172.21.15.114:0/1415329610 conn(0x7f511806f6e0 msgr2=0x7f5118070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.891+0000 7f511ec00640 1 -- 172.21.15.114:0/1415329610 shutdown_connections 2024-05-27T09:27:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.891+0000 7f511ec00640 1 -- 172.21.15.114:0/1415329610 wait complete. 2024-05-27T09:27:22.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.893+0000 7f511ec00640 1 Processor -- start 2024-05-27T09:27:22.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.893+0000 7f511ec00640 1 -- start start 2024-05-27T09:27:22.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.895+0000 7f511ec00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51180c8740 0x7f51180cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.895+0000 7f511ec00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5118142810 0x7f5118152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.895+0000 7f511ec00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5118159fe0 0x7f5118152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.895+0000 7f511ec00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f51180735e0 con 0x7f5118142810 2024-05-27T09:27:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.895+0000 7f511ec00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5118073300 con 0x7f51180c8740 2024-05-27T09:27:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.895+0000 7f511ec00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f51180c4b80 con 0x7f5118159fe0 2024-05-27T09:27:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.895+0000 7f5117400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5118142810 0x7f5118152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.895+0000 7f5117e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51180c8740 0x7f51180cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.895+0000 7f5117400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5118142810 0x7f5118152570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44340/0 (socket says 172.21.15.114:44340) 2024-05-27T09:27:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.895+0000 7f511cc00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5118159fe0 0x7f5118152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.895+0000 7f5117400640 1 -- 172.21.15.114:0/950452191 learned_addr learned my addr 172.21.15.114:0/950452191 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.896+0000 7f5117400640 1 -- 172.21.15.114:0/950452191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5118159fe0 msgr2=0x7f5118152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.896+0000 7f5117400640 1 --2- 172.21.15.114:0/950452191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5118159fe0 0x7f5118152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:22.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.896+0000 7f5117400640 1 -- 172.21.15.114:0/950452191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51180c8740 msgr2=0x7f51180cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:22.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.896+0000 7f5117400640 1 --2- 172.21.15.114:0/950452191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51180c8740 0x7f51180cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:22.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.896+0000 7f5117400640 1 -- 172.21.15.114:0/950452191 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51180cbdb0 con 0x7f5118142810 2024-05-27T09:27:22.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.896+0000 7f5117400640 1 --2- 172.21.15.114:0/950452191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5118142810 0x7f5118152570 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f510400e990 tx=0x7f510400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:22.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.896+0000 7f510fe00640 1 -- 172.21.15.114:0/950452191 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104017070 con 0x7f5118142810 2024-05-27T09:27:22.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.896+0000 7f510fe00640 1 -- 172.21.15.114:0/950452191 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f510400c9e0 con 0x7f5118142810 2024-05-27T09:27:22.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.896+0000 7f511ec00640 1 -- 172.21.15.114:0/950452191 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51181e45c0 con 0x7f5118142810 2024-05-27T09:27:22.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.896+0000 7f510fe00640 1 -- 172.21.15.114:0/950452191 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510400cdb0 con 0x7f5118142810 2024-05-27T09:27:22.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.896+0000 7f511ec00640 1 -- 172.21.15.114:0/950452191 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f511813d1f0 con 0x7f5118142810 2024-05-27T09:27:22.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.897+0000 7f511ec00640 1 -- 172.21.15.114:0/950452191 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f511806a380 con 0x7f5118142810 2024-05-27T09:27:22.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.899+0000 7f510fe00640 1 -- 172.21.15.114:0/950452191 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f510401a070 con 0x7f5118142810 2024-05-27T09:27:22.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.899+0000 7f510fe00640 1 --2- 172.21.15.114:0/950452191 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f50f80762d0 0x7f50f80787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:22.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.900+0000 7f510fe00640 1 -- 172.21.15.114:0/950452191 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f51040974b0 con 0x7f5118142810 2024-05-27T09:27:22.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.900+0000 7f5117e00640 1 --2- 172.21.15.114:0/950452191 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f50f80762d0 0x7f50f80787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:22.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.901+0000 7f5117e00640 1 --2- 172.21.15.114:0/950452191 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f50f80762d0 0x7f50f80787e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5108005880 tx=0x7f5108006aa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:22.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:22.905+0000 7f510fe00640 1 -- 172.21.15.114:0/950452191 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5104060720 con 0x7f5118142810 2024-05-27T09:27:23.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.135+0000 7f511ec00640 1 -- 172.21.15.114:0/950452191 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f5118069a40 con 0x7f5118142810 2024-05-27T09:27:23.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.592+0000 7f510fe00640 1 -- 172.21.15.114:0/950452191 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v8) ==== 115+0+0 (secure 0 0 0) 0x7f5104064a20 con 0x7f5118142810 2024-05-27T09:27:23.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.596+0000 7f511ec00640 1 -- 172.21.15.114:0/950452191 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f50f80762d0 msgr2=0x7f50f80787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:23.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.596+0000 7f511ec00640 1 --2- 172.21.15.114:0/950452191 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f50f80762d0 0x7f50f80787e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5108005880 tx=0x7f5108006aa0 comp rx=0 tx=0).stop 2024-05-27T09:27:23.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.596+0000 7f511ec00640 1 -- 172.21.15.114:0/950452191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5118142810 msgr2=0x7f5118152570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:23.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.596+0000 7f511ec00640 1 --2- 172.21.15.114:0/950452191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5118142810 0x7f5118152570 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f510400e990 tx=0x7f510400ee50 comp rx=0 tx=0).stop 2024-05-27T09:27:23.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.596+0000 7f511ec00640 1 -- 172.21.15.114:0/950452191 shutdown_connections 2024-05-27T09:27:23.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.596+0000 7f511ec00640 1 --2- 172.21.15.114:0/950452191 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f50f80762d0 0x7f50f80787e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:23.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.596+0000 7f511ec00640 1 --2- 172.21.15.114:0/950452191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5118159fe0 0x7f5118152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:23.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.596+0000 7f511ec00640 1 --2- 172.21.15.114:0/950452191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5118142810 0x7f5118152570 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:23.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.596+0000 7f511ec00640 1 --2- 172.21.15.114:0/950452191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51180c8740 0x7f51180cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:23.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.596+0000 7f511ec00640 1 -- 172.21.15.114:0/950452191 >> 172.21.15.114:0/950452191 conn(0x7f511806f6e0 msgr2=0x7f51180c2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:23.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.596+0000 7f511ec00640 1 -- 172.21.15.114:0/950452191 shutdown_connections 2024-05-27T09:27:23.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.596+0000 7f511ec00640 1 -- 172.21.15.114:0/950452191 wait complete. 2024-05-27T09:27:23.631 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-27T09:27:23.631 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:27:23.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.886+0000 7f1918800640 1 Processor -- start 2024-05-27T09:27:23.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.886+0000 7f1918800640 1 -- start start 2024-05-27T09:27:23.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.888+0000 7f1918800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1910164db0 0x7f19100c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:23.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.888+0000 7f1918800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1910160b40 0x7f1910160f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:23.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.888+0000 7f1918800640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19101615d0 con 0x7f1910074860 2024-05-27T09:27:23.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.888+0000 7f1918800640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f19100cade0 con 0x7f1910164db0 2024-05-27T09:27:23.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.889+0000 7f1918800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f19100c3d60 con 0x7f1910160b40 2024-05-27T09:27:23.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.889+0000 7f1915400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f1910074860 0x7f191015aac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:41990/0 (socket says 172.21.15.114:41990) 2024-05-27T09:27:23.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.889+0000 7f1915400640 1 -- 172.21.15.114:0/552291226 learned_addr learned my addr 172.21.15.114:0/552291226 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:23.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.889+0000 7f1915e00640 1 --2- 172.21.15.114:0/552291226 >> v2:172.21.15.114:3300/0 conn(0x7f1910164db0 0x7f19100c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:23.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.889+0000 7f1916800640 1 --2- 172.21.15.114:0/552291226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1910160b40 0x7f1910160f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:23.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.889+0000 7f1914a00640 1 -- 172.21.15.114:0/552291226 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3370576159 0 0) 0x7f19101615d0 con 0x7f1910074860 2024-05-27T09:27:23.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.889+0000 7f1914a00640 1 -- 172.21.15.114:0/552291226 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18fc003520 con 0x7f1910074860 2024-05-27T09:27:23.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.890+0000 7f1915e00640 1 -- 172.21.15.114:0/552291226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1910160b40 msgr2=0x7f1910160f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:23.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.890+0000 7f1915e00640 1 --2- 172.21.15.114:0/552291226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1910160b40 0x7f1910160f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:23.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.890+0000 7f1915e00640 1 -- 172.21.15.114:0/552291226 >> v1:172.21.15.114:6789/0 conn(0x7f1910074860 legacy=0x7f191015aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:23.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.890+0000 7f1915e00640 1 -- 172.21.15.114:0/552291226 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1910161890 con 0x7f1910164db0 2024-05-27T09:27:23.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.891+0000 7f1915e00640 1 --2- 172.21.15.114:0/552291226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1910164db0 0x7f19100c3820 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1908009d90 tx=0x7f190802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78eaf6ebb047e87f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:23.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.891+0000 7f1914a00640 1 -- 172.21.15.114:0/552291226 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908007740 con 0x7f1910164db0 2024-05-27T09:27:23.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.891+0000 7f1914a00640 1 -- 172.21.15.114:0/552291226 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19080078e0 con 0x7f1910164db0 2024-05-27T09:27:23.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.891+0000 7f1914a00640 1 -- 172.21.15.114:0/552291226 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908007cb0 con 0x7f1910164db0 2024-05-27T09:27:23.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.892+0000 7f1918800640 1 -- 172.21.15.114:0/552291226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1910164db0 msgr2=0x7f19100c3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:23.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.892+0000 7f1918800640 1 --2- 172.21.15.114:0/552291226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1910164db0 0x7f19100c3820 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1908009d90 tx=0x7f190802a6f0 comp rx=0 tx=0).stop 2024-05-27T09:27:23.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.892+0000 7f1918800640 1 -- 172.21.15.114:0/552291226 shutdown_connections 2024-05-27T09:27:23.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.892+0000 7f1918800640 1 --2- 172.21.15.114:0/552291226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1910160b40 0x7f1910160f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:23.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.892+0000 7f1918800640 1 --2- 172.21.15.114:0/552291226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1910164db0 0x7f19100c3820 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:23.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.892+0000 7f1918800640 1 -- 172.21.15.114:0/552291226 >> 172.21.15.114:0/552291226 conn(0x7f191006f6e0 msgr2=0x7f1910070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:23.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.892+0000 7f1918800640 1 -- 172.21.15.114:0/552291226 shutdown_connections 2024-05-27T09:27:23.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.892+0000 7f1918800640 1 -- 172.21.15.114:0/552291226 wait complete. 2024-05-27T09:27:23.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.894+0000 7f1918800640 1 Processor -- start 2024-05-27T09:27:23.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.894+0000 7f1918800640 1 -- start start 2024-05-27T09:27:23.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.896+0000 7f1918800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1910074860 0x7f19100c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:23.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.896+0000 7f1918800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1910160b40 0x7f19101f6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:23.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.896+0000 7f1918800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1910164db0 0x7f19101555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:23.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.896+0000 7f1918800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1910164890 con 0x7f1910074860 2024-05-27T09:27:23.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.896+0000 7f1918800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f19101645b0 con 0x7f1910164db0 2024-05-27T09:27:23.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.896+0000 7f1918800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f1910073310 con 0x7f1910160b40 2024-05-27T09:27:23.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.896+0000 7f1915e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1910074860 0x7f19100c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:23.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.896+0000 7f1915e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1910074860 0x7f19100c5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44354/0 (socket says 172.21.15.114:44354) 2024-05-27T09:27:23.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.896+0000 7f1915e00640 1 -- 172.21.15.114:0/2027977742 learned_addr learned my addr 172.21.15.114:0/2027977742 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:23.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.896+0000 7f1915400640 1 --2- 172.21.15.114:0/2027977742 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1910160b40 0x7f19101f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:23.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.896+0000 7f1916800640 1 --2- 172.21.15.114:0/2027977742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1910164db0 0x7f19101555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:23.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.897+0000 7f1915e00640 1 -- 172.21.15.114:0/2027977742 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1910160b40 msgr2=0x7f19101f6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:23.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.897+0000 7f1915e00640 1 --2- 172.21.15.114:0/2027977742 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1910160b40 0x7f19101f6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:23.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.897+0000 7f1915e00640 1 -- 172.21.15.114:0/2027977742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1910164db0 msgr2=0x7f19101555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:23.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.897+0000 7f1915e00640 1 --2- 172.21.15.114:0/2027977742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1910164db0 0x7f19101555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:23.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.897+0000 7f1915e00640 1 -- 172.21.15.114:0/2027977742 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19101f6c70 con 0x7f1910074860 2024-05-27T09:27:23.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.897+0000 7f1915e00640 1 --2- 172.21.15.114:0/2027977742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1910074860 0x7f19100c5a40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1908009d90 tx=0x7f1908004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:23.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.897+0000 7f1906000640 1 -- 172.21.15.114:0/2027977742 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190803f070 con 0x7f1910074860 2024-05-27T09:27:23.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.897+0000 7f1906000640 1 -- 172.21.15.114:0/2027977742 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1908031c40 con 0x7f1910074860 2024-05-27T09:27:23.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.897+0000 7f1918800640 1 -- 172.21.15.114:0/2027977742 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19100cb8c0 con 0x7f1910074860 2024-05-27T09:27:23.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.897+0000 7f1906000640 1 -- 172.21.15.114:0/2027977742 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908009450 con 0x7f1910074860 2024-05-27T09:27:23.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.897+0000 7f1918800640 1 -- 172.21.15.114:0/2027977742 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19100cbe80 con 0x7f1910074860 2024-05-27T09:27:23.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.899+0000 7f1918800640 1 -- 172.21.15.114:0/2027977742 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f191006a380 con 0x7f1910074860 2024-05-27T09:27:23.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.900+0000 7f1906000640 1 -- 172.21.15.114:0/2027977742 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f190803b410 con 0x7f1910074860 2024-05-27T09:27:23.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.901+0000 7f1906000640 1 --2- 172.21.15.114:0/2027977742 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f18f4076300 0x7f18f4078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:23.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.901+0000 7f1906000640 1 -- 172.21.15.114:0/2027977742 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f19080b8cb0 con 0x7f1910074860 2024-05-27T09:27:23.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.902+0000 7f1915400640 1 --2- 172.21.15.114:0/2027977742 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f18f4076300 0x7f18f4078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:23.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.902+0000 7f1915400640 1 --2- 172.21.15.114:0/2027977742 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f18f4076300 0x7f18f4078810 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f191015aef0 tx=0x7f1900006aa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:23.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:23.906+0000 7f1906000640 1 -- 172.21.15.114:0/2027977742 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1908081f20 con 0x7f1910074860 2024-05-27T09:27:24.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.154+0000 7f1918800640 1 -- 172.21.15.114:0/2027977742 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1910073e30 con 0x7f1910074860 2024-05-27T09:27:24.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.155+0000 7f1906000640 1 -- 172.21.15.114:0/2027977742 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+6852 (secure 0 0 0) 0x7f1908086220 con 0x7f1910074860 2024-05-27T09:27:24.157 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:24.157 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/444766663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":444766663},{"type":"v1","addr":"172.21.15.163:6839","nonce":444766663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:27:23.601458+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":4300,"mds_1":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:creating","state_seq":1,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-05-27T09:27:24.158 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 9 2024-05-27T09:27:24.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.158+0000 7f1918800640 1 -- 172.21.15.114:0/2027977742 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f18f4076300 msgr2=0x7f18f4078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:24.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.158+0000 7f1918800640 1 --2- 172.21.15.114:0/2027977742 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f18f4076300 0x7f18f4078810 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f191015aef0 tx=0x7f1900006aa0 comp rx=0 tx=0).stop 2024-05-27T09:27:24.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.158+0000 7f1918800640 1 -- 172.21.15.114:0/2027977742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1910074860 msgr2=0x7f19100c5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:24.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.158+0000 7f1918800640 1 --2- 172.21.15.114:0/2027977742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1910074860 0x7f19100c5a40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1908009d90 tx=0x7f1908004a40 comp rx=0 tx=0).stop 2024-05-27T09:27:24.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.159+0000 7f1918800640 1 -- 172.21.15.114:0/2027977742 shutdown_connections 2024-05-27T09:27:24.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.159+0000 7f1918800640 1 --2- 172.21.15.114:0/2027977742 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f18f4076300 0x7f18f4078810 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:24.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.159+0000 7f1918800640 1 --2- 172.21.15.114:0/2027977742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1910164db0 0x7f19101555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:24.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.159+0000 7f1918800640 1 --2- 172.21.15.114:0/2027977742 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1910160b40 0x7f19101f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:24.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.159+0000 7f1918800640 1 --2- 172.21.15.114:0/2027977742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1910074860 0x7f19100c5a40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:24.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.159+0000 7f1918800640 1 -- 172.21.15.114:0/2027977742 >> 172.21.15.114:0/2027977742 conn(0x7f191006f6e0 msgr2=0x7f19100736f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:24.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.159+0000 7f1918800640 1 -- 172.21.15.114:0/2027977742 shutdown_connections 2024-05-27T09:27:24.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.159+0000 7f1918800640 1 -- 172.21.15.114:0/2027977742 wait complete. 2024-05-27T09:27:24.189 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T09:27:24.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.464+0000 7f1a9da00640 1 Processor -- start 2024-05-27T09:27:24.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.464+0000 7f1a9da00640 1 -- start start 2024-05-27T09:27:24.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.466+0000 7f1a9da00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1a981593b0 0x7f1a98142720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:24.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.466+0000 7f1a9da00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1a980c2990 0x7f1a980c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:24.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.466+0000 7f1a9da00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a980739a0 con 0x7f1a98073260 2024-05-27T09:27:24.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.466+0000 7f1a9da00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f1a98074d20 con 0x7f1a981593b0 2024-05-27T09:27:24.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.466+0000 7f1a9da00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f1a98073c80 con 0x7f1a980c2990 2024-05-27T09:27:24.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.466+0000 7f1a96a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1a981593b0 0x7f1a98142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:24.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.467+0000 7f1a96a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1a981593b0 0x7f1a98142720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44358/0 (socket says 172.21.15.114:44358) 2024-05-27T09:27:24.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.467+0000 7f1a96a00640 1 -- 172.21.15.114:0/680321334 learned_addr learned my addr 172.21.15.114:0/680321334 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:24.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.467+0000 7f1a97400640 1 --2- 172.21.15.114:0/680321334 >> v2:172.21.15.163:3300/0 conn(0x7f1a980c2990 0x7f1a980c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:24.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.467+0000 7f1a96a00640 1 -- 172.21.15.114:0/680321334 >> v2:172.21.15.163:3300/0 conn(0x7f1a980c2990 msgr2=0x7f1a980c2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:24.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.467+0000 7f1a96a00640 1 --2- 172.21.15.114:0/680321334 >> v2:172.21.15.163:3300/0 conn(0x7f1a980c2990 0x7f1a980c2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:24.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.467+0000 7f1a96a00640 1 -- 172.21.15.114:0/680321334 >> v1:172.21.15.163:6789/0 conn(0x7f1a98073260 legacy=0x7f1a9815a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:24.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.467+0000 7f1a96a00640 1 -- 172.21.15.114:0/680321334 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a98073f60 con 0x7f1a981593b0 2024-05-27T09:27:24.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.467+0000 7f1a97400640 1 --2- 172.21.15.114:0/680321334 >> v2:172.21.15.163:3300/0 conn(0x7f1a980c2990 0x7f1a980c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:24.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.468+0000 7f1a96a00640 1 --2- 172.21.15.114:0/680321334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a981593b0 0x7f1a98142720 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1a980731f0 tx=0x7f1a8802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0cc8b6ba6a1790 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:24.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.469+0000 7f1a96000640 1 -- 172.21.15.114:0/680321334 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88037070 con 0x7f1a981593b0 2024-05-27T09:27:24.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.469+0000 7f1a96000640 1 -- 172.21.15.114:0/680321334 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8802ae50 con 0x7f1a981593b0 2024-05-27T09:27:24.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.469+0000 7f1a96000640 1 -- 172.21.15.114:0/680321334 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88032860 con 0x7f1a981593b0 2024-05-27T09:27:24.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.469+0000 7f1a9da00640 1 -- 172.21.15.114:0/680321334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a981593b0 msgr2=0x7f1a98142720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:24.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.469+0000 7f1a9da00640 1 --2- 172.21.15.114:0/680321334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a981593b0 0x7f1a98142720 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1a980731f0 tx=0x7f1a8802a300 comp rx=0 tx=0).stop 2024-05-27T09:27:24.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.469+0000 7f1a9da00640 1 -- 172.21.15.114:0/680321334 shutdown_connections 2024-05-27T09:27:24.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.469+0000 7f1a9da00640 1 --2- 172.21.15.114:0/680321334 >> v2:172.21.15.163:3300/0 conn(0x7f1a980c2990 0x7f1a980c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:24.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.469+0000 7f1a9da00640 1 --2- 172.21.15.114:0/680321334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a981593b0 0x7f1a98142720 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:24.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.469+0000 7f1a9da00640 1 -- 172.21.15.114:0/680321334 >> 172.21.15.114:0/680321334 conn(0x7f1a9806f6e0 msgr2=0x7f1a98070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:24.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.469+0000 7f1a9da00640 1 -- 172.21.15.114:0/680321334 shutdown_connections 2024-05-27T09:27:24.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.470+0000 7f1a9da00640 1 -- 172.21.15.114:0/680321334 wait complete. 2024-05-27T09:27:24.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.472+0000 7f1a9da00640 1 Processor -- start 2024-05-27T09:27:24.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.472+0000 7f1a9da00640 1 -- start start 2024-05-27T09:27:24.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.473+0000 7f1a9da00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a98073260 0x7f1a981f4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:24.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.473+0000 7f1a9da00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a980c2990 0x7f1a9806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:24.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.473+0000 7f1a9da00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1a981593b0 0x7f1a981418b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:24.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.473+0000 7f1a9da00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1a98074800 con 0x7f1a98073260 2024-05-27T09:27:24.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.473+0000 7f1a9da00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1a98074520 con 0x7f1a980c2990 2024-05-27T09:27:24.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.473+0000 7f1a9da00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f1a98073670 con 0x7f1a981593b0 2024-05-27T09:27:24.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.474+0000 7f1a96a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a98073260 0x7f1a981f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:24.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.474+0000 7f1a96a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a98073260 0x7f1a981f4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44366/0 (socket says 172.21.15.114:44366) 2024-05-27T09:27:24.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.474+0000 7f1a96a00640 1 -- 172.21.15.114:0/514521807 learned_addr learned my addr 172.21.15.114:0/514521807 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:24.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.474+0000 7f1a97400640 1 --2- 172.21.15.114:0/514521807 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1a981593b0 0x7f1a981418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:24.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.474+0000 7f1a8e000640 1 --2- 172.21.15.114:0/514521807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a980c2990 0x7f1a9806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:24.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.474+0000 7f1a96a00640 1 -- 172.21.15.114:0/514521807 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1a981593b0 msgr2=0x7f1a981418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:24.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.474+0000 7f1a96a00640 1 --2- 172.21.15.114:0/514521807 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1a981593b0 0x7f1a981418b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:24.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.474+0000 7f1a96a00640 1 -- 172.21.15.114:0/514521807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a980c2990 msgr2=0x7f1a9806c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:24.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.474+0000 7f1a96a00640 1 --2- 172.21.15.114:0/514521807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a980c2990 0x7f1a9806c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:24.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.474+0000 7f1a96a00640 1 -- 172.21.15.114:0/514521807 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a981f09d0 con 0x7f1a98073260 2024-05-27T09:27:24.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.474+0000 7f1a97400640 1 --2- 172.21.15.114:0/514521807 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1a981593b0 0x7f1a981418b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:27:24.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.474+0000 7f1a96a00640 1 --2- 172.21.15.114:0/514521807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a98073260 0x7f1a981f4950 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1a88000c00 tx=0x7f1a880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:24.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.475+0000 7f1a8f400640 1 -- 172.21.15.114:0/514521807 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88041020 con 0x7f1a98073260 2024-05-27T09:27:24.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.475+0000 7f1a8f400640 1 -- 172.21.15.114:0/514521807 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8803c890 con 0x7f1a98073260 2024-05-27T09:27:24.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.475+0000 7f1a8f400640 1 -- 172.21.15.114:0/514521807 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88037040 con 0x7f1a98073260 2024-05-27T09:27:24.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.475+0000 7f1a9da00640 1 -- 172.21.15.114:0/514521807 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a981e7810 con 0x7f1a98073260 2024-05-27T09:27:24.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.475+0000 7f1a9da00640 1 -- 172.21.15.114:0/514521807 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a981f0f30 con 0x7f1a98073260 2024-05-27T09:27:24.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.476+0000 7f1a9da00640 1 -- 172.21.15.114:0/514521807 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a980767d0 con 0x7f1a98073260 2024-05-27T09:27:24.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.477+0000 7f1a8f400640 1 -- 172.21.15.114:0/514521807 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1a88044020 con 0x7f1a98073260 2024-05-27T09:27:24.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.478+0000 7f1a8f400640 1 --2- 172.21.15.114:0/514521807 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1a74076300 0x7f1a74078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:24.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.478+0000 7f1a8f400640 1 -- 172.21.15.114:0/514521807 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1a880b7f80 con 0x7f1a98073260 2024-05-27T09:27:24.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.479+0000 7f1a8e000640 1 --2- 172.21.15.114:0/514521807 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1a74076300 0x7f1a74078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:24.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.479+0000 7f1a8e000640 1 --2- 172.21.15.114:0/514521807 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1a74076300 0x7f1a74078810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1a981eb310 tx=0x7f1a80009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:24.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.483+0000 7f1a8f400640 1 -- 172.21.15.114:0/514521807 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1a980767d0 con 0x7f1a98073260 2024-05-27T09:27:24.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.688+0000 7f1a9da00640 1 -- 172.21.15.114:0/514521807 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a9806a380 con 0x7f1a98073260 2024-05-27T09:27:24.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.691+0000 7f1a8f400640 1 -- 172.21.15.114:0/514521807 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16645 (secure 0 0 0) 0x7f1a8804b030 con 0x7f1a98073260 2024-05-27T09:27:24.693 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:24.693 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"fsid":"2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48","created":"2024-05-27T09:27:00.307278+0000","modified":"2024-05-27T09:27:19.530054+0000","last_up_change":"2024-05-27T09:27:11.463226+0000","last_in_change":"2024-05-27T09:27:02.972880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T09:27:10.796859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T09:27:16.063199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T09:27:17.064857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3561c750-1d63-42f6-a6ae-0ee63721e15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6817","nonce":1960550710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6819","nonce":1960550710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6824","nonce":1960550710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6821","nonce":1960550710}]},"public_addr":"172.21.15.114:6817/1960550710","cluster_addr":"172.21.15.114:6819/1960550710","heartbeat_back_addr":"172.21.15.114:6824/1960550710","heartbeat_front_addr":"172.21.15.114:6821/1960550710","state":["exists","up"]},{"osd":1,"uuid":"9a407d04-38c0-4676-928f-d04e8116bd19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6825","nonce":326580057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6827","nonce":326580057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6831","nonce":326580057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6829","nonce":326580057}]},"public_addr":"172.21.15.114:6825/326580057","cluster_addr":"172.21.15.114:6827/326580057","heartbeat_back_addr":"172.21.15.114:6831/326580057","heartbeat_front_addr":"172.21.15.114:6829/326580057","state":["exists","up"]},{"osd":2,"uuid":"255a1cc6-9357-43f7-9db5-d760f4806ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6809","nonce":865568619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6811","nonce":865568619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6815","nonce":865568619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6813","nonce":865568619}]},"public_addr":"172.21.15.114:6809/865568619","cluster_addr":"172.21.15.114:6811/865568619","heartbeat_back_addr":"172.21.15.114:6815/865568619","heartbeat_front_addr":"172.21.15.114:6813/865568619","state":["exists","up"]},{"osd":3,"uuid":"cb4d7692-9a17-41b9-927e-d8eac3e217c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6801","nonce":1601792164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6803","nonce":1601792164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6807","nonce":1601792164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6805","nonce":1601792164}]},"public_addr":"172.21.15.114:6801/1601792164","cluster_addr":"172.21.15.114:6803/1601792164","heartbeat_back_addr":"172.21.15.114:6807/1601792164","heartbeat_front_addr":"172.21.15.114:6805/1601792164","state":["exists","up"]},{"osd":4,"uuid":"cdef1940-3453-4fbd-b0f1-459f5f5c7142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6817","nonce":1982246015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6819","nonce":1982246015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6823","nonce":1982246015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6821","nonce":1982246015}]},"public_addr":"172.21.15.163:6817/1982246015","cluster_addr":"172.21.15.163:6819/1982246015","heartbeat_back_addr":"172.21.15.163:6823/1982246015","heartbeat_front_addr":"172.21.15.163:6821/1982246015","state":["exists","up"]},{"osd":5,"uuid":"220a7b32-abcc-492f-bca8-4615cc2ea111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6801","nonce":1695294004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6803","nonce":1695294004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6807","nonce":1695294004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6805","nonce":1695294004}]},"public_addr":"172.21.15.163:6801/1695294004","cluster_addr":"172.21.15.163:6803/1695294004","heartbeat_back_addr":"172.21.15.163:6807/1695294004","heartbeat_front_addr":"172.21.15.163:6805/1695294004","state":["exists","up"]},{"osd":6,"uuid":"43a74d13-d03f-493a-a6c0-e73b0a056c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6809","nonce":3649718505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6811","nonce":3649718505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6815","nonce":3649718505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6813","nonce":3649718505}]},"public_addr":"172.21.15.163:6809/3649718505","cluster_addr":"172.21.15.163:6811/3649718505","heartbeat_back_addr":"172.21.15.163:6815/3649718505","heartbeat_front_addr":"172.21.15.163:6813/3649718505","state":["exists","up"]},{"osd":7,"uuid":"134629c2-70af-4b84-83a6-7ddf53681baa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6825","nonce":2539451311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6827","nonce":2539451311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6831","nonce":2539451311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6829","nonce":2539451311}]},"public_addr":"172.21.15.163:6825/2539451311","cluster_addr":"172.21.15.163:6827/2539451311","heartbeat_back_addr":"172.21.15.163:6831/2539451311","heartbeat_front_addr":"172.21.15.163:6829/2539451311","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.388942+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.515675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.646761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.400212+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.220072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.079531+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.180419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.175008+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T09:27:24.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.694+0000 7f1a9da00640 1 -- 172.21.15.114:0/514521807 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1a74076300 msgr2=0x7f1a74078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:24.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.694+0000 7f1a9da00640 1 --2- 172.21.15.114:0/514521807 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1a74076300 0x7f1a74078810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1a981eb310 tx=0x7f1a80009040 comp rx=0 tx=0).stop 2024-05-27T09:27:24.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.694+0000 7f1a9da00640 1 -- 172.21.15.114:0/514521807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a98073260 msgr2=0x7f1a981f4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:24.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.694+0000 7f1a9da00640 1 --2- 172.21.15.114:0/514521807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a98073260 0x7f1a981f4950 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1a88000c00 tx=0x7f1a880046c0 comp rx=0 tx=0).stop 2024-05-27T09:27:24.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.694+0000 7f1a9da00640 1 -- 172.21.15.114:0/514521807 shutdown_connections 2024-05-27T09:27:24.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.694+0000 7f1a9da00640 1 --2- 172.21.15.114:0/514521807 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1a74076300 0x7f1a74078810 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:24.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.694+0000 7f1a9da00640 1 --2- 172.21.15.114:0/514521807 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1a981593b0 0x7f1a981418b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:24.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.694+0000 7f1a9da00640 1 --2- 172.21.15.114:0/514521807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a980c2990 0x7f1a9806c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:24.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.694+0000 7f1a9da00640 1 --2- 172.21.15.114:0/514521807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a98073260 0x7f1a981f4950 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:24.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.694+0000 7f1a9da00640 1 -- 172.21.15.114:0/514521807 >> 172.21.15.114:0/514521807 conn(0x7f1a9806f6e0 msgr2=0x7f1a980cb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:24.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.695+0000 7f1a9da00640 1 -- 172.21.15.114:0/514521807 shutdown_connections 2024-05-27T09:27:24.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.695+0000 7f1a9da00640 1 -- 172.21.15.114:0/514521807 wait complete. 2024-05-27T09:27:24.722 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-27T09:27:24.723 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T09:27:24.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.992+0000 7fdae3200640 1 Processor -- start 2024-05-27T09:27:24.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.992+0000 7fdae3200640 1 -- start start 2024-05-27T09:27:24.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.994+0000 7fdae3200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdadc1593b0 0x7fdadc142720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:24.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.994+0000 7fdae3200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdadc0c2990 0x7fdadc0c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:24.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.994+0000 7fdae3200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdadc0739a0 con 0x7fdadc073260 2024-05-27T09:27:24.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.994+0000 7fdae3200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fdadc074d20 con 0x7fdadc1593b0 2024-05-27T09:27:24.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.994+0000 7fdae3200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fdadc073c80 con 0x7fdadc0c2990 2024-05-27T09:27:24.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.994+0000 7fdadb400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fdadc073260 0x7fdadc15a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:46526/0 (socket says 172.21.15.114:46526) 2024-05-27T09:27:24.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.994+0000 7fdadb400640 1 -- 172.21.15.114:0/1273024956 learned_addr learned my addr 172.21.15.114:0/1273024956 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:24.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.994+0000 7fdadbe00640 1 --2- 172.21.15.114:0/1273024956 >> v2:172.21.15.163:3300/0 conn(0x7fdadc1593b0 0x7fdadc142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:24.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.994+0000 7fdae1200640 1 --2- 172.21.15.114:0/1273024956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdadc0c2990 0x7fdadc0c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:24.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.995+0000 7fdadaa00640 1 -- 172.21.15.114:0/1273024956 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2665042413 0 0) 0x7fdadc0739a0 con 0x7fdadc073260 2024-05-27T09:27:24.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.995+0000 7fdadaa00640 1 -- 172.21.15.114:0/1273024956 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdac0003520 con 0x7fdadc073260 2024-05-27T09:27:24.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.995+0000 7fdadbe00640 1 -- 172.21.15.114:0/1273024956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdadc0c2990 msgr2=0x7fdadc0c2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:24.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.995+0000 7fdadbe00640 1 --2- 172.21.15.114:0/1273024956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdadc0c2990 0x7fdadc0c2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:24.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.995+0000 7fdadbe00640 1 -- 172.21.15.114:0/1273024956 >> v1:172.21.15.114:6789/0 conn(0x7fdadc073260 legacy=0x7fdadc15a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:24.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.995+0000 7fdadbe00640 1 -- 172.21.15.114:0/1273024956 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdadc073f60 con 0x7fdadc1593b0 2024-05-27T09:27:24.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.996+0000 7fdadbe00640 1 --2- 172.21.15.114:0/1273024956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdadc1593b0 0x7fdadc142720 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009d90 tx=0x7fdad402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af776f58d6837be7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:24.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.996+0000 7fdadaa00640 1 -- 172.21.15.114:0/1273024956 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4007740 con 0x7fdadc1593b0 2024-05-27T09:27:24.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.996+0000 7fdadaa00640 1 -- 172.21.15.114:0/1273024956 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdad40078e0 con 0x7fdadc1593b0 2024-05-27T09:27:24.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.996+0000 7fdadaa00640 1 -- 172.21.15.114:0/1273024956 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4007c90 con 0x7fdadc1593b0 2024-05-27T09:27:24.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.997+0000 7fdae3200640 1 -- 172.21.15.114:0/1273024956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdadc1593b0 msgr2=0x7fdadc142720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:24.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.997+0000 7fdae3200640 1 --2- 172.21.15.114:0/1273024956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdadc1593b0 0x7fdadc142720 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009d90 tx=0x7fdad402a6f0 comp rx=0 tx=0).stop 2024-05-27T09:27:24.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.997+0000 7fdae3200640 1 -- 172.21.15.114:0/1273024956 shutdown_connections 2024-05-27T09:27:24.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.997+0000 7fdae3200640 1 --2- 172.21.15.114:0/1273024956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdadc0c2990 0x7fdadc0c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:24.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.997+0000 7fdae3200640 1 --2- 172.21.15.114:0/1273024956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdadc1593b0 0x7fdadc142720 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:24.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.997+0000 7fdae3200640 1 -- 172.21.15.114:0/1273024956 >> 172.21.15.114:0/1273024956 conn(0x7fdadc06f6e0 msgr2=0x7fdadc070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:24.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.997+0000 7fdae3200640 1 -- 172.21.15.114:0/1273024956 shutdown_connections 2024-05-27T09:27:24.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:24.997+0000 7fdae3200640 1 -- 172.21.15.114:0/1273024956 wait complete. 2024-05-27T09:27:25.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.000+0000 7fdae3200640 1 Processor -- start 2024-05-27T09:27:25.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.000+0000 7fdae3200640 1 -- start start 2024-05-27T09:27:25.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.001+0000 7fdae3200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdadc073260 0x7fdadc15c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:25.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.001+0000 7fdae3200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdadc0c2990 0x7fdadc1f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:25.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.001+0000 7fdae3200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdadc1593b0 0x7fdadc0c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:25.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.001+0000 7fdae3200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc074800 con 0x7fdadc1593b0 2024-05-27T09:27:25.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.001+0000 7fdae3200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc074520 con 0x7fdadc073260 2024-05-27T09:27:25.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.001+0000 7fdae3200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fdadc073670 con 0x7fdadc0c2990 2024-05-27T09:27:25.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.001+0000 7fdae1200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdadc1593b0 0x7fdadc0c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:25.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.001+0000 7fdadb400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdadc0c2990 0x7fdadc1f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:25.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.001+0000 7fdadbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdadc073260 0x7fdadc15c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:25.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.001+0000 7fdae1200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdadc1593b0 0x7fdadc0c7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35376/0 (socket says 172.21.15.114:35376) 2024-05-27T09:27:25.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.001+0000 7fdae1200640 1 -- 172.21.15.114:0/1040766740 learned_addr learned my addr 172.21.15.114:0/1040766740 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:25.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.001+0000 7fdadb400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdadc0c2990 0x7fdadc1f4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:60356/0 (socket says 172.21.15.114:60356) 2024-05-27T09:27:25.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.002+0000 7fdadb400640 1 -- 172.21.15.114:0/1040766740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdadc073260 msgr2=0x7fdadc15c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:25.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.002+0000 7fdadb400640 1 --2- 172.21.15.114:0/1040766740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdadc073260 0x7fdadc15c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:25.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.002+0000 7fdadb400640 1 -- 172.21.15.114:0/1040766740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdadc1593b0 msgr2=0x7fdadc0c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:25.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.002+0000 7fdadb400640 1 --2- 172.21.15.114:0/1040766740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdadc1593b0 0x7fdadc0c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:25.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.002+0000 7fdadb400640 1 -- 172.21.15.114:0/1040766740 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdadc1f4d50 con 0x7fdadc0c2990 2024-05-27T09:27:25.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.002+0000 7fdae1200640 1 --2- 172.21.15.114:0/1040766740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdadc1593b0 0x7fdadc0c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:25.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.002+0000 7fdadb400640 1 --2- 172.21.15.114:0/1040766740 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdadc0c2990 0x7fdadc1f4810 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdac400c390 tx=0x7fdac400c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:25.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.002+0000 7fdad3e00640 1 -- 172.21.15.114:0/1040766740 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4017070 con 0x7fdadc0c2990 2024-05-27T09:27:25.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.002+0000 7fdad3e00640 1 -- 172.21.15.114:0/1040766740 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac4005ce0 con 0x7fdadc0c2990 2024-05-27T09:27:25.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.002+0000 7fdad3e00640 1 -- 172.21.15.114:0/1040766740 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac40072d0 con 0x7fdadc0c2990 2024-05-27T09:27:25.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.003+0000 7fdae3200640 1 -- 172.21.15.114:0/1040766740 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdadc15d730 con 0x7fdadc0c2990 2024-05-27T09:27:25.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.003+0000 7fdae3200640 1 -- 172.21.15.114:0/1040766740 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdadc1f3d30 con 0x7fdadc0c2990 2024-05-27T09:27:25.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.004+0000 7fdae3200640 1 -- 172.21.15.114:0/1040766740 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdadc141c80 con 0x7fdadc0c2990 2024-05-27T09:27:25.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.005+0000 7fdad3e00640 1 -- 172.21.15.114:0/1040766740 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdac401a070 con 0x7fdadc0c2990 2024-05-27T09:27:25.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.006+0000 7fdad3e00640 1 --2- 172.21.15.114:0/1040766740 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdab80762d0 0x7fdab80787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:25.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.006+0000 7fdad3e00640 1 -- 172.21.15.114:0/1040766740 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdac4096d80 con 0x7fdadc0c2990 2024-05-27T09:27:25.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.006+0000 7fdadbe00640 1 --2- 172.21.15.114:0/1040766740 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdab80762d0 0x7fdab80787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:25.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.007+0000 7fdadbe00640 1 --2- 172.21.15.114:0/1040766740 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdab80762d0 0x7fdab80787e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdad4034000 tx=0x7fdad402f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:25.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.010+0000 7fdad3e00640 1 -- 172.21.15.114:0/1040766740 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdadc141c80 con 0x7fdadc0c2990 2024-05-27T09:27:25.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.215+0000 7fdae3200640 1 -- 172.21.15.114:0/1040766740 --> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdadc06a380 con 0x7fdab80762d0 2024-05-27T09:27:25.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.223+0000 7fdad3e00640 1 -- 172.21.15.114:0/1040766740 <== mgr.4108 v2:172.21.15.163:6832/1122018204 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251915 (secure 0 0 0) 0x7fdadc06a380 con 0x7fdab80762d0 2024-05-27T09:27:25.228 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:25.230 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-05-27T09:27:25.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.230+0000 7fdae3200640 1 -- 172.21.15.114:0/1040766740 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdab80762d0 msgr2=0x7fdab80787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:25.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.230+0000 7fdae3200640 1 --2- 172.21.15.114:0/1040766740 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdab80762d0 0x7fdab80787e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdad4034000 tx=0x7fdad402f000 comp rx=0 tx=0).stop 2024-05-27T09:27:25.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.230+0000 7fdae3200640 1 -- 172.21.15.114:0/1040766740 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdadc0c2990 msgr2=0x7fdadc1f4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:25.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.230+0000 7fdae3200640 1 --2- 172.21.15.114:0/1040766740 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdadc0c2990 0x7fdadc1f4810 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdac400c390 tx=0x7fdac400c850 comp rx=0 tx=0).stop 2024-05-27T09:27:25.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.230+0000 7fdae3200640 1 -- 172.21.15.114:0/1040766740 shutdown_connections 2024-05-27T09:27:25.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.230+0000 7fdae3200640 1 --2- 172.21.15.114:0/1040766740 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdab80762d0 0x7fdab80787e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:25.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.230+0000 7fdae3200640 1 --2- 172.21.15.114:0/1040766740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdadc1593b0 0x7fdadc0c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:25.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.230+0000 7fdae3200640 1 --2- 172.21.15.114:0/1040766740 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdadc0c2990 0x7fdadc1f4810 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:25.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.230+0000 7fdae3200640 1 --2- 172.21.15.114:0/1040766740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdadc073260 0x7fdadc15c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:25.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.230+0000 7fdae3200640 1 -- 172.21.15.114:0/1040766740 >> 172.21.15.114:0/1040766740 conn(0x7fdadc06f6e0 msgr2=0x7fdadc153410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:25.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.230+0000 7fdae3200640 1 -- 172.21.15.114:0/1040766740 shutdown_connections 2024-05-27T09:27:25.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.230+0000 7fdae3200640 1 -- 172.21.15.114:0/1040766740 wait complete. 2024-05-27T09:27:25.255 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-05-27T09:27:24.763558+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217932,"kb_used_data":1760,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756788,"statfs":{"total":773094113280,"available":772870950912,"internally_reserved":0,"allocated":1802240,"data_stored":1762136,"data_compressed":65078,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.233569"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544950+0000","last_change":"2024-05-27T09:27:18.585655+0000","last_active":"2024-05-27T09:27:19.544950+0000","last_peered":"2024-05-27T09:27:19.544950+0000","last_clean":"2024-05-27T09:27:19.544950+0000","last_became_active":"2024-05-27T09:27:18.585169+0000","last_became_peered":"2024-05-27T09:27:18.585169+0000","last_unstale":"2024-05-27T09:27:19.544950+0000","last_undegraded":"2024-05-27T09:27:19.544950+0000","last_fullsized":"2024-05-27T09:27:19.544950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:39:59.034873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835983+0000","last_change":"2024-05-27T09:27:18.566621+0000","last_active":"2024-05-27T09:27:20.835983+0000","last_peered":"2024-05-27T09:27:20.835983+0000","last_clean":"2024-05-27T09:27:20.835983+0000","last_became_active":"2024-05-27T09:27:18.566269+0000","last_became_peered":"2024-05-27T09:27:18.566269+0000","last_unstale":"2024-05-27T09:27:20.835983+0000","last_undegraded":"2024-05-27T09:27:20.835983+0000","last_fullsized":"2024-05-27T09:27:20.835983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:24:24.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835531+0000","last_change":"2024-05-27T09:27:18.560958+0000","last_active":"2024-05-27T09:27:20.835531+0000","last_peered":"2024-05-27T09:27:20.835531+0000","last_clean":"2024-05-27T09:27:20.835531+0000","last_became_active":"2024-05-27T09:27:18.560620+0000","last_became_peered":"2024-05-27T09:27:18.560620+0000","last_unstale":"2024-05-27T09:27:20.835531+0000","last_undegraded":"2024-05-27T09:27:20.835531+0000","last_fullsized":"2024-05-27T09:27:20.835531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:42:38.351682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542664+0000","last_change":"2024-05-27T09:27:18.569804+0000","last_active":"2024-05-27T09:27:19.542664+0000","last_peered":"2024-05-27T09:27:19.542664+0000","last_clean":"2024-05-27T09:27:19.542664+0000","last_became_active":"2024-05-27T09:27:18.569481+0000","last_became_peered":"2024-05-27T09:27:18.569481+0000","last_unstale":"2024-05-27T09:27:19.542664+0000","last_undegraded":"2024-05-27T09:27:19.542664+0000","last_fullsized":"2024-05-27T09:27:19.542664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:55:03.583659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543808+0000","last_change":"2024-05-27T09:27:18.568138+0000","last_active":"2024-05-27T09:27:19.543808+0000","last_peered":"2024-05-27T09:27:19.543808+0000","last_clean":"2024-05-27T09:27:19.543808+0000","last_became_active":"2024-05-27T09:27:18.567795+0000","last_became_peered":"2024-05-27T09:27:18.567795+0000","last_unstale":"2024-05-27T09:27:19.543808+0000","last_undegraded":"2024-05-27T09:27:19.543808+0000","last_fullsized":"2024-05-27T09:27:19.543808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:17:42.005593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.540719+0000","last_change":"2024-05-27T09:27:18.573984+0000","last_active":"2024-05-27T09:27:19.540719+0000","last_peered":"2024-05-27T09:27:19.540719+0000","last_clean":"2024-05-27T09:27:19.540719+0000","last_became_active":"2024-05-27T09:27:18.573576+0000","last_became_peered":"2024-05-27T09:27:18.573576+0000","last_unstale":"2024-05-27T09:27:19.540719+0000","last_undegraded":"2024-05-27T09:27:19.540719+0000","last_fullsized":"2024-05-27T09:27:19.540719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:06.224637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543010+0000","last_change":"2024-05-27T09:27:18.543122+0000","last_active":"2024-05-27T09:27:19.543010+0000","last_peered":"2024-05-27T09:27:19.543010+0000","last_clean":"2024-05-27T09:27:19.543010+0000","last_became_active":"2024-05-27T09:27:18.542593+0000","last_became_peered":"2024-05-27T09:27:18.542593+0000","last_unstale":"2024-05-27T09:27:19.543010+0000","last_undegraded":"2024-05-27T09:27:19.543010+0000","last_fullsized":"2024-05-27T09:27:19.543010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:28:19.579432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835946+0000","last_change":"2024-05-27T09:27:18.562292+0000","last_active":"2024-05-27T09:27:20.835946+0000","last_peered":"2024-05-27T09:27:20.835946+0000","last_clean":"2024-05-27T09:27:20.835946+0000","last_became_active":"2024-05-27T09:27:18.561788+0000","last_became_peered":"2024-05-27T09:27:18.561788+0000","last_unstale":"2024-05-27T09:27:20.835946+0000","last_undegraded":"2024-05-27T09:27:20.835946+0000","last_fullsized":"2024-05-27T09:27:20.835946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:32:39.043268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542996+0000","last_change":"2024-05-27T09:27:18.583770+0000","last_active":"2024-05-27T09:27:19.542996+0000","last_peered":"2024-05-27T09:27:19.542996+0000","last_clean":"2024-05-27T09:27:19.542996+0000","last_became_active":"2024-05-27T09:27:18.582041+0000","last_became_peered":"2024-05-27T09:27:18.582041+0000","last_unstale":"2024-05-27T09:27:19.542996+0000","last_undegraded":"2024-05-27T09:27:19.542996+0000","last_fullsized":"2024-05-27T09:27:19.542996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:19.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.545125+0000","last_change":"2024-05-27T09:27:18.574658+0000","last_active":"2024-05-27T09:27:19.545125+0000","last_peered":"2024-05-27T09:27:19.545125+0000","last_clean":"2024-05-27T09:27:19.545125+0000","last_became_active":"2024-05-27T09:27:18.574409+0000","last_became_peered":"2024-05-27T09:27:18.574409+0000","last_unstale":"2024-05-27T09:27:19.545125+0000","last_undegraded":"2024-05-27T09:27:19.545125+0000","last_fullsized":"2024-05-27T09:27:19.545125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:19.131343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544558+0000","last_change":"2024-05-27T09:27:18.579141+0000","last_active":"2024-05-27T09:27:19.544558+0000","last_peered":"2024-05-27T09:27:19.544558+0000","last_clean":"2024-05-27T09:27:19.544558+0000","last_became_active":"2024-05-27T09:27:18.578679+0000","last_became_peered":"2024-05-27T09:27:18.578679+0000","last_unstale":"2024-05-27T09:27:19.544558+0000","last_undegraded":"2024-05-27T09:27:19.544558+0000","last_fullsized":"2024-05-27T09:27:19.544558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:44.336453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544110+0000","last_change":"2024-05-27T09:27:18.547375+0000","last_active":"2024-05-27T09:27:19.544110+0000","last_peered":"2024-05-27T09:27:19.544110+0000","last_clean":"2024-05-27T09:27:19.544110+0000","last_became_active":"2024-05-27T09:27:18.544328+0000","last_became_peered":"2024-05-27T09:27:18.544328+0000","last_unstale":"2024-05-27T09:27:19.544110+0000","last_undegraded":"2024-05-27T09:27:19.544110+0000","last_fullsized":"2024-05-27T09:27:19.544110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:59:50.499894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836590+0000","last_change":"2024-05-27T09:27:18.561903+0000","last_active":"2024-05-27T09:27:20.836590+0000","last_peered":"2024-05-27T09:27:20.836590+0000","last_clean":"2024-05-27T09:27:20.836590+0000","last_became_active":"2024-05-27T09:27:18.561493+0000","last_became_peered":"2024-05-27T09:27:18.561493+0000","last_unstale":"2024-05-27T09:27:20.836590+0000","last_undegraded":"2024-05-27T09:27:20.836590+0000","last_fullsized":"2024-05-27T09:27:20.836590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:28:31.470269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836378+0000","last_change":"2024-05-27T09:27:18.563168+0000","last_active":"2024-05-27T09:27:20.836378+0000","last_peered":"2024-05-27T09:27:20.836378+0000","last_clean":"2024-05-27T09:27:20.836378+0000","last_became_active":"2024-05-27T09:27:18.562843+0000","last_became_peered":"2024-05-27T09:27:18.562843+0000","last_unstale":"2024-05-27T09:27:20.836378+0000","last_undegraded":"2024-05-27T09:27:20.836378+0000","last_fullsized":"2024-05-27T09:27:20.836378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:33:36.671764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835457+0000","last_change":"2024-05-27T09:27:18.566502+0000","last_active":"2024-05-27T09:27:20.835457+0000","last_peered":"2024-05-27T09:27:20.835457+0000","last_clean":"2024-05-27T09:27:20.835457+0000","last_became_active":"2024-05-27T09:27:18.566108+0000","last_became_peered":"2024-05-27T09:27:18.566108+0000","last_unstale":"2024-05-27T09:27:20.835457+0000","last_undegraded":"2024-05-27T09:27:20.835457+0000","last_fullsized":"2024-05-27T09:27:20.835457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:44:47.447245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836471+0000","last_change":"2024-05-27T09:27:18.582595+0000","last_active":"2024-05-27T09:27:20.836471+0000","last_peered":"2024-05-27T09:27:20.836471+0000","last_clean":"2024-05-27T09:27:20.836471+0000","last_became_active":"2024-05-27T09:27:18.581719+0000","last_became_peered":"2024-05-27T09:27:18.581719+0000","last_unstale":"2024-05-27T09:27:20.836471+0000","last_undegraded":"2024-05-27T09:27:20.836471+0000","last_fullsized":"2024-05-27T09:27:20.836471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:48:19.573654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837234+0000","last_change":"2024-05-27T09:27:18.577565+0000","last_active":"2024-05-27T09:27:20.837234+0000","last_peered":"2024-05-27T09:27:20.837234+0000","last_clean":"2024-05-27T09:27:20.837234+0000","last_became_active":"2024-05-27T09:27:18.577193+0000","last_became_peered":"2024-05-27T09:27:18.577193+0000","last_unstale":"2024-05-27T09:27:20.837234+0000","last_undegraded":"2024-05-27T09:27:20.837234+0000","last_fullsized":"2024-05-27T09:27:20.837234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:43:20.065873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835530+0000","last_change":"2024-05-27T09:27:18.571089+0000","last_active":"2024-05-27T09:27:20.835530+0000","last_peered":"2024-05-27T09:27:20.835530+0000","last_clean":"2024-05-27T09:27:20.835530+0000","last_became_active":"2024-05-27T09:27:18.570712+0000","last_became_peered":"2024-05-27T09:27:18.570712+0000","last_unstale":"2024-05-27T09:27:20.835530+0000","last_undegraded":"2024-05-27T09:27:20.835530+0000","last_fullsized":"2024-05-27T09:27:20.835530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:40:19.310581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837956+0000","last_change":"2024-05-27T09:27:18.572675+0000","last_active":"2024-05-27T09:27:20.837956+0000","last_peered":"2024-05-27T09:27:20.837956+0000","last_clean":"2024-05-27T09:27:20.837956+0000","last_became_active":"2024-05-27T09:27:18.564643+0000","last_became_peered":"2024-05-27T09:27:18.564643+0000","last_unstale":"2024-05-27T09:27:20.837956+0000","last_undegraded":"2024-05-27T09:27:20.837956+0000","last_fullsized":"2024-05-27T09:27:20.837956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:36:15.602241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544179+0000","last_change":"2024-05-27T09:27:18.544554+0000","last_active":"2024-05-27T09:27:19.544179+0000","last_peered":"2024-05-27T09:27:19.544179+0000","last_clean":"2024-05-27T09:27:19.544179+0000","last_became_active":"2024-05-27T09:27:18.543777+0000","last_became_peered":"2024-05-27T09:27:18.543777+0000","last_unstale":"2024-05-27T09:27:19.544179+0000","last_undegraded":"2024-05-27T09:27:19.544179+0000","last_fullsized":"2024-05-27T09:27:19.544179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:35:43.354270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835033+0000","last_change":"2024-05-27T09:27:18.571321+0000","last_active":"2024-05-27T09:27:20.835033+0000","last_peered":"2024-05-27T09:27:20.835033+0000","last_clean":"2024-05-27T09:27:20.835033+0000","last_became_active":"2024-05-27T09:27:18.570948+0000","last_became_peered":"2024-05-27T09:27:18.570948+0000","last_unstale":"2024-05-27T09:27:20.835033+0000","last_undegraded":"2024-05-27T09:27:20.835033+0000","last_fullsized":"2024-05-27T09:27:20.835033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:28:31.470269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539490+0000","last_change":"2024-05-27T09:27:18.544282+0000","last_active":"2024-05-27T09:27:19.539490+0000","last_peered":"2024-05-27T09:27:19.539490+0000","last_clean":"2024-05-27T09:27:19.539490+0000","last_became_active":"2024-05-27T09:27:18.543921+0000","last_became_peered":"2024-05-27T09:27:18.543921+0000","last_unstale":"2024-05-27T09:27:19.539490+0000","last_undegraded":"2024-05-27T09:27:19.539490+0000","last_fullsized":"2024-05-27T09:27:19.539490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:27.225150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835732+0000","last_change":"2024-05-27T09:27:18.575415+0000","last_active":"2024-05-27T09:27:20.835732+0000","last_peered":"2024-05-27T09:27:20.835732+0000","last_clean":"2024-05-27T09:27:20.835732+0000","last_became_active":"2024-05-27T09:27:18.575115+0000","last_became_peered":"2024-05-27T09:27:18.575115+0000","last_unstale":"2024-05-27T09:27:20.835732+0000","last_undegraded":"2024-05-27T09:27:20.835732+0000","last_fullsized":"2024-05-27T09:27:20.835732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:50:48.091125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836496+0000","last_change":"2024-05-27T09:27:18.564422+0000","last_active":"2024-05-27T09:27:20.836496+0000","last_peered":"2024-05-27T09:27:20.836496+0000","last_clean":"2024-05-27T09:27:20.836496+0000","last_became_active":"2024-05-27T09:27:18.563991+0000","last_became_peered":"2024-05-27T09:27:18.563991+0000","last_unstale":"2024-05-27T09:27:20.836496+0000","last_undegraded":"2024-05-27T09:27:20.836496+0000","last_fullsized":"2024-05-27T09:27:20.836496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:19.131343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836032+0000","last_change":"2024-05-27T09:27:18.563557+0000","last_active":"2024-05-27T09:27:20.836032+0000","last_peered":"2024-05-27T09:27:20.836032+0000","last_clean":"2024-05-27T09:27:20.836032+0000","last_became_active":"2024-05-27T09:27:18.562803+0000","last_became_peered":"2024-05-27T09:27:18.562803+0000","last_unstale":"2024-05-27T09:27:20.836032+0000","last_undegraded":"2024-05-27T09:27:20.836032+0000","last_fullsized":"2024-05-27T09:27:20.836032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:47:54.271208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542223+0000","last_change":"2024-05-27T09:27:18.590430+0000","last_active":"2024-05-27T09:27:19.542223+0000","last_peered":"2024-05-27T09:27:19.542223+0000","last_clean":"2024-05-27T09:27:19.542223+0000","last_became_active":"2024-05-27T09:27:18.590083+0000","last_became_peered":"2024-05-27T09:27:18.590083+0000","last_unstale":"2024-05-27T09:27:19.542223+0000","last_undegraded":"2024-05-27T09:27:19.542223+0000","last_fullsized":"2024-05-27T09:27:19.542223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:03.762818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837167+0000","last_change":"2024-05-27T09:27:18.558248+0000","last_active":"2024-05-27T09:27:20.837167+0000","last_peered":"2024-05-27T09:27:20.837167+0000","last_clean":"2024-05-27T09:27:20.837167+0000","last_became_active":"2024-05-27T09:27:18.557953+0000","last_became_peered":"2024-05-27T09:27:18.557953+0000","last_unstale":"2024-05-27T09:27:20.837167+0000","last_undegraded":"2024-05-27T09:27:20.837167+0000","last_fullsized":"2024-05-27T09:27:20.837167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:44:47.447245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836514+0000","last_change":"2024-05-27T09:27:18.563213+0000","last_active":"2024-05-27T09:27:20.836514+0000","last_peered":"2024-05-27T09:27:20.836514+0000","last_clean":"2024-05-27T09:27:20.836514+0000","last_became_active":"2024-05-27T09:27:18.562839+0000","last_became_peered":"2024-05-27T09:27:18.562839+0000","last_unstale":"2024-05-27T09:27:20.836514+0000","last_undegraded":"2024-05-27T09:27:20.836514+0000","last_fullsized":"2024-05-27T09:27:20.836514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:43.851698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836053+0000","last_change":"2024-05-27T09:27:18.562237+0000","last_active":"2024-05-27T09:27:20.836053+0000","last_peered":"2024-05-27T09:27:20.836053+0000","last_clean":"2024-05-27T09:27:20.836053+0000","last_became_active":"2024-05-27T09:27:18.561475+0000","last_became_peered":"2024-05-27T09:27:18.561475+0000","last_unstale":"2024-05-27T09:27:20.836053+0000","last_undegraded":"2024-05-27T09:27:20.836053+0000","last_fullsized":"2024-05-27T09:27:20.836053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:39:58.110527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837611+0000","last_change":"2024-05-27T09:27:18.563469+0000","last_active":"2024-05-27T09:27:20.837611+0000","last_peered":"2024-05-27T09:27:20.837611+0000","last_clean":"2024-05-27T09:27:20.837611+0000","last_became_active":"2024-05-27T09:27:18.563103+0000","last_became_peered":"2024-05-27T09:27:18.563103+0000","last_unstale":"2024-05-27T09:27:20.837611+0000","last_undegraded":"2024-05-27T09:27:20.837611+0000","last_fullsized":"2024-05-27T09:27:20.837611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:53:57.281169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.834984+0000","last_change":"2024-05-27T09:27:18.560551+0000","last_active":"2024-05-27T09:27:20.834984+0000","last_peered":"2024-05-27T09:27:20.834984+0000","last_clean":"2024-05-27T09:27:20.834984+0000","last_became_active":"2024-05-27T09:27:18.560244+0000","last_became_peered":"2024-05-27T09:27:18.560244+0000","last_unstale":"2024-05-27T09:27:20.834984+0000","last_undegraded":"2024-05-27T09:27:20.834984+0000","last_fullsized":"2024-05-27T09:27:20.834984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:00.131180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.538536+0000","last_change":"2024-05-27T09:27:18.568200+0000","last_active":"2024-05-27T09:27:19.538536+0000","last_peered":"2024-05-27T09:27:19.538536+0000","last_clean":"2024-05-27T09:27:19.538536+0000","last_became_active":"2024-05-27T09:27:18.567845+0000","last_became_peered":"2024-05-27T09:27:18.567845+0000","last_unstale":"2024-05-27T09:27:19.538536+0000","last_undegraded":"2024-05-27T09:27:19.538536+0000","last_fullsized":"2024-05-27T09:27:19.538536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:54.726617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.540019+0000","last_change":"2024-05-27T09:27:18.560521+0000","last_active":"2024-05-27T09:27:19.540019+0000","last_peered":"2024-05-27T09:27:19.540019+0000","last_clean":"2024-05-27T09:27:19.540019+0000","last_became_active":"2024-05-27T09:27:18.556156+0000","last_became_peered":"2024-05-27T09:27:18.556156+0000","last_unstale":"2024-05-27T09:27:19.540019+0000","last_undegraded":"2024-05-27T09:27:19.540019+0000","last_fullsized":"2024-05-27T09:27:19.540019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:03.762818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835268+0000","last_change":"2024-05-27T09:27:17.548038+0000","last_active":"2024-05-27T09:27:20.835268+0000","last_peered":"2024-05-27T09:27:20.835268+0000","last_clean":"2024-05-27T09:27:20.835268+0000","last_became_active":"2024-05-27T09:27:17.547653+0000","last_became_peered":"2024-05-27T09:27:17.547653+0000","last_unstale":"2024-05-27T09:27:20.835268+0000","last_undegraded":"2024-05-27T09:27:20.835268+0000","last_fullsized":"2024-05-27T09:27:20.835268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835265+0000","last_change":"2024-05-27T09:27:18.543048+0000","last_active":"2024-05-27T09:27:20.835265+0000","last_peered":"2024-05-27T09:27:20.835265+0000","last_clean":"2024-05-27T09:27:20.835265+0000","last_became_active":"2024-05-27T09:27:18.542715+0000","last_became_peered":"2024-05-27T09:27:18.542715+0000","last_unstale":"2024-05-27T09:27:20.835265+0000","last_undegraded":"2024-05-27T09:27:20.835265+0000","last_fullsized":"2024-05-27T09:27:20.835265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:46:49.444709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837211+0000","last_change":"2024-05-27T09:27:17.570014+0000","last_active":"2024-05-27T09:27:20.837211+0000","last_peered":"2024-05-27T09:27:20.837211+0000","last_clean":"2024-05-27T09:27:20.837211+0000","last_became_active":"2024-05-27T09:27:17.569673+0000","last_became_peered":"2024-05-27T09:27:17.569673+0000","last_unstale":"2024-05-27T09:27:20.837211+0000","last_undegraded":"2024-05-27T09:27:20.837211+0000","last_fullsized":"2024-05-27T09:27:20.837211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:26.210041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836496+0000","last_change":"2024-05-27T09:27:17.565489+0000","last_active":"2024-05-27T09:27:20.836496+0000","last_peered":"2024-05-27T09:27:20.836496+0000","last_clean":"2024-05-27T09:27:20.836496+0000","last_became_active":"2024-05-27T09:27:17.565137+0000","last_became_peered":"2024-05-27T09:27:17.565137+0000","last_unstale":"2024-05-27T09:27:20.836496+0000","last_undegraded":"2024-05-27T09:27:20.836496+0000","last_fullsized":"2024-05-27T09:27:20.836496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836835+0000","last_change":"2024-05-27T09:27:18.565294+0000","last_active":"2024-05-27T09:27:20.836835+0000","last_peered":"2024-05-27T09:27:20.836835+0000","last_clean":"2024-05-27T09:27:20.836835+0000","last_became_active":"2024-05-27T09:27:18.564115+0000","last_became_peered":"2024-05-27T09:27:18.564115+0000","last_unstale":"2024-05-27T09:27:20.836835+0000","last_undegraded":"2024-05-27T09:27:20.836835+0000","last_fullsized":"2024-05-27T09:27:20.836835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:57:20.488849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836662+0000","last_change":"2024-05-27T09:27:17.561204+0000","last_active":"2024-05-27T09:27:20.836662+0000","last_peered":"2024-05-27T09:27:20.836662+0000","last_clean":"2024-05-27T09:27:20.836662+0000","last_became_active":"2024-05-27T09:27:17.560149+0000","last_became_peered":"2024-05-27T09:27:17.560149+0000","last_unstale":"2024-05-27T09:27:20.836662+0000","last_undegraded":"2024-05-27T09:27:20.836662+0000","last_fullsized":"2024-05-27T09:27:20.836662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.284270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836343+0000","last_change":"2024-05-27T09:27:18.542204+0000","last_active":"2024-05-27T09:27:20.836343+0000","last_peered":"2024-05-27T09:27:20.836343+0000","last_clean":"2024-05-27T09:27:20.836343+0000","last_became_active":"2024-05-27T09:27:18.541946+0000","last_became_peered":"2024-05-27T09:27:18.541946+0000","last_unstale":"2024-05-27T09:27:20.836343+0000","last_undegraded":"2024-05-27T09:27:20.836343+0000","last_fullsized":"2024-05-27T09:27:20.836343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:50:48.091125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.541754+0000","last_change":"2024-05-27T09:27:17.561714+0000","last_active":"2024-05-27T09:27:19.541754+0000","last_peered":"2024-05-27T09:27:19.541754+0000","last_clean":"2024-05-27T09:27:19.541754+0000","last_became_active":"2024-05-27T09:27:17.561355+0000","last_became_peered":"2024-05-27T09:27:17.561355+0000","last_unstale":"2024-05-27T09:27:19.541754+0000","last_undegraded":"2024-05-27T09:27:19.541754+0000","last_fullsized":"2024-05-27T09:27:19.541754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:07:02.481451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542128+0000","last_change":"2024-05-27T09:27:18.588199+0000","last_active":"2024-05-27T09:27:19.542128+0000","last_peered":"2024-05-27T09:27:19.542128+0000","last_clean":"2024-05-27T09:27:19.542128+0000","last_became_active":"2024-05-27T09:27:18.587921+0000","last_became_peered":"2024-05-27T09:27:18.587921+0000","last_unstale":"2024-05-27T09:27:19.542128+0000","last_undegraded":"2024-05-27T09:27:19.542128+0000","last_fullsized":"2024-05-27T09:27:19.542128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:17.895211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544807+0000","last_change":"2024-05-27T09:27:17.565854+0000","last_active":"2024-05-27T09:27:19.544807+0000","last_peered":"2024-05-27T09:27:19.544807+0000","last_clean":"2024-05-27T09:27:19.544807+0000","last_became_active":"2024-05-27T09:27:17.565488+0000","last_became_peered":"2024-05-27T09:27:17.565488+0000","last_unstale":"2024-05-27T09:27:19.544807+0000","last_undegraded":"2024-05-27T09:27:19.544807+0000","last_fullsized":"2024-05-27T09:27:19.544807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:53.711508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836966+0000","last_change":"2024-05-27T09:27:18.585592+0000","last_active":"2024-05-27T09:27:20.836966+0000","last_peered":"2024-05-27T09:27:20.836966+0000","last_clean":"2024-05-27T09:27:20.836966+0000","last_became_active":"2024-05-27T09:27:18.585243+0000","last_became_peered":"2024-05-27T09:27:18.585243+0000","last_unstale":"2024-05-27T09:27:20.836966+0000","last_undegraded":"2024-05-27T09:27:20.836966+0000","last_fullsized":"2024-05-27T09:27:20.836966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:28.872653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835317+0000","last_change":"2024-05-27T09:27:17.548651+0000","last_active":"2024-05-27T09:27:20.835317+0000","last_peered":"2024-05-27T09:27:20.835317+0000","last_clean":"2024-05-27T09:27:20.835317+0000","last_became_active":"2024-05-27T09:27:17.548307+0000","last_became_peered":"2024-05-27T09:27:17.548307+0000","last_unstale":"2024-05-27T09:27:20.835317+0000","last_undegraded":"2024-05-27T09:27:20.835317+0000","last_fullsized":"2024-05-27T09:27:20.835317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:48.905422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837273+0000","last_change":"2024-05-27T09:27:18.582895+0000","last_active":"2024-05-27T09:27:20.837273+0000","last_peered":"2024-05-27T09:27:20.837273+0000","last_clean":"2024-05-27T09:27:20.837273+0000","last_became_active":"2024-05-27T09:27:18.581742+0000","last_became_peered":"2024-05-27T09:27:18.581742+0000","last_unstale":"2024-05-27T09:27:20.837273+0000","last_undegraded":"2024-05-27T09:27:20.837273+0000","last_fullsized":"2024-05-27T09:27:20.837273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:23:26.888905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542758+0000","last_change":"2024-05-27T09:27:17.556419+0000","last_active":"2024-05-27T09:27:19.542758+0000","last_peered":"2024-05-27T09:27:19.542758+0000","last_clean":"2024-05-27T09:27:19.542758+0000","last_became_active":"2024-05-27T09:27:17.556036+0000","last_became_peered":"2024-05-27T09:27:17.556036+0000","last_unstale":"2024-05-27T09:27:19.542758+0000","last_undegraded":"2024-05-27T09:27:19.542758+0000","last_fullsized":"2024-05-27T09:27:19.542758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:48:20.718085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544281+0000","last_change":"2024-05-27T09:27:18.568187+0000","last_active":"2024-05-27T09:27:19.544281+0000","last_peered":"2024-05-27T09:27:19.544281+0000","last_clean":"2024-05-27T09:27:19.544281+0000","last_became_active":"2024-05-27T09:27:18.565500+0000","last_became_peered":"2024-05-27T09:27:18.565500+0000","last_unstale":"2024-05-27T09:27:19.544281+0000","last_undegraded":"2024-05-27T09:27:19.544281+0000","last_fullsized":"2024-05-27T09:27:19.544281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:16:08.688048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835570+0000","last_change":"2024-05-27T09:27:17.572168+0000","last_active":"2024-05-27T09:27:20.835570+0000","last_peered":"2024-05-27T09:27:20.835570+0000","last_clean":"2024-05-27T09:27:20.835570+0000","last_became_active":"2024-05-27T09:27:17.571709+0000","last_became_peered":"2024-05-27T09:27:17.571709+0000","last_unstale":"2024-05-27T09:27:20.835570+0000","last_undegraded":"2024-05-27T09:27:20.835570+0000","last_fullsized":"2024-05-27T09:27:20.835570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836574+0000","last_change":"2024-05-27T09:27:18.564930+0000","last_active":"2024-05-27T09:27:20.836574+0000","last_peered":"2024-05-27T09:27:20.836574+0000","last_clean":"2024-05-27T09:27:20.836574+0000","last_became_active":"2024-05-27T09:27:18.564589+0000","last_became_peered":"2024-05-27T09:27:18.564589+0000","last_unstale":"2024-05-27T09:27:20.836574+0000","last_undegraded":"2024-05-27T09:27:20.836574+0000","last_fullsized":"2024-05-27T09:27:20.836574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:21:00.644551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837541+0000","last_change":"2024-05-27T09:27:17.561344+0000","last_active":"2024-05-27T09:27:20.837541+0000","last_peered":"2024-05-27T09:27:20.837541+0000","last_clean":"2024-05-27T09:27:20.837541+0000","last_became_active":"2024-05-27T09:27:17.554860+0000","last_became_peered":"2024-05-27T09:27:17.554860+0000","last_unstale":"2024-05-27T09:27:20.837541+0000","last_undegraded":"2024-05-27T09:27:20.837541+0000","last_fullsized":"2024-05-27T09:27:20.837541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:16.880101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837236+0000","last_change":"2024-05-27T09:27:18.575317+0000","last_active":"2024-05-27T09:27:20.837236+0000","last_peered":"2024-05-27T09:27:20.837236+0000","last_clean":"2024-05-27T09:27:20.837236+0000","last_became_active":"2024-05-27T09:27:18.572024+0000","last_became_peered":"2024-05-27T09:27:18.572024+0000","last_unstale":"2024-05-27T09:27:20.837236+0000","last_undegraded":"2024-05-27T09:27:20.837236+0000","last_fullsized":"2024-05-27T09:27:20.837236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:23:35.418697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543181+0000","last_change":"2024-05-27T09:27:17.575859+0000","last_active":"2024-05-27T09:27:19.543181+0000","last_peered":"2024-05-27T09:27:19.543181+0000","last_clean":"2024-05-27T09:27:19.543181+0000","last_became_active":"2024-05-27T09:27:17.574695+0000","last_became_peered":"2024-05-27T09:27:17.574695+0000","last_unstale":"2024-05-27T09:27:19.543181+0000","last_undegraded":"2024-05-27T09:27:19.543181+0000","last_fullsized":"2024-05-27T09:27:19.543181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:24:19.379819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539060+0000","last_change":"2024-05-27T09:27:18.555137+0000","last_active":"2024-05-27T09:27:19.539060+0000","last_peered":"2024-05-27T09:27:19.539060+0000","last_clean":"2024-05-27T09:27:19.539060+0000","last_became_active":"2024-05-27T09:27:18.554932+0000","last_became_peered":"2024-05-27T09:27:18.554932+0000","last_unstale":"2024-05-27T09:27:19.539060+0000","last_undegraded":"2024-05-27T09:27:19.539060+0000","last_fullsized":"2024-05-27T09:27:19.539060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:06.224637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835234+0000","last_change":"2024-05-27T09:27:17.547895+0000","last_active":"2024-05-27T09:27:20.835234+0000","last_peered":"2024-05-27T09:27:20.835234+0000","last_clean":"2024-05-27T09:27:20.835234+0000","last_became_active":"2024-05-27T09:27:17.547523+0000","last_became_peered":"2024-05-27T09:27:17.547523+0000","last_unstale":"2024-05-27T09:27:20.835234+0000","last_undegraded":"2024-05-27T09:27:20.835234+0000","last_fullsized":"2024-05-27T09:27:20.835234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836706+0000","last_change":"2024-05-27T09:27:18.584284+0000","last_active":"2024-05-27T09:27:20.836706+0000","last_peered":"2024-05-27T09:27:20.836706+0000","last_clean":"2024-05-27T09:27:20.836706+0000","last_became_active":"2024-05-27T09:27:18.582360+0000","last_became_peered":"2024-05-27T09:27:18.582360+0000","last_unstale":"2024-05-27T09:27:20.836706+0000","last_undegraded":"2024-05-27T09:27:20.836706+0000","last_fullsized":"2024-05-27T09:27:20.836706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:32.535070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.545248+0000","last_change":"2024-05-27T09:27:17.571481+0000","last_active":"2024-05-27T09:27:19.545248+0000","last_peered":"2024-05-27T09:27:19.545248+0000","last_clean":"2024-05-27T09:27:19.545248+0000","last_became_active":"2024-05-27T09:27:17.571146+0000","last_became_peered":"2024-05-27T09:27:17.571146+0000","last_unstale":"2024-05-27T09:27:19.545248+0000","last_undegraded":"2024-05-27T09:27:19.545248+0000","last_fullsized":"2024-05-27T09:27:19.545248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:18.038221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542481+0000","last_change":"2024-05-27T09:27:18.580425+0000","last_active":"2024-05-27T09:27:19.542481+0000","last_peered":"2024-05-27T09:27:19.542481+0000","last_clean":"2024-05-27T09:27:19.542481+0000","last_became_active":"2024-05-27T09:27:18.579440+0000","last_became_peered":"2024-05-27T09:27:18.579440+0000","last_unstale":"2024-05-27T09:27:19.542481+0000","last_undegraded":"2024-05-27T09:27:19.542481+0000","last_fullsized":"2024-05-27T09:27:19.542481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:19.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835526+0000","last_change":"2024-05-27T09:27:17.551295+0000","last_active":"2024-05-27T09:27:20.835526+0000","last_peered":"2024-05-27T09:27:20.835526+0000","last_clean":"2024-05-27T09:27:20.835526+0000","last_became_active":"2024-05-27T09:27:17.550861+0000","last_became_peered":"2024-05-27T09:27:17.550861+0000","last_unstale":"2024-05-27T09:27:20.835526+0000","last_undegraded":"2024-05-27T09:27:20.835526+0000","last_fullsized":"2024-05-27T09:27:20.835526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:13.955191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.545103+0000","last_change":"2024-05-27T09:27:18.547199+0000","last_active":"2024-05-27T09:27:19.545103+0000","last_peered":"2024-05-27T09:27:19.545103+0000","last_clean":"2024-05-27T09:27:19.545103+0000","last_became_active":"2024-05-27T09:27:18.543683+0000","last_became_peered":"2024-05-27T09:27:18.543683+0000","last_unstale":"2024-05-27T09:27:19.545103+0000","last_undegraded":"2024-05-27T09:27:19.545103+0000","last_fullsized":"2024-05-27T09:27:19.545103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:48:19.573654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835661+0000","last_change":"2024-05-27T09:27:17.559255+0000","last_active":"2024-05-27T09:27:20.835661+0000","last_peered":"2024-05-27T09:27:20.835661+0000","last_clean":"2024-05-27T09:27:20.835661+0000","last_became_active":"2024-05-27T09:27:17.558973+0000","last_became_peered":"2024-05-27T09:27:17.558973+0000","last_unstale":"2024-05-27T09:27:20.835661+0000","last_undegraded":"2024-05-27T09:27:20.835661+0000","last_fullsized":"2024-05-27T09:27:20.835661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544602+0000","last_change":"2024-05-27T09:27:18.557423+0000","last_active":"2024-05-27T09:27:19.544602+0000","last_peered":"2024-05-27T09:27:19.544602+0000","last_clean":"2024-05-27T09:27:19.544602+0000","last_became_active":"2024-05-27T09:27:18.557076+0000","last_became_peered":"2024-05-27T09:27:18.557076+0000","last_unstale":"2024-05-27T09:27:19.544602+0000","last_undegraded":"2024-05-27T09:27:19.544602+0000","last_fullsized":"2024-05-27T09:27:19.544602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:11:04.612363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836934+0000","last_change":"2024-05-27T09:27:17.573700+0000","last_active":"2024-05-27T09:27:20.836934+0000","last_peered":"2024-05-27T09:27:20.836934+0000","last_clean":"2024-05-27T09:27:20.836934+0000","last_became_active":"2024-05-27T09:27:17.573484+0000","last_became_peered":"2024-05-27T09:27:17.573484+0000","last_unstale":"2024-05-27T09:27:20.836934+0000","last_undegraded":"2024-05-27T09:27:20.836934+0000","last_fullsized":"2024-05-27T09:27:20.836934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:48.905422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837609+0000","last_change":"2024-05-27T09:27:18.582777+0000","last_active":"2024-05-27T09:27:20.837609+0000","last_peered":"2024-05-27T09:27:20.837609+0000","last_clean":"2024-05-27T09:27:20.837609+0000","last_became_active":"2024-05-27T09:27:18.574959+0000","last_became_peered":"2024-05-27T09:27:18.574959+0000","last_unstale":"2024-05-27T09:27:20.837609+0000","last_undegraded":"2024-05-27T09:27:20.837609+0000","last_fullsized":"2024-05-27T09:27:20.837609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:40:41.206755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":110,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539210+0000","last_change":"2024-05-27T09:27:12.693423+0000","last_active":"2024-05-27T09:27:19.539210+0000","last_peered":"2024-05-27T09:27:19.539210+0000","last_clean":"2024-05-27T09:27:19.539210+0000","last_became_active":"2024-05-27T09:27:12.690301+0000","last_became_peered":"2024-05-27T09:27:12.690301+0000","last_unstale":"2024-05-27T09:27:19.539210+0000","last_undegraded":"2024-05-27T09:27:19.539210+0000","last_fullsized":"2024-05-27T09:27:19.539210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:11.463226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:11.463226+0000","last_clean_scrub_stamp":"2024-05-27T09:27:11.463226+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:01.810071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837730+0000","last_change":"2024-05-27T09:27:17.567670+0000","last_active":"2024-05-27T09:27:20.837730+0000","last_peered":"2024-05-27T09:27:20.837730+0000","last_clean":"2024-05-27T09:27:20.837730+0000","last_became_active":"2024-05-27T09:27:17.566292+0000","last_became_peered":"2024-05-27T09:27:17.566292+0000","last_unstale":"2024-05-27T09:27:20.837730+0000","last_undegraded":"2024-05-27T09:27:20.837730+0000","last_fullsized":"2024-05-27T09:27:20.837730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:18.038221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836742+0000","last_change":"2024-05-27T09:27:18.571124+0000","last_active":"2024-05-27T09:27:20.836742+0000","last_peered":"2024-05-27T09:27:20.836742+0000","last_clean":"2024-05-27T09:27:20.836742+0000","last_became_active":"2024-05-27T09:27:18.570782+0000","last_became_peered":"2024-05-27T09:27:18.570782+0000","last_unstale":"2024-05-27T09:27:20.836742+0000","last_undegraded":"2024-05-27T09:27:20.836742+0000","last_fullsized":"2024-05-27T09:27:20.836742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:39:58.110527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836667+0000","last_change":"2024-05-27T09:27:17.563259+0000","last_active":"2024-05-27T09:27:20.836667+0000","last_peered":"2024-05-27T09:27:20.836667+0000","last_clean":"2024-05-27T09:27:20.836667+0000","last_became_active":"2024-05-27T09:27:17.563010+0000","last_became_peered":"2024-05-27T09:27:17.563010+0000","last_unstale":"2024-05-27T09:27:20.836667+0000","last_undegraded":"2024-05-27T09:27:20.836667+0000","last_fullsized":"2024-05-27T09:27:20.836667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:53.711508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836371+0000","last_change":"2024-05-27T09:27:18.574466+0000","last_active":"2024-05-27T09:27:20.836371+0000","last_peered":"2024-05-27T09:27:20.836371+0000","last_clean":"2024-05-27T09:27:20.836371+0000","last_became_active":"2024-05-27T09:27:18.574090+0000","last_became_peered":"2024-05-27T09:27:18.574090+0000","last_unstale":"2024-05-27T09:27:20.836371+0000","last_undegraded":"2024-05-27T09:27:20.836371+0000","last_fullsized":"2024-05-27T09:27:20.836371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:41:22.515484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835113+0000","last_change":"2024-05-27T09:27:17.551310+0000","last_active":"2024-05-27T09:27:20.835113+0000","last_peered":"2024-05-27T09:27:20.835113+0000","last_clean":"2024-05-27T09:27:20.835113+0000","last_became_active":"2024-05-27T09:27:17.550982+0000","last_became_peered":"2024-05-27T09:27:17.550982+0000","last_unstale":"2024-05-27T09:27:20.835113+0000","last_undegraded":"2024-05-27T09:27:20.835113+0000","last_fullsized":"2024-05-27T09:27:20.835113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836574+0000","last_change":"2024-05-27T09:27:18.559416+0000","last_active":"2024-05-27T09:27:20.836574+0000","last_peered":"2024-05-27T09:27:20.836574+0000","last_clean":"2024-05-27T09:27:20.836574+0000","last_became_active":"2024-05-27T09:27:18.559017+0000","last_became_peered":"2024-05-27T09:27:18.559017+0000","last_unstale":"2024-05-27T09:27:20.836574+0000","last_undegraded":"2024-05-27T09:27:20.836574+0000","last_fullsized":"2024-05-27T09:27:20.836574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:33:48.400306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837680+0000","last_change":"2024-05-27T09:27:17.563947+0000","last_active":"2024-05-27T09:27:20.837680+0000","last_peered":"2024-05-27T09:27:20.837680+0000","last_clean":"2024-05-27T09:27:20.837680+0000","last_became_active":"2024-05-27T09:27:17.563640+0000","last_became_peered":"2024-05-27T09:27:17.563640+0000","last_unstale":"2024-05-27T09:27:20.837680+0000","last_undegraded":"2024-05-27T09:27:20.837680+0000","last_fullsized":"2024-05-27T09:27:20.837680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:02.747708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543615+0000","last_change":"2024-05-27T09:27:18.575835+0000","last_active":"2024-05-27T09:27:19.543615+0000","last_peered":"2024-05-27T09:27:19.543615+0000","last_clean":"2024-05-27T09:27:19.543615+0000","last_became_active":"2024-05-27T09:27:18.574710+0000","last_became_peered":"2024-05-27T09:27:18.574710+0000","last_unstale":"2024-05-27T09:27:19.543615+0000","last_undegraded":"2024-05-27T09:27:19.543615+0000","last_fullsized":"2024-05-27T09:27:19.543615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:16:00.934068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835781+0000","last_change":"2024-05-27T09:27:17.550749+0000","last_active":"2024-05-27T09:27:20.835781+0000","last_peered":"2024-05-27T09:27:20.835781+0000","last_clean":"2024-05-27T09:27:20.835781+0000","last_became_active":"2024-05-27T09:27:17.550378+0000","last_became_peered":"2024-05-27T09:27:17.550378+0000","last_unstale":"2024-05-27T09:27:20.835781+0000","last_undegraded":"2024-05-27T09:27:20.835781+0000","last_fullsized":"2024-05-27T09:27:20.835781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:20.167629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.541045+0000","last_change":"2024-05-27T09:27:18.567233+0000","last_active":"2024-05-27T09:27:19.541045+0000","last_peered":"2024-05-27T09:27:19.541045+0000","last_clean":"2024-05-27T09:27:19.541045+0000","last_became_active":"2024-05-27T09:27:18.566326+0000","last_became_peered":"2024-05-27T09:27:18.566326+0000","last_unstale":"2024-05-27T09:27:19.541045+0000","last_undegraded":"2024-05-27T09:27:19.541045+0000","last_fullsized":"2024-05-27T09:27:19.541045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:27.225150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837758+0000","last_change":"2024-05-27T09:27:17.557912+0000","last_active":"2024-05-27T09:27:20.837758+0000","last_peered":"2024-05-27T09:27:20.837758+0000","last_clean":"2024-05-27T09:27:20.837758+0000","last_became_active":"2024-05-27T09:27:17.557216+0000","last_became_peered":"2024-05-27T09:27:17.557216+0000","last_unstale":"2024-05-27T09:27:20.837758+0000","last_undegraded":"2024-05-27T09:27:20.837758+0000","last_fullsized":"2024-05-27T09:27:20.837758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:21:22.020724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836847+0000","last_change":"2024-05-27T09:27:18.563649+0000","last_active":"2024-05-27T09:27:20.836847+0000","last_peered":"2024-05-27T09:27:20.836847+0000","last_clean":"2024-05-27T09:27:20.836847+0000","last_became_active":"2024-05-27T09:27:18.562565+0000","last_became_peered":"2024-05-27T09:27:18.562565+0000","last_unstale":"2024-05-27T09:27:20.836847+0000","last_undegraded":"2024-05-27T09:27:20.836847+0000","last_fullsized":"2024-05-27T09:27:20.836847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:46:49.444709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835661+0000","last_change":"2024-05-27T09:27:17.571911+0000","last_active":"2024-05-27T09:27:20.835661+0000","last_peered":"2024-05-27T09:27:20.835661+0000","last_clean":"2024-05-27T09:27:20.835661+0000","last_became_active":"2024-05-27T09:27:17.571551+0000","last_became_peered":"2024-05-27T09:27:17.571551+0000","last_unstale":"2024-05-27T09:27:20.835661+0000","last_undegraded":"2024-05-27T09:27:20.835661+0000","last_fullsized":"2024-05-27T09:27:20.835661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:20.167629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835899+0000","last_change":"2024-05-27T09:27:18.558780+0000","last_active":"2024-05-27T09:27:20.835899+0000","last_peered":"2024-05-27T09:27:20.835899+0000","last_clean":"2024-05-27T09:27:20.835899+0000","last_became_active":"2024-05-27T09:27:18.558463+0000","last_became_peered":"2024-05-27T09:27:18.558463+0000","last_unstale":"2024-05-27T09:27:20.835899+0000","last_undegraded":"2024-05-27T09:27:20.835899+0000","last_fullsized":"2024-05-27T09:27:20.835899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:42:38.351682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837243+0000","last_change":"2024-05-27T09:27:17.567973+0000","last_active":"2024-05-27T09:27:20.837243+0000","last_peered":"2024-05-27T09:27:20.837243+0000","last_clean":"2024-05-27T09:27:20.837243+0000","last_became_active":"2024-05-27T09:27:17.567650+0000","last_became_peered":"2024-05-27T09:27:17.567650+0000","last_unstale":"2024-05-27T09:27:20.837243+0000","last_undegraded":"2024-05-27T09:27:20.837243+0000","last_fullsized":"2024-05-27T09:27:20.837243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:25:52.924185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835408+0000","last_change":"2024-05-27T09:27:18.554845+0000","last_active":"2024-05-27T09:27:20.835408+0000","last_peered":"2024-05-27T09:27:20.835408+0000","last_clean":"2024-05-27T09:27:20.835408+0000","last_became_active":"2024-05-27T09:27:18.554462+0000","last_became_peered":"2024-05-27T09:27:18.554462+0000","last_unstale":"2024-05-27T09:27:20.835408+0000","last_undegraded":"2024-05-27T09:27:20.835408+0000","last_fullsized":"2024-05-27T09:27:20.835408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:44:47.447245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.541867+0000","last_change":"2024-05-27T09:27:17.570300+0000","last_active":"2024-05-27T09:27:19.541867+0000","last_peered":"2024-05-27T09:27:19.541867+0000","last_clean":"2024-05-27T09:27:19.541867+0000","last_became_active":"2024-05-27T09:27:17.569685+0000","last_became_peered":"2024-05-27T09:27:17.569685+0000","last_unstale":"2024-05-27T09:27:19.541867+0000","last_undegraded":"2024-05-27T09:27:19.541867+0000","last_fullsized":"2024-05-27T09:27:19.541867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:26.210041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836430+0000","last_change":"2024-05-27T09:27:18.562774+0000","last_active":"2024-05-27T09:27:20.836430+0000","last_peered":"2024-05-27T09:27:20.836430+0000","last_clean":"2024-05-27T09:27:20.836430+0000","last_became_active":"2024-05-27T09:27:18.562455+0000","last_became_peered":"2024-05-27T09:27:18.562455+0000","last_unstale":"2024-05-27T09:27:20.836430+0000","last_undegraded":"2024-05-27T09:27:20.836430+0000","last_fullsized":"2024-05-27T09:27:20.836430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:39:59.034873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836403+0000","last_change":"2024-05-27T09:27:17.563172+0000","last_active":"2024-05-27T09:27:20.836403+0000","last_peered":"2024-05-27T09:27:20.836403+0000","last_clean":"2024-05-27T09:27:20.836403+0000","last_became_active":"2024-05-27T09:27:17.562941+0000","last_became_peered":"2024-05-27T09:27:17.562941+0000","last_unstale":"2024-05-27T09:27:20.836403+0000","last_undegraded":"2024-05-27T09:27:20.836403+0000","last_fullsized":"2024-05-27T09:27:20.836403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:51.742078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836477+0000","last_change":"2024-05-27T09:27:18.555617+0000","last_active":"2024-05-27T09:27:20.836477+0000","last_peered":"2024-05-27T09:27:20.836477+0000","last_clean":"2024-05-27T09:27:20.836477+0000","last_became_active":"2024-05-27T09:27:18.555280+0000","last_became_peered":"2024-05-27T09:27:18.555280+0000","last_unstale":"2024-05-27T09:27:20.836477+0000","last_undegraded":"2024-05-27T09:27:20.836477+0000","last_fullsized":"2024-05-27T09:27:20.836477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:32:39.043268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837070+0000","last_change":"2024-05-27T09:27:17.548266+0000","last_active":"2024-05-27T09:27:20.837070+0000","last_peered":"2024-05-27T09:27:20.837070+0000","last_clean":"2024-05-27T09:27:20.837070+0000","last_became_active":"2024-05-27T09:27:17.547938+0000","last_became_peered":"2024-05-27T09:27:17.547938+0000","last_unstale":"2024-05-27T09:27:20.837070+0000","last_undegraded":"2024-05-27T09:27:20.837070+0000","last_fullsized":"2024-05-27T09:27:20.837070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835677+0000","last_change":"2024-05-27T09:27:18.563586+0000","last_active":"2024-05-27T09:27:20.835677+0000","last_peered":"2024-05-27T09:27:20.835677+0000","last_clean":"2024-05-27T09:27:20.835677+0000","last_became_active":"2024-05-27T09:27:18.563227+0000","last_became_peered":"2024-05-27T09:27:18.563227+0000","last_unstale":"2024-05-27T09:27:20.835677+0000","last_undegraded":"2024-05-27T09:27:20.835677+0000","last_fullsized":"2024-05-27T09:27:20.835677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:53.222040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837432+0000","last_change":"2024-05-27T09:27:17.562200+0000","last_active":"2024-05-27T09:27:20.837432+0000","last_peered":"2024-05-27T09:27:20.837432+0000","last_clean":"2024-05-27T09:27:20.837432+0000","last_became_active":"2024-05-27T09:27:17.561857+0000","last_became_peered":"2024-05-27T09:27:17.561857+0000","last_unstale":"2024-05-27T09:27:20.837432+0000","last_undegraded":"2024-05-27T09:27:20.837432+0000","last_fullsized":"2024-05-27T09:27:20.837432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:05.209528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836936+0000","last_change":"2024-05-27T09:27:18.572676+0000","last_active":"2024-05-27T09:27:20.836936+0000","last_peered":"2024-05-27T09:27:20.836936+0000","last_clean":"2024-05-27T09:27:20.836936+0000","last_became_active":"2024-05-27T09:27:18.572339+0000","last_became_peered":"2024-05-27T09:27:18.572339+0000","last_unstale":"2024-05-27T09:27:20.836936+0000","last_undegraded":"2024-05-27T09:27:20.836936+0000","last_fullsized":"2024-05-27T09:27:20.836936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:09:23.795528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837085+0000","last_change":"2024-05-27T09:27:18.575394+0000","last_active":"2024-05-27T09:27:20.837085+0000","last_peered":"2024-05-27T09:27:20.837085+0000","last_clean":"2024-05-27T09:27:20.837085+0000","last_became_active":"2024-05-27T09:27:18.572307+0000","last_became_peered":"2024-05-27T09:27:18.572307+0000","last_unstale":"2024-05-27T09:27:20.837085+0000","last_undegraded":"2024-05-27T09:27:20.837085+0000","last_fullsized":"2024-05-27T09:27:20.837085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:29:05.116582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544332+0000","last_change":"2024-05-27T09:27:17.570634+0000","last_active":"2024-05-27T09:27:19.544332+0000","last_peered":"2024-05-27T09:27:19.544332+0000","last_clean":"2024-05-27T09:27:19.544332+0000","last_became_active":"2024-05-27T09:27:17.569866+0000","last_became_peered":"2024-05-27T09:27:17.569866+0000","last_unstale":"2024-05-27T09:27:19.544332+0000","last_undegraded":"2024-05-27T09:27:19.544332+0000","last_fullsized":"2024-05-27T09:27:19.544332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:21:38.268535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539980+0000","last_change":"2024-05-27T09:27:18.549339+0000","last_active":"2024-05-27T09:27:19.539980+0000","last_peered":"2024-05-27T09:27:19.539980+0000","last_clean":"2024-05-27T09:27:19.539980+0000","last_became_active":"2024-05-27T09:27:18.549035+0000","last_became_peered":"2024-05-27T09:27:18.549035+0000","last_unstale":"2024-05-27T09:27:19.539980+0000","last_undegraded":"2024-05-27T09:27:19.539980+0000","last_fullsized":"2024-05-27T09:27:19.539980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:17.895211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544344+0000","last_change":"2024-05-27T09:27:17.568143+0000","last_active":"2024-05-27T09:27:19.544344+0000","last_peered":"2024-05-27T09:27:19.544344+0000","last_clean":"2024-05-27T09:27:19.544344+0000","last_became_active":"2024-05-27T09:27:17.567235+0000","last_became_peered":"2024-05-27T09:27:17.567235+0000","last_unstale":"2024-05-27T09:27:19.544344+0000","last_undegraded":"2024-05-27T09:27:19.544344+0000","last_fullsized":"2024-05-27T09:27:19.544344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.284270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836871+0000","last_change":"2024-05-27T09:27:18.563270+0000","last_active":"2024-05-27T09:27:20.836871+0000","last_peered":"2024-05-27T09:27:20.836871+0000","last_clean":"2024-05-27T09:27:20.836871+0000","last_became_active":"2024-05-27T09:27:18.562919+0000","last_became_peered":"2024-05-27T09:27:18.562919+0000","last_unstale":"2024-05-27T09:27:20.836871+0000","last_undegraded":"2024-05-27T09:27:20.836871+0000","last_fullsized":"2024-05-27T09:27:20.836871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:55:47.799529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836603+0000","last_change":"2024-05-27T09:27:17.551193+0000","last_active":"2024-05-27T09:27:20.836603+0000","last_peered":"2024-05-27T09:27:20.836603+0000","last_clean":"2024-05-27T09:27:20.836603+0000","last_became_active":"2024-05-27T09:27:17.550849+0000","last_became_peered":"2024-05-27T09:27:17.550849+0000","last_unstale":"2024-05-27T09:27:20.836603+0000","last_undegraded":"2024-05-27T09:27:20.836603+0000","last_fullsized":"2024-05-27T09:27:20.836603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:51.742078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836183+0000","last_change":"2024-05-27T09:27:18.579803+0000","last_active":"2024-05-27T09:27:20.836183+0000","last_peered":"2024-05-27T09:27:20.836183+0000","last_clean":"2024-05-27T09:27:20.836183+0000","last_became_active":"2024-05-27T09:27:18.579574+0000","last_became_peered":"2024-05-27T09:27:18.579574+0000","last_unstale":"2024-05-27T09:27:20.836183+0000","last_undegraded":"2024-05-27T09:27:20.836183+0000","last_fullsized":"2024-05-27T09:27:20.836183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:11:22.320778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836021+0000","last_change":"2024-05-27T09:27:17.549711+0000","last_active":"2024-05-27T09:27:20.836021+0000","last_peered":"2024-05-27T09:27:20.836021+0000","last_clean":"2024-05-27T09:27:20.836021+0000","last_became_active":"2024-05-27T09:27:17.549291+0000","last_became_peered":"2024-05-27T09:27:17.549291+0000","last_unstale":"2024-05-27T09:27:20.836021+0000","last_undegraded":"2024-05-27T09:27:20.836021+0000","last_fullsized":"2024-05-27T09:27:20.836021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:16:22.057803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65078,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":72675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":72675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":663043,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":72675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":72675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":72675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":663043,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":72675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T09:27:25.260 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T09:27:25.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.573+0000 7ff37b400640 1 Processor -- start 2024-05-27T09:27:25.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.573+0000 7ff37b400640 1 -- start start 2024-05-27T09:27:25.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.575+0000 7ff37b400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff3741593b0 0x7ff374142720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:25.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.575+0000 7ff37b400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff3740c2990 0x7ff3740c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:25.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.575+0000 7ff37b400640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3740739a0 con 0x7ff374073260 2024-05-27T09:27:25.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.575+0000 7ff37b400640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7ff374074d20 con 0x7ff3740c2990 2024-05-27T09:27:25.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.575+0000 7ff37b400640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7ff374073c80 con 0x7ff3741593b0 2024-05-27T09:27:25.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.576+0000 7ff379400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff3740c2990 0x7ff3740c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:25.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.576+0000 7ff379400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff3740c2990 0x7ff3740c2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35388/0 (socket says 172.21.15.114:35388) 2024-05-27T09:27:25.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.576+0000 7ff379400640 1 -- 172.21.15.114:0/1182765499 learned_addr learned my addr 172.21.15.114:0/1182765499 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:25.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.576+0000 7ff378a00640 1 --2- 172.21.15.114:0/1182765499 >> v2:172.21.15.163:3300/0 conn(0x7ff3741593b0 0x7ff374142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:25.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.576+0000 7ff379400640 1 -- 172.21.15.114:0/1182765499 >> v2:172.21.15.163:3300/0 conn(0x7ff3741593b0 msgr2=0x7ff374142720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:25.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.576+0000 7ff379400640 1 --2- 172.21.15.114:0/1182765499 >> v2:172.21.15.163:3300/0 conn(0x7ff3741593b0 0x7ff374142720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:25.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.576+0000 7ff379400640 1 -- 172.21.15.114:0/1182765499 >> v1:172.21.15.163:6789/0 conn(0x7ff374073260 legacy=0x7ff37415a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:25.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.576+0000 7ff379400640 1 -- 172.21.15.114:0/1182765499 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff374073f60 con 0x7ff3740c2990 2024-05-27T09:27:25.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.576+0000 7ff373400640 1 -- 172.21.15.114:0/1182765499 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2916589050 0 0) 0x7ff3740739a0 con 0x7ff374073260 2024-05-27T09:27:25.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.577+0000 7ff379400640 1 --2- 172.21.15.114:0/1182765499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3740c2990 0x7ff3740c2da0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff3740cb530 tx=0x7ff36c02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abc21409f6753df server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:25.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.578+0000 7ff373400640 1 -- 172.21.15.114:0/1182765499 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c036070 con 0x7ff3740c2990 2024-05-27T09:27:25.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.578+0000 7ff373400640 1 -- 172.21.15.114:0/1182765499 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff36c031600 con 0x7ff3740c2990 2024-05-27T09:27:25.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.578+0000 7ff373400640 1 -- 172.21.15.114:0/1182765499 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c0319f0 con 0x7ff3740c2990 2024-05-27T09:27:25.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.578+0000 7ff37b400640 1 -- 172.21.15.114:0/1182765499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3740c2990 msgr2=0x7ff3740c2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:25.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.578+0000 7ff37b400640 1 --2- 172.21.15.114:0/1182765499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3740c2990 0x7ff3740c2da0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff3740cb530 tx=0x7ff36c02a7c0 comp rx=0 tx=0).stop 2024-05-27T09:27:25.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.578+0000 7ff37b400640 1 -- 172.21.15.114:0/1182765499 shutdown_connections 2024-05-27T09:27:25.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.578+0000 7ff37b400640 1 --2- 172.21.15.114:0/1182765499 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3740c2990 0x7ff3740c2da0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:25.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.578+0000 7ff37b400640 1 --2- 172.21.15.114:0/1182765499 >> v2:172.21.15.163:3300/0 conn(0x7ff3741593b0 0x7ff374142720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:25.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.578+0000 7ff37b400640 1 -- 172.21.15.114:0/1182765499 >> 172.21.15.114:0/1182765499 conn(0x7ff37406f6e0 msgr2=0x7ff374070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:25.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.579+0000 7ff37b400640 1 -- 172.21.15.114:0/1182765499 shutdown_connections 2024-05-27T09:27:25.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.579+0000 7ff37b400640 1 -- 172.21.15.114:0/1182765499 wait complete. 2024-05-27T09:27:25.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.581+0000 7ff37b400640 1 Processor -- start 2024-05-27T09:27:25.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.581+0000 7ff37b400640 1 -- start start 2024-05-27T09:27:25.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.582+0000 7ff37b400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff374073260 0x7ff374154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:25.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.582+0000 7ff37b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3740c2990 0x7ff3740c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:25.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.582+0000 7ff37b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3741593b0 0x7ff3740b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:25.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.583+0000 7ff37b400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff374074800 con 0x7ff3740c2990 2024-05-27T09:27:25.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.583+0000 7ff37b400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff374074520 con 0x7ff3741593b0 2024-05-27T09:27:25.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.583+0000 7ff37b400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7ff374073670 con 0x7ff374073260 2024-05-27T09:27:25.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.583+0000 7ff373e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3740c2990 0x7ff3740c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:25.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.583+0000 7ff373e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3740c2990 0x7ff3740c9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35400/0 (socket says 172.21.15.114:35400) 2024-05-27T09:27:25.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.583+0000 7ff373e00640 1 -- 172.21.15.114:0/1377408267 learned_addr learned my addr 172.21.15.114:0/1377408267 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:25.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.583+0000 7ff378a00640 1 --2- 172.21.15.114:0/1377408267 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff374073260 0x7ff374154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:25.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.583+0000 7ff379400640 1 --2- 172.21.15.114:0/1377408267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3741593b0 0x7ff3740b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:25.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.583+0000 7ff373e00640 1 -- 172.21.15.114:0/1377408267 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff374073260 msgr2=0x7ff374154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:25.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.583+0000 7ff373e00640 1 --2- 172.21.15.114:0/1377408267 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff374073260 0x7ff374154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:25.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.583+0000 7ff373e00640 1 -- 172.21.15.114:0/1377408267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3741593b0 msgr2=0x7ff3740b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:25.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.583+0000 7ff373e00640 1 --2- 172.21.15.114:0/1377408267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3741593b0 0x7ff3740b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:25.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.583+0000 7ff373e00640 1 -- 172.21.15.114:0/1377408267 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff374141cb0 con 0x7ff3740c2990 2024-05-27T09:27:25.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.583+0000 7ff373e00640 1 --2- 172.21.15.114:0/1377408267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3740c2990 0x7ff3740c9f40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff36000c3c0 tx=0x7ff36000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:25.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.584+0000 7ff370c00640 1 -- 172.21.15.114:0/1377408267 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360017070 con 0x7ff3740c2990 2024-05-27T09:27:25.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.584+0000 7ff370c00640 1 -- 172.21.15.114:0/1377408267 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3600035f0 con 0x7ff3740c2990 2024-05-27T09:27:25.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.584+0000 7ff37b400640 1 -- 172.21.15.114:0/1377408267 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3741f63a0 con 0x7ff3740c2990 2024-05-27T09:27:25.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.584+0000 7ff370c00640 1 -- 172.21.15.114:0/1377408267 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360003a20 con 0x7ff3740c2990 2024-05-27T09:27:25.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.584+0000 7ff37b400640 1 -- 172.21.15.114:0/1377408267 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3741f6960 con 0x7ff3740c2990 2024-05-27T09:27:25.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.585+0000 7ff37b400640 1 -- 172.21.15.114:0/1377408267 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff37406a380 con 0x7ff3740c2990 2024-05-27T09:27:25.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.586+0000 7ff370c00640 1 -- 172.21.15.114:0/1377408267 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff36001a070 con 0x7ff3740c2990 2024-05-27T09:27:25.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.587+0000 7ff370c00640 1 --2- 172.21.15.114:0/1377408267 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff3540762d0 0x7ff3540787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:25.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.588+0000 7ff370c00640 1 -- 172.21.15.114:0/1377408267 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff360096980 con 0x7ff3740c2990 2024-05-27T09:27:25.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.588+0000 7ff378a00640 1 --2- 172.21.15.114:0/1377408267 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff3540762d0 0x7ff3540787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:25.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.589+0000 7ff378a00640 1 --2- 172.21.15.114:0/1377408267 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff3540762d0 0x7ff3540787e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff368009aa0 tx=0x7ff368009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:25.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.594+0000 7ff370c00640 1 -- 172.21.15.114:0/1377408267 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff36005fbf0 con 0x7ff3740c2990 2024-05-27T09:27:25.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.795+0000 7ff37b400640 1 -- 172.21.15.114:0/1377408267 --> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff374156cd0 con 0x7ff3540762d0 2024-05-27T09:27:25.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.802+0000 7ff370c00640 1 -- 172.21.15.114:0/1377408267 <== mgr.4108 v2:172.21.15.163:6832/1122018204 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251915 (secure 0 0 0) 0x7ff374156cd0 con 0x7ff3540762d0 2024-05-27T09:27:25.807 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:25.808 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-05-27T09:27:25.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.808+0000 7ff37b400640 1 -- 172.21.15.114:0/1377408267 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff3540762d0 msgr2=0x7ff3540787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:25.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.808+0000 7ff37b400640 1 --2- 172.21.15.114:0/1377408267 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff3540762d0 0x7ff3540787e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff368009aa0 tx=0x7ff368009190 comp rx=0 tx=0).stop 2024-05-27T09:27:25.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.808+0000 7ff37b400640 1 -- 172.21.15.114:0/1377408267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3740c2990 msgr2=0x7ff3740c9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:25.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.808+0000 7ff37b400640 1 --2- 172.21.15.114:0/1377408267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3740c2990 0x7ff3740c9f40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff36000c3c0 tx=0x7ff36000c880 comp rx=0 tx=0).stop 2024-05-27T09:27:25.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.808+0000 7ff37b400640 1 -- 172.21.15.114:0/1377408267 shutdown_connections 2024-05-27T09:27:25.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.808+0000 7ff37b400640 1 --2- 172.21.15.114:0/1377408267 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff3540762d0 0x7ff3540787e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:25.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.808+0000 7ff37b400640 1 --2- 172.21.15.114:0/1377408267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3741593b0 0x7ff3740b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:25.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.808+0000 7ff37b400640 1 --2- 172.21.15.114:0/1377408267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3740c2990 0x7ff3740c9f40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:25.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.808+0000 7ff37b400640 1 --2- 172.21.15.114:0/1377408267 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff374073260 0x7ff374154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:25.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.808+0000 7ff37b400640 1 -- 172.21.15.114:0/1377408267 >> 172.21.15.114:0/1377408267 conn(0x7ff37406f6e0 msgr2=0x7ff3740cce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:25.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.808+0000 7ff37b400640 1 -- 172.21.15.114:0/1377408267 shutdown_connections 2024-05-27T09:27:25.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:25.808+0000 7ff37b400640 1 -- 172.21.15.114:0/1377408267 wait complete. 2024-05-27T09:27:25.834 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-05-27T09:27:24.763558+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217932,"kb_used_data":1760,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756788,"statfs":{"total":773094113280,"available":772870950912,"internally_reserved":0,"allocated":1802240,"data_stored":1762136,"data_compressed":65078,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.233569"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544950+0000","last_change":"2024-05-27T09:27:18.585655+0000","last_active":"2024-05-27T09:27:19.544950+0000","last_peered":"2024-05-27T09:27:19.544950+0000","last_clean":"2024-05-27T09:27:19.544950+0000","last_became_active":"2024-05-27T09:27:18.585169+0000","last_became_peered":"2024-05-27T09:27:18.585169+0000","last_unstale":"2024-05-27T09:27:19.544950+0000","last_undegraded":"2024-05-27T09:27:19.544950+0000","last_fullsized":"2024-05-27T09:27:19.544950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:39:59.034873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835983+0000","last_change":"2024-05-27T09:27:18.566621+0000","last_active":"2024-05-27T09:27:20.835983+0000","last_peered":"2024-05-27T09:27:20.835983+0000","last_clean":"2024-05-27T09:27:20.835983+0000","last_became_active":"2024-05-27T09:27:18.566269+0000","last_became_peered":"2024-05-27T09:27:18.566269+0000","last_unstale":"2024-05-27T09:27:20.835983+0000","last_undegraded":"2024-05-27T09:27:20.835983+0000","last_fullsized":"2024-05-27T09:27:20.835983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:24:24.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835531+0000","last_change":"2024-05-27T09:27:18.560958+0000","last_active":"2024-05-27T09:27:20.835531+0000","last_peered":"2024-05-27T09:27:20.835531+0000","last_clean":"2024-05-27T09:27:20.835531+0000","last_became_active":"2024-05-27T09:27:18.560620+0000","last_became_peered":"2024-05-27T09:27:18.560620+0000","last_unstale":"2024-05-27T09:27:20.835531+0000","last_undegraded":"2024-05-27T09:27:20.835531+0000","last_fullsized":"2024-05-27T09:27:20.835531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:42:38.351682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542664+0000","last_change":"2024-05-27T09:27:18.569804+0000","last_active":"2024-05-27T09:27:19.542664+0000","last_peered":"2024-05-27T09:27:19.542664+0000","last_clean":"2024-05-27T09:27:19.542664+0000","last_became_active":"2024-05-27T09:27:18.569481+0000","last_became_peered":"2024-05-27T09:27:18.569481+0000","last_unstale":"2024-05-27T09:27:19.542664+0000","last_undegraded":"2024-05-27T09:27:19.542664+0000","last_fullsized":"2024-05-27T09:27:19.542664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:55:03.583659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543808+0000","last_change":"2024-05-27T09:27:18.568138+0000","last_active":"2024-05-27T09:27:19.543808+0000","last_peered":"2024-05-27T09:27:19.543808+0000","last_clean":"2024-05-27T09:27:19.543808+0000","last_became_active":"2024-05-27T09:27:18.567795+0000","last_became_peered":"2024-05-27T09:27:18.567795+0000","last_unstale":"2024-05-27T09:27:19.543808+0000","last_undegraded":"2024-05-27T09:27:19.543808+0000","last_fullsized":"2024-05-27T09:27:19.543808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:17:42.005593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.540719+0000","last_change":"2024-05-27T09:27:18.573984+0000","last_active":"2024-05-27T09:27:19.540719+0000","last_peered":"2024-05-27T09:27:19.540719+0000","last_clean":"2024-05-27T09:27:19.540719+0000","last_became_active":"2024-05-27T09:27:18.573576+0000","last_became_peered":"2024-05-27T09:27:18.573576+0000","last_unstale":"2024-05-27T09:27:19.540719+0000","last_undegraded":"2024-05-27T09:27:19.540719+0000","last_fullsized":"2024-05-27T09:27:19.540719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:06.224637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543010+0000","last_change":"2024-05-27T09:27:18.543122+0000","last_active":"2024-05-27T09:27:19.543010+0000","last_peered":"2024-05-27T09:27:19.543010+0000","last_clean":"2024-05-27T09:27:19.543010+0000","last_became_active":"2024-05-27T09:27:18.542593+0000","last_became_peered":"2024-05-27T09:27:18.542593+0000","last_unstale":"2024-05-27T09:27:19.543010+0000","last_undegraded":"2024-05-27T09:27:19.543010+0000","last_fullsized":"2024-05-27T09:27:19.543010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:28:19.579432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835946+0000","last_change":"2024-05-27T09:27:18.562292+0000","last_active":"2024-05-27T09:27:20.835946+0000","last_peered":"2024-05-27T09:27:20.835946+0000","last_clean":"2024-05-27T09:27:20.835946+0000","last_became_active":"2024-05-27T09:27:18.561788+0000","last_became_peered":"2024-05-27T09:27:18.561788+0000","last_unstale":"2024-05-27T09:27:20.835946+0000","last_undegraded":"2024-05-27T09:27:20.835946+0000","last_fullsized":"2024-05-27T09:27:20.835946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:32:39.043268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542996+0000","last_change":"2024-05-27T09:27:18.583770+0000","last_active":"2024-05-27T09:27:19.542996+0000","last_peered":"2024-05-27T09:27:19.542996+0000","last_clean":"2024-05-27T09:27:19.542996+0000","last_became_active":"2024-05-27T09:27:18.582041+0000","last_became_peered":"2024-05-27T09:27:18.582041+0000","last_unstale":"2024-05-27T09:27:19.542996+0000","last_undegraded":"2024-05-27T09:27:19.542996+0000","last_fullsized":"2024-05-27T09:27:19.542996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:19.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.545125+0000","last_change":"2024-05-27T09:27:18.574658+0000","last_active":"2024-05-27T09:27:19.545125+0000","last_peered":"2024-05-27T09:27:19.545125+0000","last_clean":"2024-05-27T09:27:19.545125+0000","last_became_active":"2024-05-27T09:27:18.574409+0000","last_became_peered":"2024-05-27T09:27:18.574409+0000","last_unstale":"2024-05-27T09:27:19.545125+0000","last_undegraded":"2024-05-27T09:27:19.545125+0000","last_fullsized":"2024-05-27T09:27:19.545125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:19.131343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544558+0000","last_change":"2024-05-27T09:27:18.579141+0000","last_active":"2024-05-27T09:27:19.544558+0000","last_peered":"2024-05-27T09:27:19.544558+0000","last_clean":"2024-05-27T09:27:19.544558+0000","last_became_active":"2024-05-27T09:27:18.578679+0000","last_became_peered":"2024-05-27T09:27:18.578679+0000","last_unstale":"2024-05-27T09:27:19.544558+0000","last_undegraded":"2024-05-27T09:27:19.544558+0000","last_fullsized":"2024-05-27T09:27:19.544558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:44.336453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544110+0000","last_change":"2024-05-27T09:27:18.547375+0000","last_active":"2024-05-27T09:27:19.544110+0000","last_peered":"2024-05-27T09:27:19.544110+0000","last_clean":"2024-05-27T09:27:19.544110+0000","last_became_active":"2024-05-27T09:27:18.544328+0000","last_became_peered":"2024-05-27T09:27:18.544328+0000","last_unstale":"2024-05-27T09:27:19.544110+0000","last_undegraded":"2024-05-27T09:27:19.544110+0000","last_fullsized":"2024-05-27T09:27:19.544110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:59:50.499894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836590+0000","last_change":"2024-05-27T09:27:18.561903+0000","last_active":"2024-05-27T09:27:20.836590+0000","last_peered":"2024-05-27T09:27:20.836590+0000","last_clean":"2024-05-27T09:27:20.836590+0000","last_became_active":"2024-05-27T09:27:18.561493+0000","last_became_peered":"2024-05-27T09:27:18.561493+0000","last_unstale":"2024-05-27T09:27:20.836590+0000","last_undegraded":"2024-05-27T09:27:20.836590+0000","last_fullsized":"2024-05-27T09:27:20.836590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:28:31.470269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836378+0000","last_change":"2024-05-27T09:27:18.563168+0000","last_active":"2024-05-27T09:27:20.836378+0000","last_peered":"2024-05-27T09:27:20.836378+0000","last_clean":"2024-05-27T09:27:20.836378+0000","last_became_active":"2024-05-27T09:27:18.562843+0000","last_became_peered":"2024-05-27T09:27:18.562843+0000","last_unstale":"2024-05-27T09:27:20.836378+0000","last_undegraded":"2024-05-27T09:27:20.836378+0000","last_fullsized":"2024-05-27T09:27:20.836378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:33:36.671764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835457+0000","last_change":"2024-05-27T09:27:18.566502+0000","last_active":"2024-05-27T09:27:20.835457+0000","last_peered":"2024-05-27T09:27:20.835457+0000","last_clean":"2024-05-27T09:27:20.835457+0000","last_became_active":"2024-05-27T09:27:18.566108+0000","last_became_peered":"2024-05-27T09:27:18.566108+0000","last_unstale":"2024-05-27T09:27:20.835457+0000","last_undegraded":"2024-05-27T09:27:20.835457+0000","last_fullsized":"2024-05-27T09:27:20.835457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:44:47.447245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836471+0000","last_change":"2024-05-27T09:27:18.582595+0000","last_active":"2024-05-27T09:27:20.836471+0000","last_peered":"2024-05-27T09:27:20.836471+0000","last_clean":"2024-05-27T09:27:20.836471+0000","last_became_active":"2024-05-27T09:27:18.581719+0000","last_became_peered":"2024-05-27T09:27:18.581719+0000","last_unstale":"2024-05-27T09:27:20.836471+0000","last_undegraded":"2024-05-27T09:27:20.836471+0000","last_fullsized":"2024-05-27T09:27:20.836471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:48:19.573654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837234+0000","last_change":"2024-05-27T09:27:18.577565+0000","last_active":"2024-05-27T09:27:20.837234+0000","last_peered":"2024-05-27T09:27:20.837234+0000","last_clean":"2024-05-27T09:27:20.837234+0000","last_became_active":"2024-05-27T09:27:18.577193+0000","last_became_peered":"2024-05-27T09:27:18.577193+0000","last_unstale":"2024-05-27T09:27:20.837234+0000","last_undegraded":"2024-05-27T09:27:20.837234+0000","last_fullsized":"2024-05-27T09:27:20.837234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:43:20.065873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835530+0000","last_change":"2024-05-27T09:27:18.571089+0000","last_active":"2024-05-27T09:27:20.835530+0000","last_peered":"2024-05-27T09:27:20.835530+0000","last_clean":"2024-05-27T09:27:20.835530+0000","last_became_active":"2024-05-27T09:27:18.570712+0000","last_became_peered":"2024-05-27T09:27:18.570712+0000","last_unstale":"2024-05-27T09:27:20.835530+0000","last_undegraded":"2024-05-27T09:27:20.835530+0000","last_fullsized":"2024-05-27T09:27:20.835530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:40:19.310581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837956+0000","last_change":"2024-05-27T09:27:18.572675+0000","last_active":"2024-05-27T09:27:20.837956+0000","last_peered":"2024-05-27T09:27:20.837956+0000","last_clean":"2024-05-27T09:27:20.837956+0000","last_became_active":"2024-05-27T09:27:18.564643+0000","last_became_peered":"2024-05-27T09:27:18.564643+0000","last_unstale":"2024-05-27T09:27:20.837956+0000","last_undegraded":"2024-05-27T09:27:20.837956+0000","last_fullsized":"2024-05-27T09:27:20.837956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:36:15.602241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544179+0000","last_change":"2024-05-27T09:27:18.544554+0000","last_active":"2024-05-27T09:27:19.544179+0000","last_peered":"2024-05-27T09:27:19.544179+0000","last_clean":"2024-05-27T09:27:19.544179+0000","last_became_active":"2024-05-27T09:27:18.543777+0000","last_became_peered":"2024-05-27T09:27:18.543777+0000","last_unstale":"2024-05-27T09:27:19.544179+0000","last_undegraded":"2024-05-27T09:27:19.544179+0000","last_fullsized":"2024-05-27T09:27:19.544179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:35:43.354270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835033+0000","last_change":"2024-05-27T09:27:18.571321+0000","last_active":"2024-05-27T09:27:20.835033+0000","last_peered":"2024-05-27T09:27:20.835033+0000","last_clean":"2024-05-27T09:27:20.835033+0000","last_became_active":"2024-05-27T09:27:18.570948+0000","last_became_peered":"2024-05-27T09:27:18.570948+0000","last_unstale":"2024-05-27T09:27:20.835033+0000","last_undegraded":"2024-05-27T09:27:20.835033+0000","last_fullsized":"2024-05-27T09:27:20.835033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:28:31.470269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539490+0000","last_change":"2024-05-27T09:27:18.544282+0000","last_active":"2024-05-27T09:27:19.539490+0000","last_peered":"2024-05-27T09:27:19.539490+0000","last_clean":"2024-05-27T09:27:19.539490+0000","last_became_active":"2024-05-27T09:27:18.543921+0000","last_became_peered":"2024-05-27T09:27:18.543921+0000","last_unstale":"2024-05-27T09:27:19.539490+0000","last_undegraded":"2024-05-27T09:27:19.539490+0000","last_fullsized":"2024-05-27T09:27:19.539490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:27.225150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835732+0000","last_change":"2024-05-27T09:27:18.575415+0000","last_active":"2024-05-27T09:27:20.835732+0000","last_peered":"2024-05-27T09:27:20.835732+0000","last_clean":"2024-05-27T09:27:20.835732+0000","last_became_active":"2024-05-27T09:27:18.575115+0000","last_became_peered":"2024-05-27T09:27:18.575115+0000","last_unstale":"2024-05-27T09:27:20.835732+0000","last_undegraded":"2024-05-27T09:27:20.835732+0000","last_fullsized":"2024-05-27T09:27:20.835732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:50:48.091125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836496+0000","last_change":"2024-05-27T09:27:18.564422+0000","last_active":"2024-05-27T09:27:20.836496+0000","last_peered":"2024-05-27T09:27:20.836496+0000","last_clean":"2024-05-27T09:27:20.836496+0000","last_became_active":"2024-05-27T09:27:18.563991+0000","last_became_peered":"2024-05-27T09:27:18.563991+0000","last_unstale":"2024-05-27T09:27:20.836496+0000","last_undegraded":"2024-05-27T09:27:20.836496+0000","last_fullsized":"2024-05-27T09:27:20.836496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:19.131343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836032+0000","last_change":"2024-05-27T09:27:18.563557+0000","last_active":"2024-05-27T09:27:20.836032+0000","last_peered":"2024-05-27T09:27:20.836032+0000","last_clean":"2024-05-27T09:27:20.836032+0000","last_became_active":"2024-05-27T09:27:18.562803+0000","last_became_peered":"2024-05-27T09:27:18.562803+0000","last_unstale":"2024-05-27T09:27:20.836032+0000","last_undegraded":"2024-05-27T09:27:20.836032+0000","last_fullsized":"2024-05-27T09:27:20.836032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:47:54.271208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542223+0000","last_change":"2024-05-27T09:27:18.590430+0000","last_active":"2024-05-27T09:27:19.542223+0000","last_peered":"2024-05-27T09:27:19.542223+0000","last_clean":"2024-05-27T09:27:19.542223+0000","last_became_active":"2024-05-27T09:27:18.590083+0000","last_became_peered":"2024-05-27T09:27:18.590083+0000","last_unstale":"2024-05-27T09:27:19.542223+0000","last_undegraded":"2024-05-27T09:27:19.542223+0000","last_fullsized":"2024-05-27T09:27:19.542223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:03.762818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837167+0000","last_change":"2024-05-27T09:27:18.558248+0000","last_active":"2024-05-27T09:27:20.837167+0000","last_peered":"2024-05-27T09:27:20.837167+0000","last_clean":"2024-05-27T09:27:20.837167+0000","last_became_active":"2024-05-27T09:27:18.557953+0000","last_became_peered":"2024-05-27T09:27:18.557953+0000","last_unstale":"2024-05-27T09:27:20.837167+0000","last_undegraded":"2024-05-27T09:27:20.837167+0000","last_fullsized":"2024-05-27T09:27:20.837167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:44:47.447245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836514+0000","last_change":"2024-05-27T09:27:18.563213+0000","last_active":"2024-05-27T09:27:20.836514+0000","last_peered":"2024-05-27T09:27:20.836514+0000","last_clean":"2024-05-27T09:27:20.836514+0000","last_became_active":"2024-05-27T09:27:18.562839+0000","last_became_peered":"2024-05-27T09:27:18.562839+0000","last_unstale":"2024-05-27T09:27:20.836514+0000","last_undegraded":"2024-05-27T09:27:20.836514+0000","last_fullsized":"2024-05-27T09:27:20.836514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:43.851698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836053+0000","last_change":"2024-05-27T09:27:18.562237+0000","last_active":"2024-05-27T09:27:20.836053+0000","last_peered":"2024-05-27T09:27:20.836053+0000","last_clean":"2024-05-27T09:27:20.836053+0000","last_became_active":"2024-05-27T09:27:18.561475+0000","last_became_peered":"2024-05-27T09:27:18.561475+0000","last_unstale":"2024-05-27T09:27:20.836053+0000","last_undegraded":"2024-05-27T09:27:20.836053+0000","last_fullsized":"2024-05-27T09:27:20.836053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:39:58.110527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837611+0000","last_change":"2024-05-27T09:27:18.563469+0000","last_active":"2024-05-27T09:27:20.837611+0000","last_peered":"2024-05-27T09:27:20.837611+0000","last_clean":"2024-05-27T09:27:20.837611+0000","last_became_active":"2024-05-27T09:27:18.563103+0000","last_became_peered":"2024-05-27T09:27:18.563103+0000","last_unstale":"2024-05-27T09:27:20.837611+0000","last_undegraded":"2024-05-27T09:27:20.837611+0000","last_fullsized":"2024-05-27T09:27:20.837611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:53:57.281169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.834984+0000","last_change":"2024-05-27T09:27:18.560551+0000","last_active":"2024-05-27T09:27:20.834984+0000","last_peered":"2024-05-27T09:27:20.834984+0000","last_clean":"2024-05-27T09:27:20.834984+0000","last_became_active":"2024-05-27T09:27:18.560244+0000","last_became_peered":"2024-05-27T09:27:18.560244+0000","last_unstale":"2024-05-27T09:27:20.834984+0000","last_undegraded":"2024-05-27T09:27:20.834984+0000","last_fullsized":"2024-05-27T09:27:20.834984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:00.131180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.538536+0000","last_change":"2024-05-27T09:27:18.568200+0000","last_active":"2024-05-27T09:27:19.538536+0000","last_peered":"2024-05-27T09:27:19.538536+0000","last_clean":"2024-05-27T09:27:19.538536+0000","last_became_active":"2024-05-27T09:27:18.567845+0000","last_became_peered":"2024-05-27T09:27:18.567845+0000","last_unstale":"2024-05-27T09:27:19.538536+0000","last_undegraded":"2024-05-27T09:27:19.538536+0000","last_fullsized":"2024-05-27T09:27:19.538536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:54.726617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.540019+0000","last_change":"2024-05-27T09:27:18.560521+0000","last_active":"2024-05-27T09:27:19.540019+0000","last_peered":"2024-05-27T09:27:19.540019+0000","last_clean":"2024-05-27T09:27:19.540019+0000","last_became_active":"2024-05-27T09:27:18.556156+0000","last_became_peered":"2024-05-27T09:27:18.556156+0000","last_unstale":"2024-05-27T09:27:19.540019+0000","last_undegraded":"2024-05-27T09:27:19.540019+0000","last_fullsized":"2024-05-27T09:27:19.540019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:03.762818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835268+0000","last_change":"2024-05-27T09:27:17.548038+0000","last_active":"2024-05-27T09:27:20.835268+0000","last_peered":"2024-05-27T09:27:20.835268+0000","last_clean":"2024-05-27T09:27:20.835268+0000","last_became_active":"2024-05-27T09:27:17.547653+0000","last_became_peered":"2024-05-27T09:27:17.547653+0000","last_unstale":"2024-05-27T09:27:20.835268+0000","last_undegraded":"2024-05-27T09:27:20.835268+0000","last_fullsized":"2024-05-27T09:27:20.835268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835265+0000","last_change":"2024-05-27T09:27:18.543048+0000","last_active":"2024-05-27T09:27:20.835265+0000","last_peered":"2024-05-27T09:27:20.835265+0000","last_clean":"2024-05-27T09:27:20.835265+0000","last_became_active":"2024-05-27T09:27:18.542715+0000","last_became_peered":"2024-05-27T09:27:18.542715+0000","last_unstale":"2024-05-27T09:27:20.835265+0000","last_undegraded":"2024-05-27T09:27:20.835265+0000","last_fullsized":"2024-05-27T09:27:20.835265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:46:49.444709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837211+0000","last_change":"2024-05-27T09:27:17.570014+0000","last_active":"2024-05-27T09:27:20.837211+0000","last_peered":"2024-05-27T09:27:20.837211+0000","last_clean":"2024-05-27T09:27:20.837211+0000","last_became_active":"2024-05-27T09:27:17.569673+0000","last_became_peered":"2024-05-27T09:27:17.569673+0000","last_unstale":"2024-05-27T09:27:20.837211+0000","last_undegraded":"2024-05-27T09:27:20.837211+0000","last_fullsized":"2024-05-27T09:27:20.837211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:26.210041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836496+0000","last_change":"2024-05-27T09:27:17.565489+0000","last_active":"2024-05-27T09:27:20.836496+0000","last_peered":"2024-05-27T09:27:20.836496+0000","last_clean":"2024-05-27T09:27:20.836496+0000","last_became_active":"2024-05-27T09:27:17.565137+0000","last_became_peered":"2024-05-27T09:27:17.565137+0000","last_unstale":"2024-05-27T09:27:20.836496+0000","last_undegraded":"2024-05-27T09:27:20.836496+0000","last_fullsized":"2024-05-27T09:27:20.836496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836835+0000","last_change":"2024-05-27T09:27:18.565294+0000","last_active":"2024-05-27T09:27:20.836835+0000","last_peered":"2024-05-27T09:27:20.836835+0000","last_clean":"2024-05-27T09:27:20.836835+0000","last_became_active":"2024-05-27T09:27:18.564115+0000","last_became_peered":"2024-05-27T09:27:18.564115+0000","last_unstale":"2024-05-27T09:27:20.836835+0000","last_undegraded":"2024-05-27T09:27:20.836835+0000","last_fullsized":"2024-05-27T09:27:20.836835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:57:20.488849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836662+0000","last_change":"2024-05-27T09:27:17.561204+0000","last_active":"2024-05-27T09:27:20.836662+0000","last_peered":"2024-05-27T09:27:20.836662+0000","last_clean":"2024-05-27T09:27:20.836662+0000","last_became_active":"2024-05-27T09:27:17.560149+0000","last_became_peered":"2024-05-27T09:27:17.560149+0000","last_unstale":"2024-05-27T09:27:20.836662+0000","last_undegraded":"2024-05-27T09:27:20.836662+0000","last_fullsized":"2024-05-27T09:27:20.836662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.284270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836343+0000","last_change":"2024-05-27T09:27:18.542204+0000","last_active":"2024-05-27T09:27:20.836343+0000","last_peered":"2024-05-27T09:27:20.836343+0000","last_clean":"2024-05-27T09:27:20.836343+0000","last_became_active":"2024-05-27T09:27:18.541946+0000","last_became_peered":"2024-05-27T09:27:18.541946+0000","last_unstale":"2024-05-27T09:27:20.836343+0000","last_undegraded":"2024-05-27T09:27:20.836343+0000","last_fullsized":"2024-05-27T09:27:20.836343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:50:48.091125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.541754+0000","last_change":"2024-05-27T09:27:17.561714+0000","last_active":"2024-05-27T09:27:19.541754+0000","last_peered":"2024-05-27T09:27:19.541754+0000","last_clean":"2024-05-27T09:27:19.541754+0000","last_became_active":"2024-05-27T09:27:17.561355+0000","last_became_peered":"2024-05-27T09:27:17.561355+0000","last_unstale":"2024-05-27T09:27:19.541754+0000","last_undegraded":"2024-05-27T09:27:19.541754+0000","last_fullsized":"2024-05-27T09:27:19.541754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:07:02.481451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542128+0000","last_change":"2024-05-27T09:27:18.588199+0000","last_active":"2024-05-27T09:27:19.542128+0000","last_peered":"2024-05-27T09:27:19.542128+0000","last_clean":"2024-05-27T09:27:19.542128+0000","last_became_active":"2024-05-27T09:27:18.587921+0000","last_became_peered":"2024-05-27T09:27:18.587921+0000","last_unstale":"2024-05-27T09:27:19.542128+0000","last_undegraded":"2024-05-27T09:27:19.542128+0000","last_fullsized":"2024-05-27T09:27:19.542128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:17.895211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544807+0000","last_change":"2024-05-27T09:27:17.565854+0000","last_active":"2024-05-27T09:27:19.544807+0000","last_peered":"2024-05-27T09:27:19.544807+0000","last_clean":"2024-05-27T09:27:19.544807+0000","last_became_active":"2024-05-27T09:27:17.565488+0000","last_became_peered":"2024-05-27T09:27:17.565488+0000","last_unstale":"2024-05-27T09:27:19.544807+0000","last_undegraded":"2024-05-27T09:27:19.544807+0000","last_fullsized":"2024-05-27T09:27:19.544807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:53.711508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836966+0000","last_change":"2024-05-27T09:27:18.585592+0000","last_active":"2024-05-27T09:27:20.836966+0000","last_peered":"2024-05-27T09:27:20.836966+0000","last_clean":"2024-05-27T09:27:20.836966+0000","last_became_active":"2024-05-27T09:27:18.585243+0000","last_became_peered":"2024-05-27T09:27:18.585243+0000","last_unstale":"2024-05-27T09:27:20.836966+0000","last_undegraded":"2024-05-27T09:27:20.836966+0000","last_fullsized":"2024-05-27T09:27:20.836966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:28.872653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835317+0000","last_change":"2024-05-27T09:27:17.548651+0000","last_active":"2024-05-27T09:27:20.835317+0000","last_peered":"2024-05-27T09:27:20.835317+0000","last_clean":"2024-05-27T09:27:20.835317+0000","last_became_active":"2024-05-27T09:27:17.548307+0000","last_became_peered":"2024-05-27T09:27:17.548307+0000","last_unstale":"2024-05-27T09:27:20.835317+0000","last_undegraded":"2024-05-27T09:27:20.835317+0000","last_fullsized":"2024-05-27T09:27:20.835317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:48.905422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837273+0000","last_change":"2024-05-27T09:27:18.582895+0000","last_active":"2024-05-27T09:27:20.837273+0000","last_peered":"2024-05-27T09:27:20.837273+0000","last_clean":"2024-05-27T09:27:20.837273+0000","last_became_active":"2024-05-27T09:27:18.581742+0000","last_became_peered":"2024-05-27T09:27:18.581742+0000","last_unstale":"2024-05-27T09:27:20.837273+0000","last_undegraded":"2024-05-27T09:27:20.837273+0000","last_fullsized":"2024-05-27T09:27:20.837273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:23:26.888905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542758+0000","last_change":"2024-05-27T09:27:17.556419+0000","last_active":"2024-05-27T09:27:19.542758+0000","last_peered":"2024-05-27T09:27:19.542758+0000","last_clean":"2024-05-27T09:27:19.542758+0000","last_became_active":"2024-05-27T09:27:17.556036+0000","last_became_peered":"2024-05-27T09:27:17.556036+0000","last_unstale":"2024-05-27T09:27:19.542758+0000","last_undegraded":"2024-05-27T09:27:19.542758+0000","last_fullsized":"2024-05-27T09:27:19.542758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:48:20.718085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544281+0000","last_change":"2024-05-27T09:27:18.568187+0000","last_active":"2024-05-27T09:27:19.544281+0000","last_peered":"2024-05-27T09:27:19.544281+0000","last_clean":"2024-05-27T09:27:19.544281+0000","last_became_active":"2024-05-27T09:27:18.565500+0000","last_became_peered":"2024-05-27T09:27:18.565500+0000","last_unstale":"2024-05-27T09:27:19.544281+0000","last_undegraded":"2024-05-27T09:27:19.544281+0000","last_fullsized":"2024-05-27T09:27:19.544281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:16:08.688048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835570+0000","last_change":"2024-05-27T09:27:17.572168+0000","last_active":"2024-05-27T09:27:20.835570+0000","last_peered":"2024-05-27T09:27:20.835570+0000","last_clean":"2024-05-27T09:27:20.835570+0000","last_became_active":"2024-05-27T09:27:17.571709+0000","last_became_peered":"2024-05-27T09:27:17.571709+0000","last_unstale":"2024-05-27T09:27:20.835570+0000","last_undegraded":"2024-05-27T09:27:20.835570+0000","last_fullsized":"2024-05-27T09:27:20.835570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836574+0000","last_change":"2024-05-27T09:27:18.564930+0000","last_active":"2024-05-27T09:27:20.836574+0000","last_peered":"2024-05-27T09:27:20.836574+0000","last_clean":"2024-05-27T09:27:20.836574+0000","last_became_active":"2024-05-27T09:27:18.564589+0000","last_became_peered":"2024-05-27T09:27:18.564589+0000","last_unstale":"2024-05-27T09:27:20.836574+0000","last_undegraded":"2024-05-27T09:27:20.836574+0000","last_fullsized":"2024-05-27T09:27:20.836574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:21:00.644551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837541+0000","last_change":"2024-05-27T09:27:17.561344+0000","last_active":"2024-05-27T09:27:20.837541+0000","last_peered":"2024-05-27T09:27:20.837541+0000","last_clean":"2024-05-27T09:27:20.837541+0000","last_became_active":"2024-05-27T09:27:17.554860+0000","last_became_peered":"2024-05-27T09:27:17.554860+0000","last_unstale":"2024-05-27T09:27:20.837541+0000","last_undegraded":"2024-05-27T09:27:20.837541+0000","last_fullsized":"2024-05-27T09:27:20.837541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:16.880101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837236+0000","last_change":"2024-05-27T09:27:18.575317+0000","last_active":"2024-05-27T09:27:20.837236+0000","last_peered":"2024-05-27T09:27:20.837236+0000","last_clean":"2024-05-27T09:27:20.837236+0000","last_became_active":"2024-05-27T09:27:18.572024+0000","last_became_peered":"2024-05-27T09:27:18.572024+0000","last_unstale":"2024-05-27T09:27:20.837236+0000","last_undegraded":"2024-05-27T09:27:20.837236+0000","last_fullsized":"2024-05-27T09:27:20.837236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:23:35.418697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543181+0000","last_change":"2024-05-27T09:27:17.575859+0000","last_active":"2024-05-27T09:27:19.543181+0000","last_peered":"2024-05-27T09:27:19.543181+0000","last_clean":"2024-05-27T09:27:19.543181+0000","last_became_active":"2024-05-27T09:27:17.574695+0000","last_became_peered":"2024-05-27T09:27:17.574695+0000","last_unstale":"2024-05-27T09:27:19.543181+0000","last_undegraded":"2024-05-27T09:27:19.543181+0000","last_fullsized":"2024-05-27T09:27:19.543181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:24:19.379819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539060+0000","last_change":"2024-05-27T09:27:18.555137+0000","last_active":"2024-05-27T09:27:19.539060+0000","last_peered":"2024-05-27T09:27:19.539060+0000","last_clean":"2024-05-27T09:27:19.539060+0000","last_became_active":"2024-05-27T09:27:18.554932+0000","last_became_peered":"2024-05-27T09:27:18.554932+0000","last_unstale":"2024-05-27T09:27:19.539060+0000","last_undegraded":"2024-05-27T09:27:19.539060+0000","last_fullsized":"2024-05-27T09:27:19.539060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:06.224637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835234+0000","last_change":"2024-05-27T09:27:17.547895+0000","last_active":"2024-05-27T09:27:20.835234+0000","last_peered":"2024-05-27T09:27:20.835234+0000","last_clean":"2024-05-27T09:27:20.835234+0000","last_became_active":"2024-05-27T09:27:17.547523+0000","last_became_peered":"2024-05-27T09:27:17.547523+0000","last_unstale":"2024-05-27T09:27:20.835234+0000","last_undegraded":"2024-05-27T09:27:20.835234+0000","last_fullsized":"2024-05-27T09:27:20.835234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836706+0000","last_change":"2024-05-27T09:27:18.584284+0000","last_active":"2024-05-27T09:27:20.836706+0000","last_peered":"2024-05-27T09:27:20.836706+0000","last_clean":"2024-05-27T09:27:20.836706+0000","last_became_active":"2024-05-27T09:27:18.582360+0000","last_became_peered":"2024-05-27T09:27:18.582360+0000","last_unstale":"2024-05-27T09:27:20.836706+0000","last_undegraded":"2024-05-27T09:27:20.836706+0000","last_fullsized":"2024-05-27T09:27:20.836706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:32.535070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.545248+0000","last_change":"2024-05-27T09:27:17.571481+0000","last_active":"2024-05-27T09:27:19.545248+0000","last_peered":"2024-05-27T09:27:19.545248+0000","last_clean":"2024-05-27T09:27:19.545248+0000","last_became_active":"2024-05-27T09:27:17.571146+0000","last_became_peered":"2024-05-27T09:27:17.571146+0000","last_unstale":"2024-05-27T09:27:19.545248+0000","last_undegraded":"2024-05-27T09:27:19.545248+0000","last_fullsized":"2024-05-27T09:27:19.545248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:18.038221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542481+0000","last_change":"2024-05-27T09:27:18.580425+0000","last_active":"2024-05-27T09:27:19.542481+0000","last_peered":"2024-05-27T09:27:19.542481+0000","last_clean":"2024-05-27T09:27:19.542481+0000","last_became_active":"2024-05-27T09:27:18.579440+0000","last_became_peered":"2024-05-27T09:27:18.579440+0000","last_unstale":"2024-05-27T09:27:19.542481+0000","last_undegraded":"2024-05-27T09:27:19.542481+0000","last_fullsized":"2024-05-27T09:27:19.542481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:19.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835526+0000","last_change":"2024-05-27T09:27:17.551295+0000","last_active":"2024-05-27T09:27:20.835526+0000","last_peered":"2024-05-27T09:27:20.835526+0000","last_clean":"2024-05-27T09:27:20.835526+0000","last_became_active":"2024-05-27T09:27:17.550861+0000","last_became_peered":"2024-05-27T09:27:17.550861+0000","last_unstale":"2024-05-27T09:27:20.835526+0000","last_undegraded":"2024-05-27T09:27:20.835526+0000","last_fullsized":"2024-05-27T09:27:20.835526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:13.955191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.545103+0000","last_change":"2024-05-27T09:27:18.547199+0000","last_active":"2024-05-27T09:27:19.545103+0000","last_peered":"2024-05-27T09:27:19.545103+0000","last_clean":"2024-05-27T09:27:19.545103+0000","last_became_active":"2024-05-27T09:27:18.543683+0000","last_became_peered":"2024-05-27T09:27:18.543683+0000","last_unstale":"2024-05-27T09:27:19.545103+0000","last_undegraded":"2024-05-27T09:27:19.545103+0000","last_fullsized":"2024-05-27T09:27:19.545103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:48:19.573654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835661+0000","last_change":"2024-05-27T09:27:17.559255+0000","last_active":"2024-05-27T09:27:20.835661+0000","last_peered":"2024-05-27T09:27:20.835661+0000","last_clean":"2024-05-27T09:27:20.835661+0000","last_became_active":"2024-05-27T09:27:17.558973+0000","last_became_peered":"2024-05-27T09:27:17.558973+0000","last_unstale":"2024-05-27T09:27:20.835661+0000","last_undegraded":"2024-05-27T09:27:20.835661+0000","last_fullsized":"2024-05-27T09:27:20.835661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544602+0000","last_change":"2024-05-27T09:27:18.557423+0000","last_active":"2024-05-27T09:27:19.544602+0000","last_peered":"2024-05-27T09:27:19.544602+0000","last_clean":"2024-05-27T09:27:19.544602+0000","last_became_active":"2024-05-27T09:27:18.557076+0000","last_became_peered":"2024-05-27T09:27:18.557076+0000","last_unstale":"2024-05-27T09:27:19.544602+0000","last_undegraded":"2024-05-27T09:27:19.544602+0000","last_fullsized":"2024-05-27T09:27:19.544602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:11:04.612363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836934+0000","last_change":"2024-05-27T09:27:17.573700+0000","last_active":"2024-05-27T09:27:20.836934+0000","last_peered":"2024-05-27T09:27:20.836934+0000","last_clean":"2024-05-27T09:27:20.836934+0000","last_became_active":"2024-05-27T09:27:17.573484+0000","last_became_peered":"2024-05-27T09:27:17.573484+0000","last_unstale":"2024-05-27T09:27:20.836934+0000","last_undegraded":"2024-05-27T09:27:20.836934+0000","last_fullsized":"2024-05-27T09:27:20.836934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:48.905422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837609+0000","last_change":"2024-05-27T09:27:18.582777+0000","last_active":"2024-05-27T09:27:20.837609+0000","last_peered":"2024-05-27T09:27:20.837609+0000","last_clean":"2024-05-27T09:27:20.837609+0000","last_became_active":"2024-05-27T09:27:18.574959+0000","last_became_peered":"2024-05-27T09:27:18.574959+0000","last_unstale":"2024-05-27T09:27:20.837609+0000","last_undegraded":"2024-05-27T09:27:20.837609+0000","last_fullsized":"2024-05-27T09:27:20.837609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:40:41.206755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":110,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539210+0000","last_change":"2024-05-27T09:27:12.693423+0000","last_active":"2024-05-27T09:27:19.539210+0000","last_peered":"2024-05-27T09:27:19.539210+0000","last_clean":"2024-05-27T09:27:19.539210+0000","last_became_active":"2024-05-27T09:27:12.690301+0000","last_became_peered":"2024-05-27T09:27:12.690301+0000","last_unstale":"2024-05-27T09:27:19.539210+0000","last_undegraded":"2024-05-27T09:27:19.539210+0000","last_fullsized":"2024-05-27T09:27:19.539210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:11.463226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:11.463226+0000","last_clean_scrub_stamp":"2024-05-27T09:27:11.463226+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:01.810071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837730+0000","last_change":"2024-05-27T09:27:17.567670+0000","last_active":"2024-05-27T09:27:20.837730+0000","last_peered":"2024-05-27T09:27:20.837730+0000","last_clean":"2024-05-27T09:27:20.837730+0000","last_became_active":"2024-05-27T09:27:17.566292+0000","last_became_peered":"2024-05-27T09:27:17.566292+0000","last_unstale":"2024-05-27T09:27:20.837730+0000","last_undegraded":"2024-05-27T09:27:20.837730+0000","last_fullsized":"2024-05-27T09:27:20.837730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:18.038221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836742+0000","last_change":"2024-05-27T09:27:18.571124+0000","last_active":"2024-05-27T09:27:20.836742+0000","last_peered":"2024-05-27T09:27:20.836742+0000","last_clean":"2024-05-27T09:27:20.836742+0000","last_became_active":"2024-05-27T09:27:18.570782+0000","last_became_peered":"2024-05-27T09:27:18.570782+0000","last_unstale":"2024-05-27T09:27:20.836742+0000","last_undegraded":"2024-05-27T09:27:20.836742+0000","last_fullsized":"2024-05-27T09:27:20.836742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:39:58.110527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836667+0000","last_change":"2024-05-27T09:27:17.563259+0000","last_active":"2024-05-27T09:27:20.836667+0000","last_peered":"2024-05-27T09:27:20.836667+0000","last_clean":"2024-05-27T09:27:20.836667+0000","last_became_active":"2024-05-27T09:27:17.563010+0000","last_became_peered":"2024-05-27T09:27:17.563010+0000","last_unstale":"2024-05-27T09:27:20.836667+0000","last_undegraded":"2024-05-27T09:27:20.836667+0000","last_fullsized":"2024-05-27T09:27:20.836667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:53.711508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836371+0000","last_change":"2024-05-27T09:27:18.574466+0000","last_active":"2024-05-27T09:27:20.836371+0000","last_peered":"2024-05-27T09:27:20.836371+0000","last_clean":"2024-05-27T09:27:20.836371+0000","last_became_active":"2024-05-27T09:27:18.574090+0000","last_became_peered":"2024-05-27T09:27:18.574090+0000","last_unstale":"2024-05-27T09:27:20.836371+0000","last_undegraded":"2024-05-27T09:27:20.836371+0000","last_fullsized":"2024-05-27T09:27:20.836371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:41:22.515484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835113+0000","last_change":"2024-05-27T09:27:17.551310+0000","last_active":"2024-05-27T09:27:20.835113+0000","last_peered":"2024-05-27T09:27:20.835113+0000","last_clean":"2024-05-27T09:27:20.835113+0000","last_became_active":"2024-05-27T09:27:17.550982+0000","last_became_peered":"2024-05-27T09:27:17.550982+0000","last_unstale":"2024-05-27T09:27:20.835113+0000","last_undegraded":"2024-05-27T09:27:20.835113+0000","last_fullsized":"2024-05-27T09:27:20.835113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836574+0000","last_change":"2024-05-27T09:27:18.559416+0000","last_active":"2024-05-27T09:27:20.836574+0000","last_peered":"2024-05-27T09:27:20.836574+0000","last_clean":"2024-05-27T09:27:20.836574+0000","last_became_active":"2024-05-27T09:27:18.559017+0000","last_became_peered":"2024-05-27T09:27:18.559017+0000","last_unstale":"2024-05-27T09:27:20.836574+0000","last_undegraded":"2024-05-27T09:27:20.836574+0000","last_fullsized":"2024-05-27T09:27:20.836574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:33:48.400306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837680+0000","last_change":"2024-05-27T09:27:17.563947+0000","last_active":"2024-05-27T09:27:20.837680+0000","last_peered":"2024-05-27T09:27:20.837680+0000","last_clean":"2024-05-27T09:27:20.837680+0000","last_became_active":"2024-05-27T09:27:17.563640+0000","last_became_peered":"2024-05-27T09:27:17.563640+0000","last_unstale":"2024-05-27T09:27:20.837680+0000","last_undegraded":"2024-05-27T09:27:20.837680+0000","last_fullsized":"2024-05-27T09:27:20.837680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:02.747708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543615+0000","last_change":"2024-05-27T09:27:18.575835+0000","last_active":"2024-05-27T09:27:19.543615+0000","last_peered":"2024-05-27T09:27:19.543615+0000","last_clean":"2024-05-27T09:27:19.543615+0000","last_became_active":"2024-05-27T09:27:18.574710+0000","last_became_peered":"2024-05-27T09:27:18.574710+0000","last_unstale":"2024-05-27T09:27:19.543615+0000","last_undegraded":"2024-05-27T09:27:19.543615+0000","last_fullsized":"2024-05-27T09:27:19.543615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:16:00.934068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835781+0000","last_change":"2024-05-27T09:27:17.550749+0000","last_active":"2024-05-27T09:27:20.835781+0000","last_peered":"2024-05-27T09:27:20.835781+0000","last_clean":"2024-05-27T09:27:20.835781+0000","last_became_active":"2024-05-27T09:27:17.550378+0000","last_became_peered":"2024-05-27T09:27:17.550378+0000","last_unstale":"2024-05-27T09:27:20.835781+0000","last_undegraded":"2024-05-27T09:27:20.835781+0000","last_fullsized":"2024-05-27T09:27:20.835781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:20.167629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.541045+0000","last_change":"2024-05-27T09:27:18.567233+0000","last_active":"2024-05-27T09:27:19.541045+0000","last_peered":"2024-05-27T09:27:19.541045+0000","last_clean":"2024-05-27T09:27:19.541045+0000","last_became_active":"2024-05-27T09:27:18.566326+0000","last_became_peered":"2024-05-27T09:27:18.566326+0000","last_unstale":"2024-05-27T09:27:19.541045+0000","last_undegraded":"2024-05-27T09:27:19.541045+0000","last_fullsized":"2024-05-27T09:27:19.541045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:27.225150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837758+0000","last_change":"2024-05-27T09:27:17.557912+0000","last_active":"2024-05-27T09:27:20.837758+0000","last_peered":"2024-05-27T09:27:20.837758+0000","last_clean":"2024-05-27T09:27:20.837758+0000","last_became_active":"2024-05-27T09:27:17.557216+0000","last_became_peered":"2024-05-27T09:27:17.557216+0000","last_unstale":"2024-05-27T09:27:20.837758+0000","last_undegraded":"2024-05-27T09:27:20.837758+0000","last_fullsized":"2024-05-27T09:27:20.837758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:21:22.020724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836847+0000","last_change":"2024-05-27T09:27:18.563649+0000","last_active":"2024-05-27T09:27:20.836847+0000","last_peered":"2024-05-27T09:27:20.836847+0000","last_clean":"2024-05-27T09:27:20.836847+0000","last_became_active":"2024-05-27T09:27:18.562565+0000","last_became_peered":"2024-05-27T09:27:18.562565+0000","last_unstale":"2024-05-27T09:27:20.836847+0000","last_undegraded":"2024-05-27T09:27:20.836847+0000","last_fullsized":"2024-05-27T09:27:20.836847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:46:49.444709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835661+0000","last_change":"2024-05-27T09:27:17.571911+0000","last_active":"2024-05-27T09:27:20.835661+0000","last_peered":"2024-05-27T09:27:20.835661+0000","last_clean":"2024-05-27T09:27:20.835661+0000","last_became_active":"2024-05-27T09:27:17.571551+0000","last_became_peered":"2024-05-27T09:27:17.571551+0000","last_unstale":"2024-05-27T09:27:20.835661+0000","last_undegraded":"2024-05-27T09:27:20.835661+0000","last_fullsized":"2024-05-27T09:27:20.835661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:20.167629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835899+0000","last_change":"2024-05-27T09:27:18.558780+0000","last_active":"2024-05-27T09:27:20.835899+0000","last_peered":"2024-05-27T09:27:20.835899+0000","last_clean":"2024-05-27T09:27:20.835899+0000","last_became_active":"2024-05-27T09:27:18.558463+0000","last_became_peered":"2024-05-27T09:27:18.558463+0000","last_unstale":"2024-05-27T09:27:20.835899+0000","last_undegraded":"2024-05-27T09:27:20.835899+0000","last_fullsized":"2024-05-27T09:27:20.835899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:42:38.351682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837243+0000","last_change":"2024-05-27T09:27:17.567973+0000","last_active":"2024-05-27T09:27:20.837243+0000","last_peered":"2024-05-27T09:27:20.837243+0000","last_clean":"2024-05-27T09:27:20.837243+0000","last_became_active":"2024-05-27T09:27:17.567650+0000","last_became_peered":"2024-05-27T09:27:17.567650+0000","last_unstale":"2024-05-27T09:27:20.837243+0000","last_undegraded":"2024-05-27T09:27:20.837243+0000","last_fullsized":"2024-05-27T09:27:20.837243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:25:52.924185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835408+0000","last_change":"2024-05-27T09:27:18.554845+0000","last_active":"2024-05-27T09:27:20.835408+0000","last_peered":"2024-05-27T09:27:20.835408+0000","last_clean":"2024-05-27T09:27:20.835408+0000","last_became_active":"2024-05-27T09:27:18.554462+0000","last_became_peered":"2024-05-27T09:27:18.554462+0000","last_unstale":"2024-05-27T09:27:20.835408+0000","last_undegraded":"2024-05-27T09:27:20.835408+0000","last_fullsized":"2024-05-27T09:27:20.835408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:44:47.447245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.541867+0000","last_change":"2024-05-27T09:27:17.570300+0000","last_active":"2024-05-27T09:27:19.541867+0000","last_peered":"2024-05-27T09:27:19.541867+0000","last_clean":"2024-05-27T09:27:19.541867+0000","last_became_active":"2024-05-27T09:27:17.569685+0000","last_became_peered":"2024-05-27T09:27:17.569685+0000","last_unstale":"2024-05-27T09:27:19.541867+0000","last_undegraded":"2024-05-27T09:27:19.541867+0000","last_fullsized":"2024-05-27T09:27:19.541867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:26.210041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836430+0000","last_change":"2024-05-27T09:27:18.562774+0000","last_active":"2024-05-27T09:27:20.836430+0000","last_peered":"2024-05-27T09:27:20.836430+0000","last_clean":"2024-05-27T09:27:20.836430+0000","last_became_active":"2024-05-27T09:27:18.562455+0000","last_became_peered":"2024-05-27T09:27:18.562455+0000","last_unstale":"2024-05-27T09:27:20.836430+0000","last_undegraded":"2024-05-27T09:27:20.836430+0000","last_fullsized":"2024-05-27T09:27:20.836430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:39:59.034873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836403+0000","last_change":"2024-05-27T09:27:17.563172+0000","last_active":"2024-05-27T09:27:20.836403+0000","last_peered":"2024-05-27T09:27:20.836403+0000","last_clean":"2024-05-27T09:27:20.836403+0000","last_became_active":"2024-05-27T09:27:17.562941+0000","last_became_peered":"2024-05-27T09:27:17.562941+0000","last_unstale":"2024-05-27T09:27:20.836403+0000","last_undegraded":"2024-05-27T09:27:20.836403+0000","last_fullsized":"2024-05-27T09:27:20.836403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:51.742078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836477+0000","last_change":"2024-05-27T09:27:18.555617+0000","last_active":"2024-05-27T09:27:20.836477+0000","last_peered":"2024-05-27T09:27:20.836477+0000","last_clean":"2024-05-27T09:27:20.836477+0000","last_became_active":"2024-05-27T09:27:18.555280+0000","last_became_peered":"2024-05-27T09:27:18.555280+0000","last_unstale":"2024-05-27T09:27:20.836477+0000","last_undegraded":"2024-05-27T09:27:20.836477+0000","last_fullsized":"2024-05-27T09:27:20.836477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:32:39.043268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837070+0000","last_change":"2024-05-27T09:27:17.548266+0000","last_active":"2024-05-27T09:27:20.837070+0000","last_peered":"2024-05-27T09:27:20.837070+0000","last_clean":"2024-05-27T09:27:20.837070+0000","last_became_active":"2024-05-27T09:27:17.547938+0000","last_became_peered":"2024-05-27T09:27:17.547938+0000","last_unstale":"2024-05-27T09:27:20.837070+0000","last_undegraded":"2024-05-27T09:27:20.837070+0000","last_fullsized":"2024-05-27T09:27:20.837070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835677+0000","last_change":"2024-05-27T09:27:18.563586+0000","last_active":"2024-05-27T09:27:20.835677+0000","last_peered":"2024-05-27T09:27:20.835677+0000","last_clean":"2024-05-27T09:27:20.835677+0000","last_became_active":"2024-05-27T09:27:18.563227+0000","last_became_peered":"2024-05-27T09:27:18.563227+0000","last_unstale":"2024-05-27T09:27:20.835677+0000","last_undegraded":"2024-05-27T09:27:20.835677+0000","last_fullsized":"2024-05-27T09:27:20.835677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:53.222040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837432+0000","last_change":"2024-05-27T09:27:17.562200+0000","last_active":"2024-05-27T09:27:20.837432+0000","last_peered":"2024-05-27T09:27:20.837432+0000","last_clean":"2024-05-27T09:27:20.837432+0000","last_became_active":"2024-05-27T09:27:17.561857+0000","last_became_peered":"2024-05-27T09:27:17.561857+0000","last_unstale":"2024-05-27T09:27:20.837432+0000","last_undegraded":"2024-05-27T09:27:20.837432+0000","last_fullsized":"2024-05-27T09:27:20.837432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:05.209528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836936+0000","last_change":"2024-05-27T09:27:18.572676+0000","last_active":"2024-05-27T09:27:20.836936+0000","last_peered":"2024-05-27T09:27:20.836936+0000","last_clean":"2024-05-27T09:27:20.836936+0000","last_became_active":"2024-05-27T09:27:18.572339+0000","last_became_peered":"2024-05-27T09:27:18.572339+0000","last_unstale":"2024-05-27T09:27:20.836936+0000","last_undegraded":"2024-05-27T09:27:20.836936+0000","last_fullsized":"2024-05-27T09:27:20.836936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:09:23.795528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837085+0000","last_change":"2024-05-27T09:27:18.575394+0000","last_active":"2024-05-27T09:27:20.837085+0000","last_peered":"2024-05-27T09:27:20.837085+0000","last_clean":"2024-05-27T09:27:20.837085+0000","last_became_active":"2024-05-27T09:27:18.572307+0000","last_became_peered":"2024-05-27T09:27:18.572307+0000","last_unstale":"2024-05-27T09:27:20.837085+0000","last_undegraded":"2024-05-27T09:27:20.837085+0000","last_fullsized":"2024-05-27T09:27:20.837085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:29:05.116582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544332+0000","last_change":"2024-05-27T09:27:17.570634+0000","last_active":"2024-05-27T09:27:19.544332+0000","last_peered":"2024-05-27T09:27:19.544332+0000","last_clean":"2024-05-27T09:27:19.544332+0000","last_became_active":"2024-05-27T09:27:17.569866+0000","last_became_peered":"2024-05-27T09:27:17.569866+0000","last_unstale":"2024-05-27T09:27:19.544332+0000","last_undegraded":"2024-05-27T09:27:19.544332+0000","last_fullsized":"2024-05-27T09:27:19.544332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:21:38.268535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539980+0000","last_change":"2024-05-27T09:27:18.549339+0000","last_active":"2024-05-27T09:27:19.539980+0000","last_peered":"2024-05-27T09:27:19.539980+0000","last_clean":"2024-05-27T09:27:19.539980+0000","last_became_active":"2024-05-27T09:27:18.549035+0000","last_became_peered":"2024-05-27T09:27:18.549035+0000","last_unstale":"2024-05-27T09:27:19.539980+0000","last_undegraded":"2024-05-27T09:27:19.539980+0000","last_fullsized":"2024-05-27T09:27:19.539980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:17.895211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544344+0000","last_change":"2024-05-27T09:27:17.568143+0000","last_active":"2024-05-27T09:27:19.544344+0000","last_peered":"2024-05-27T09:27:19.544344+0000","last_clean":"2024-05-27T09:27:19.544344+0000","last_became_active":"2024-05-27T09:27:17.567235+0000","last_became_peered":"2024-05-27T09:27:17.567235+0000","last_unstale":"2024-05-27T09:27:19.544344+0000","last_undegraded":"2024-05-27T09:27:19.544344+0000","last_fullsized":"2024-05-27T09:27:19.544344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.284270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836871+0000","last_change":"2024-05-27T09:27:18.563270+0000","last_active":"2024-05-27T09:27:20.836871+0000","last_peered":"2024-05-27T09:27:20.836871+0000","last_clean":"2024-05-27T09:27:20.836871+0000","last_became_active":"2024-05-27T09:27:18.562919+0000","last_became_peered":"2024-05-27T09:27:18.562919+0000","last_unstale":"2024-05-27T09:27:20.836871+0000","last_undegraded":"2024-05-27T09:27:20.836871+0000","last_fullsized":"2024-05-27T09:27:20.836871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:55:47.799529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836603+0000","last_change":"2024-05-27T09:27:17.551193+0000","last_active":"2024-05-27T09:27:20.836603+0000","last_peered":"2024-05-27T09:27:20.836603+0000","last_clean":"2024-05-27T09:27:20.836603+0000","last_became_active":"2024-05-27T09:27:17.550849+0000","last_became_peered":"2024-05-27T09:27:17.550849+0000","last_unstale":"2024-05-27T09:27:20.836603+0000","last_undegraded":"2024-05-27T09:27:20.836603+0000","last_fullsized":"2024-05-27T09:27:20.836603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:51.742078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836183+0000","last_change":"2024-05-27T09:27:18.579803+0000","last_active":"2024-05-27T09:27:20.836183+0000","last_peered":"2024-05-27T09:27:20.836183+0000","last_clean":"2024-05-27T09:27:20.836183+0000","last_became_active":"2024-05-27T09:27:18.579574+0000","last_became_peered":"2024-05-27T09:27:18.579574+0000","last_unstale":"2024-05-27T09:27:20.836183+0000","last_undegraded":"2024-05-27T09:27:20.836183+0000","last_fullsized":"2024-05-27T09:27:20.836183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:11:22.320778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836021+0000","last_change":"2024-05-27T09:27:17.549711+0000","last_active":"2024-05-27T09:27:20.836021+0000","last_peered":"2024-05-27T09:27:20.836021+0000","last_clean":"2024-05-27T09:27:20.836021+0000","last_became_active":"2024-05-27T09:27:17.549291+0000","last_became_peered":"2024-05-27T09:27:17.549291+0000","last_unstale":"2024-05-27T09:27:20.836021+0000","last_undegraded":"2024-05-27T09:27:20.836021+0000","last_fullsized":"2024-05-27T09:27:20.836021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:16:22.057803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65078,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":72675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":72675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":663043,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":72675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":72675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":72675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":663043,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":72675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T09:27:25.838 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-27T09:27:25.838 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-27T09:27:25.838 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-27T09:27:25.838 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-27T09:27:25.839 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-27T09:27:26.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.136+0000 7f595ec00640 1 Processor -- start 2024-05-27T09:27:26.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.136+0000 7f595ec00640 1 -- start start 2024-05-27T09:27:26.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.138+0000 7f595ec00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5958074a80 0x7f595815cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:26.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.139+0000 7f595ec00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5958156880 0x7f595806edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:26.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.139+0000 7f595ec00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59580c8470 con 0x7f5958070060 2024-05-27T09:27:26.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.139+0000 7f595ec00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f59580c8220 con 0x7f5958156880 2024-05-27T09:27:26.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.139+0000 7f595ec00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f59580c8750 con 0x7f5958074a80 2024-05-27T09:27:26.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.139+0000 7f595ce00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5958156880 0x7f595806edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:26.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.139+0000 7f595ce00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5958156880 0x7f595806edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35414/0 (socket says 172.21.15.114:35414) 2024-05-27T09:27:26.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.139+0000 7f595ce00640 1 -- 172.21.15.114:0/1531688521 learned_addr learned my addr 172.21.15.114:0/1531688521 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:26.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.139+0000 7f595d800640 1 --2- 172.21.15.114:0/1531688521 >> v2:172.21.15.163:3300/0 conn(0x7f5958074a80 0x7f595815cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:26.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.139+0000 7f595ce00640 1 -- 172.21.15.114:0/1531688521 >> v2:172.21.15.163:3300/0 conn(0x7f5958074a80 msgr2=0x7f595815cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:26.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.139+0000 7f595ce00640 1 --2- 172.21.15.114:0/1531688521 >> v2:172.21.15.163:3300/0 conn(0x7f5958074a80 0x7f595815cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:26.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.139+0000 7f595ce00640 1 -- 172.21.15.114:0/1531688521 >> v1:172.21.15.114:6789/0 conn(0x7f5958070060 legacy=0x7f59580c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:26.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.139+0000 7f595ce00640 1 -- 172.21.15.114:0/1531688521 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59580c8dd0 con 0x7f5958156880 2024-05-27T09:27:26.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.139+0000 7f5957e00640 1 -- 172.21.15.114:0/1531688521 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2664991317 0 0) 0x7f59580c8470 con 0x7f5958070060 2024-05-27T09:27:26.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.140+0000 7f595ce00640 1 --2- 172.21.15.114:0/1531688521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5958156880 0x7f595806edf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f59580c83a0 tx=0x7f594402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a900be983bc9cf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:26.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.141+0000 7f5957e00640 1 -- 172.21.15.114:0/1531688521 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5944037070 con 0x7f5958156880 2024-05-27T09:27:26.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.141+0000 7f5957e00640 1 -- 172.21.15.114:0/1531688521 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5944032440 con 0x7f5958156880 2024-05-27T09:27:26.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.141+0000 7f5957e00640 1 -- 172.21.15.114:0/1531688521 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5944032810 con 0x7f5958156880 2024-05-27T09:27:26.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.142+0000 7f595ec00640 1 -- 172.21.15.114:0/1531688521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5958156880 msgr2=0x7f595806edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:26.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.142+0000 7f595ec00640 1 --2- 172.21.15.114:0/1531688521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5958156880 0x7f595806edf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f59580c83a0 tx=0x7f594402a270 comp rx=0 tx=0).stop 2024-05-27T09:27:26.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.142+0000 7f595ec00640 1 -- 172.21.15.114:0/1531688521 shutdown_connections 2024-05-27T09:27:26.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.142+0000 7f595ec00640 1 --2- 172.21.15.114:0/1531688521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5958156880 0x7f595806edf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:26.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.142+0000 7f595ec00640 1 --2- 172.21.15.114:0/1531688521 >> v2:172.21.15.163:3300/0 conn(0x7f5958074a80 0x7f595815cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:26.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.142+0000 7f595ec00640 1 -- 172.21.15.114:0/1531688521 >> 172.21.15.114:0/1531688521 conn(0x7f59580ca320 msgr2=0x7f59580ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:26.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.142+0000 7f595ec00640 1 -- 172.21.15.114:0/1531688521 shutdown_connections 2024-05-27T09:27:26.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.142+0000 7f595ec00640 1 -- 172.21.15.114:0/1531688521 wait complete. 2024-05-27T09:27:26.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.144+0000 7f595ec00640 1 Processor -- start 2024-05-27T09:27:26.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.144+0000 7f595ec00640 1 -- start start 2024-05-27T09:27:26.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.146+0000 7f595ec00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5958070060 0x7f5958155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:26.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.146+0000 7f595ec00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5958074a80 0x7f595814b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:26.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.146+0000 7f595ec00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5958156880 0x7f595814bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:26.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.146+0000 7f595ec00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f595815fa50 con 0x7f5958070060 2024-05-27T09:27:26.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.146+0000 7f595ec00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f595815c8e0 con 0x7f5958074a80 2024-05-27T09:27:26.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.146+0000 7f595ec00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f59581553b0 con 0x7f5958156880 2024-05-27T09:27:26.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.146+0000 7f595d800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5958070060 0x7f5958155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:26.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.146+0000 7f595d800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5958070060 0x7f5958155da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35424/0 (socket says 172.21.15.114:35424) 2024-05-27T09:27:26.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.146+0000 7f595d800640 1 -- 172.21.15.114:0/2814920778 learned_addr learned my addr 172.21.15.114:0/2814920778 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:26.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.146+0000 7f595ce00640 1 --2- 172.21.15.114:0/2814920778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5958074a80 0x7f595814b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:26.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.146+0000 7f595e200640 1 --2- 172.21.15.114:0/2814920778 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5958156880 0x7f595814bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:26.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.146+0000 7f595d800640 1 -- 172.21.15.114:0/2814920778 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5958156880 msgr2=0x7f595814bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:26.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.146+0000 7f595d800640 1 --2- 172.21.15.114:0/2814920778 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5958156880 0x7f595814bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:26.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.146+0000 7f595d800640 1 -- 172.21.15.114:0/2814920778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5958074a80 msgr2=0x7f595814b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:26.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.146+0000 7f595d800640 1 --2- 172.21.15.114:0/2814920778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5958074a80 0x7f595814b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:26.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.146+0000 7f595d800640 1 -- 172.21.15.114:0/2814920778 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59580cc450 con 0x7f5958070060 2024-05-27T09:27:26.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.146+0000 7f595e200640 1 --2- 172.21.15.114:0/2814920778 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5958156880 0x7f595814bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:27:26.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.147+0000 7f595d800640 1 --2- 172.21.15.114:0/2814920778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5958070060 0x7f5958155da0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f594c00e870 tx=0x7f594c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:26.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.147+0000 7f5955600640 1 -- 172.21.15.114:0/2814920778 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c017070 con 0x7f5958070060 2024-05-27T09:27:26.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.147+0000 7f5955600640 1 -- 172.21.15.114:0/2814920778 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f594c00c940 con 0x7f5958070060 2024-05-27T09:27:26.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.147+0000 7f5955600640 1 -- 172.21.15.114:0/2814920778 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c00cd30 con 0x7f5958070060 2024-05-27T09:27:26.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.147+0000 7f595ec00640 1 -- 172.21.15.114:0/2814920778 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5958159cb0 con 0x7f5958070060 2024-05-27T09:27:26.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.147+0000 7f595ec00640 1 -- 172.21.15.114:0/2814920778 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f595815a250 con 0x7f5958070060 2024-05-27T09:27:26.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.148+0000 7f595ec00640 1 -- 172.21.15.114:0/2814920778 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5958050860 con 0x7f5958070060 2024-05-27T09:27:26.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.149+0000 7f5955600640 1 -- 172.21.15.114:0/2814920778 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f594c01a070 con 0x7f5958070060 2024-05-27T09:27:26.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.151+0000 7f5955600640 1 --2- 172.21.15.114:0/2814920778 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5938076300 0x7f5938078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:26.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.151+0000 7f5955600640 1 -- 172.21.15.114:0/2814920778 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f594c097830 con 0x7f5958070060 2024-05-27T09:27:26.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.151+0000 7f595ce00640 1 --2- 172.21.15.114:0/2814920778 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5938076300 0x7f5938078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:26.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.152+0000 7f595ce00640 1 --2- 172.21.15.114:0/2814920778 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5938076300 0x7f5938078810 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5944034000 tx=0x7f594402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:26.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.156+0000 7f5955600640 1 -- 172.21.15.114:0/2814920778 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f594c060aa0 con 0x7f5958070060 2024-05-27T09:27:26.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.392+0000 7f595ec00640 1 -- 172.21.15.114:0/2814920778 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f59580c28d0 con 0x7f5958070060 2024-05-27T09:27:26.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.402+0000 7f5955600640 1 -- 172.21.15.114:0/2814920778 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+189527 (secure 0 0 0) 0x7f594c064da0 con 0x7f5958070060 2024-05-27T09:27:26.406 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:26.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.407+0000 7f595ec00640 1 -- 172.21.15.114:0/2814920778 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5938076300 msgr2=0x7f5938078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:26.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.407+0000 7f595ec00640 1 --2- 172.21.15.114:0/2814920778 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5938076300 0x7f5938078810 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5944034000 tx=0x7f594402e000 comp rx=0 tx=0).stop 2024-05-27T09:27:26.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.407+0000 7f595ec00640 1 -- 172.21.15.114:0/2814920778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5958070060 msgr2=0x7f5958155da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:26.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.407+0000 7f595ec00640 1 --2- 172.21.15.114:0/2814920778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5958070060 0x7f5958155da0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f594c00e870 tx=0x7f594c00ed30 comp rx=0 tx=0).stop 2024-05-27T09:27:26.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.407+0000 7f595ec00640 1 -- 172.21.15.114:0/2814920778 shutdown_connections 2024-05-27T09:27:26.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.407+0000 7f595ec00640 1 --2- 172.21.15.114:0/2814920778 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5938076300 0x7f5938078810 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:26.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.407+0000 7f595ec00640 1 --2- 172.21.15.114:0/2814920778 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5958156880 0x7f595814bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:26.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.407+0000 7f595ec00640 1 --2- 172.21.15.114:0/2814920778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5958074a80 0x7f595814b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:26.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.407+0000 7f595ec00640 1 --2- 172.21.15.114:0/2814920778 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5958070060 0x7f5958155da0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:26.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.407+0000 7f595ec00640 1 -- 172.21.15.114:0/2814920778 >> 172.21.15.114:0/2814920778 conn(0x7f59580ca320 msgr2=0x7f59580c4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:26.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.407+0000 7f595ec00640 1 -- 172.21.15.114:0/2814920778 shutdown_connections 2024-05-27T09:27:26.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.407+0000 7f595ec00640 1 -- 172.21.15.114:0/2814920778 wait complete. 2024-05-27T09:27:26.432 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":5,"flags":0,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1122018204},{"type":"v1","addr":"172.21.15.163:6833","nonce":1122018204}]},"active_addr":"172.21.15.163:6833/1122018204","active_change":"2024-05-27T09:27:04.750383+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4100,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":2998228580}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":314984028}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":2498531326}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":4223268802}]}]} 2024-05-27T09:27:26.436 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-27T09:27:26.436 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-27T09:27:26.436 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T09:27:26.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.707+0000 7f12b6800640 1 Processor -- start 2024-05-27T09:27:26.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.707+0000 7f12b6800640 1 -- start start 2024-05-27T09:27:26.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.709+0000 7f12b6800640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f12b01593b0 0x7f12b0142720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:26.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.709+0000 7f12b6800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f12b0073260 0x7f12b015a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:26.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.709+0000 7f12b6800640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12b00739a0 con 0x7f12b015ade0 2024-05-27T09:27:26.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.709+0000 7f12b6800640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f12b0074d20 con 0x7f12b0073260 2024-05-27T09:27:26.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.709+0000 7f12b6800640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f12b0073c80 con 0x7f12b01593b0 2024-05-27T09:27:26.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.709+0000 7f12aea00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f12b0073260 0x7f12b015a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:26.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.710+0000 7f12afe00640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f12b015ade0 0x7f12b00c2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:44612/0 (socket says 172.21.15.114:44612) 2024-05-27T09:27:26.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.710+0000 7f12afe00640 1 -- 172.21.15.114:0/4100085001 learned_addr learned my addr 172.21.15.114:0/4100085001 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:26.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.709+0000 7f12af400640 1 --2- 172.21.15.114:0/4100085001 >> v2:172.21.15.163:3300/0 conn(0x7f12b01593b0 0x7f12b0142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:26.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.710+0000 7f12aea00640 1 -- 172.21.15.114:0/4100085001 >> v2:172.21.15.163:3300/0 conn(0x7f12b01593b0 msgr2=0x7f12b0142720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:26.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.710+0000 7f12aea00640 1 --2- 172.21.15.114:0/4100085001 >> v2:172.21.15.163:3300/0 conn(0x7f12b01593b0 0x7f12b0142720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:26.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.710+0000 7f12aea00640 1 -- 172.21.15.114:0/4100085001 >> v1:172.21.15.163:6789/0 conn(0x7f12b015ade0 legacy=0x7f12b00c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:26.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.710+0000 7f12aea00640 1 -- 172.21.15.114:0/4100085001 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b0073f60 con 0x7f12b0073260 2024-05-27T09:27:26.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.711+0000 7f12aea00640 1 --2- 172.21.15.114:0/4100085001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b0073260 0x7f12b015a8a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f12b00cb530 tx=0x7f129c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d46acb16102d8e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:26.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.711+0000 7f12ae000640 1 -- 172.21.15.114:0/4100085001 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c037070 con 0x7f12b0073260 2024-05-27T09:27:26.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.711+0000 7f12ae000640 1 -- 172.21.15.114:0/4100085001 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f129c032440 con 0x7f12b0073260 2024-05-27T09:27:26.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.711+0000 7f12ae000640 1 -- 172.21.15.114:0/4100085001 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c032810 con 0x7f12b0073260 2024-05-27T09:27:26.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.712+0000 7f12b6800640 1 -- 172.21.15.114:0/4100085001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b0073260 msgr2=0x7f12b015a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:26.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.712+0000 7f12b6800640 1 --2- 172.21.15.114:0/4100085001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b0073260 0x7f12b015a8a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f12b00cb530 tx=0x7f129c02a230 comp rx=0 tx=0).stop 2024-05-27T09:27:26.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.712+0000 7f12b6800640 1 -- 172.21.15.114:0/4100085001 shutdown_connections 2024-05-27T09:27:26.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.712+0000 7f12b6800640 1 --2- 172.21.15.114:0/4100085001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b0073260 0x7f12b015a8a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:26.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.712+0000 7f12b6800640 1 --2- 172.21.15.114:0/4100085001 >> v2:172.21.15.163:3300/0 conn(0x7f12b01593b0 0x7f12b0142720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:26.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.712+0000 7f12b6800640 1 -- 172.21.15.114:0/4100085001 >> 172.21.15.114:0/4100085001 conn(0x7f12b006f6e0 msgr2=0x7f12b0070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:26.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.712+0000 7f12b6800640 1 -- 172.21.15.114:0/4100085001 shutdown_connections 2024-05-27T09:27:26.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.713+0000 7f12b6800640 1 -- 172.21.15.114:0/4100085001 wait complete. 2024-05-27T09:27:26.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.715+0000 7f12b6800640 1 Processor -- start 2024-05-27T09:27:26.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.715+0000 7f12b6800640 1 -- start start 2024-05-27T09:27:26.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.716+0000 7f12b6800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12b0073260 0x7f12b0154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:26.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.716+0000 7f12b6800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b01593b0 0x7f12b01f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:26.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.716+0000 7f12b6800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f12b015ade0 0x7f12b015c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:26.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.716+0000 7f12b6800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f12b0074800 con 0x7f12b01593b0 2024-05-27T09:27:26.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.716+0000 7f12b6800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f12b0074520 con 0x7f12b0073260 2024-05-27T09:27:26.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.716+0000 7f12b6800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f12b0073670 con 0x7f12b015ade0 2024-05-27T09:27:26.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.716+0000 7f12aea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b01593b0 0x7f12b01f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:26.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.717+0000 7f12aea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b01593b0 0x7f12b01f3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35444/0 (socket says 172.21.15.114:35444) 2024-05-27T09:27:26.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.717+0000 7f12aea00640 1 -- 172.21.15.114:0/4023414344 learned_addr learned my addr 172.21.15.114:0/4023414344 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:26.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.717+0000 7f12afe00640 1 --2- 172.21.15.114:0/4023414344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f12b015ade0 0x7f12b015c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:26.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.717+0000 7f12af400640 1 --2- 172.21.15.114:0/4023414344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12b0073260 0x7f12b0154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:26.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.717+0000 7f12aea00640 1 -- 172.21.15.114:0/4023414344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f12b015ade0 msgr2=0x7f12b015c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:26.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.717+0000 7f12aea00640 1 --2- 172.21.15.114:0/4023414344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f12b015ade0 0x7f12b015c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:26.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.717+0000 7f12aea00640 1 -- 172.21.15.114:0/4023414344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12b0073260 msgr2=0x7f12b0154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:26.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.717+0000 7f12aea00640 1 --2- 172.21.15.114:0/4023414344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12b0073260 0x7f12b0154b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:26.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.717+0000 7f12aea00640 1 -- 172.21.15.114:0/4023414344 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b01f2ea0 con 0x7f12b01593b0 2024-05-27T09:27:26.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.717+0000 7f12aea00640 1 --2- 172.21.15.114:0/4023414344 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b01593b0 0x7f12b01f3940 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f129c000c00 tx=0x7f129c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:26.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.718+0000 7f12a7400640 1 -- 172.21.15.114:0/4023414344 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c037040 con 0x7f12b01593b0 2024-05-27T09:27:26.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.718+0000 7f12a7400640 1 -- 172.21.15.114:0/4023414344 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f129c042070 con 0x7f12b01593b0 2024-05-27T09:27:26.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.718+0000 7f12a7400640 1 -- 172.21.15.114:0/4023414344 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c030960 con 0x7f12b01593b0 2024-05-27T09:27:26.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.718+0000 7f12b6800640 1 -- 172.21.15.114:0/4023414344 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b0141f90 con 0x7f12b01593b0 2024-05-27T09:27:26.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.718+0000 7f12b6800640 1 -- 172.21.15.114:0/4023414344 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b01f3140 con 0x7f12b01593b0 2024-05-27T09:27:26.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.719+0000 7f12b6800640 1 -- 172.21.15.114:0/4023414344 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12b006a380 con 0x7f12b01593b0 2024-05-27T09:27:26.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.720+0000 7f12a7400640 1 -- 172.21.15.114:0/4023414344 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f129c030b00 con 0x7f12b01593b0 2024-05-27T09:27:26.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.721+0000 7f12a7400640 1 --2- 172.21.15.114:0/4023414344 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f12900762d0 0x7f12900787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:26.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.721+0000 7f12a7400640 1 -- 172.21.15.114:0/4023414344 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f129c0b7d70 con 0x7f12b01593b0 2024-05-27T09:27:26.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.722+0000 7f12af400640 1 --2- 172.21.15.114:0/4023414344 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f12900762d0 0x7f12900787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:26.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.722+0000 7f12af400640 1 --2- 172.21.15.114:0/4023414344 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f12900762d0 0x7f12900787e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009aa0 tx=0x7f12a0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:26.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.726+0000 7f12a7400640 1 -- 172.21.15.114:0/4023414344 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f129c04c020 con 0x7f12b01593b0 2024-05-27T09:27:26.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.932+0000 7f12b6800640 1 -- 172.21.15.114:0/4023414344 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12b00c41f0 con 0x7f12b01593b0 2024-05-27T09:27:26.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.935+0000 7f12a7400640 1 -- 172.21.15.114:0/4023414344 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16645 (secure 0 0 0) 0x7f129c080fe0 con 0x7f12b01593b0 2024-05-27T09:27:26.936 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:26.936 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"fsid":"2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48","created":"2024-05-27T09:27:00.307278+0000","modified":"2024-05-27T09:27:19.530054+0000","last_up_change":"2024-05-27T09:27:11.463226+0000","last_in_change":"2024-05-27T09:27:02.972880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T09:27:10.796859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T09:27:16.063199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T09:27:17.064857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3561c750-1d63-42f6-a6ae-0ee63721e15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6817","nonce":1960550710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6819","nonce":1960550710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6824","nonce":1960550710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6821","nonce":1960550710}]},"public_addr":"172.21.15.114:6817/1960550710","cluster_addr":"172.21.15.114:6819/1960550710","heartbeat_back_addr":"172.21.15.114:6824/1960550710","heartbeat_front_addr":"172.21.15.114:6821/1960550710","state":["exists","up"]},{"osd":1,"uuid":"9a407d04-38c0-4676-928f-d04e8116bd19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6825","nonce":326580057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6827","nonce":326580057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6831","nonce":326580057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6829","nonce":326580057}]},"public_addr":"172.21.15.114:6825/326580057","cluster_addr":"172.21.15.114:6827/326580057","heartbeat_back_addr":"172.21.15.114:6831/326580057","heartbeat_front_addr":"172.21.15.114:6829/326580057","state":["exists","up"]},{"osd":2,"uuid":"255a1cc6-9357-43f7-9db5-d760f4806ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6809","nonce":865568619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6811","nonce":865568619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6815","nonce":865568619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6813","nonce":865568619}]},"public_addr":"172.21.15.114:6809/865568619","cluster_addr":"172.21.15.114:6811/865568619","heartbeat_back_addr":"172.21.15.114:6815/865568619","heartbeat_front_addr":"172.21.15.114:6813/865568619","state":["exists","up"]},{"osd":3,"uuid":"cb4d7692-9a17-41b9-927e-d8eac3e217c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6801","nonce":1601792164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6803","nonce":1601792164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6807","nonce":1601792164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6805","nonce":1601792164}]},"public_addr":"172.21.15.114:6801/1601792164","cluster_addr":"172.21.15.114:6803/1601792164","heartbeat_back_addr":"172.21.15.114:6807/1601792164","heartbeat_front_addr":"172.21.15.114:6805/1601792164","state":["exists","up"]},{"osd":4,"uuid":"cdef1940-3453-4fbd-b0f1-459f5f5c7142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6817","nonce":1982246015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6819","nonce":1982246015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6823","nonce":1982246015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6821","nonce":1982246015}]},"public_addr":"172.21.15.163:6817/1982246015","cluster_addr":"172.21.15.163:6819/1982246015","heartbeat_back_addr":"172.21.15.163:6823/1982246015","heartbeat_front_addr":"172.21.15.163:6821/1982246015","state":["exists","up"]},{"osd":5,"uuid":"220a7b32-abcc-492f-bca8-4615cc2ea111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6801","nonce":1695294004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6803","nonce":1695294004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6807","nonce":1695294004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6805","nonce":1695294004}]},"public_addr":"172.21.15.163:6801/1695294004","cluster_addr":"172.21.15.163:6803/1695294004","heartbeat_back_addr":"172.21.15.163:6807/1695294004","heartbeat_front_addr":"172.21.15.163:6805/1695294004","state":["exists","up"]},{"osd":6,"uuid":"43a74d13-d03f-493a-a6c0-e73b0a056c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6809","nonce":3649718505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6811","nonce":3649718505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6815","nonce":3649718505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6813","nonce":3649718505}]},"public_addr":"172.21.15.163:6809/3649718505","cluster_addr":"172.21.15.163:6811/3649718505","heartbeat_back_addr":"172.21.15.163:6815/3649718505","heartbeat_front_addr":"172.21.15.163:6813/3649718505","state":["exists","up"]},{"osd":7,"uuid":"134629c2-70af-4b84-83a6-7ddf53681baa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6825","nonce":2539451311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6827","nonce":2539451311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6831","nonce":2539451311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6829","nonce":2539451311}]},"public_addr":"172.21.15.163:6825/2539451311","cluster_addr":"172.21.15.163:6827/2539451311","heartbeat_back_addr":"172.21.15.163:6831/2539451311","heartbeat_front_addr":"172.21.15.163:6829/2539451311","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.388942+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.515675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.646761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.400212+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.220072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.079531+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.180419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.175008+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T09:27:26.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.938+0000 7f12b6800640 1 -- 172.21.15.114:0/4023414344 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f12900762d0 msgr2=0x7f12900787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:26.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.938+0000 7f12b6800640 1 --2- 172.21.15.114:0/4023414344 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f12900762d0 0x7f12900787e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009aa0 tx=0x7f12a0009190 comp rx=0 tx=0).stop 2024-05-27T09:27:26.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.938+0000 7f12b6800640 1 -- 172.21.15.114:0/4023414344 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b01593b0 msgr2=0x7f12b01f3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:26.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.938+0000 7f12b6800640 1 --2- 172.21.15.114:0/4023414344 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b01593b0 0x7f12b01f3940 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f129c000c00 tx=0x7f129c0046c0 comp rx=0 tx=0).stop 2024-05-27T09:27:26.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.938+0000 7f12b6800640 1 -- 172.21.15.114:0/4023414344 shutdown_connections 2024-05-27T09:27:26.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.938+0000 7f12b6800640 1 --2- 172.21.15.114:0/4023414344 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f12900762d0 0x7f12900787e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:26.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.938+0000 7f12b6800640 1 --2- 172.21.15.114:0/4023414344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f12b015ade0 0x7f12b015c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:26.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.938+0000 7f12b6800640 1 --2- 172.21.15.114:0/4023414344 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b01593b0 0x7f12b01f3940 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:26.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.938+0000 7f12b6800640 1 --2- 172.21.15.114:0/4023414344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12b0073260 0x7f12b0154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:26.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.938+0000 7f12b6800640 1 -- 172.21.15.114:0/4023414344 >> 172.21.15.114:0/4023414344 conn(0x7f12b006f6e0 msgr2=0x7f12b0071330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:26.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.938+0000 7f12b6800640 1 -- 172.21.15.114:0/4023414344 shutdown_connections 2024-05-27T09:27:26.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:26.938+0000 7f12b6800640 1 -- 172.21.15.114:0/4023414344 wait complete. 2024-05-27T09:27:26.965 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-27T09:27:26.965 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T09:27:27.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.237+0000 7f23ba600640 1 Processor -- start 2024-05-27T09:27:27.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.238+0000 7f23ba600640 1 -- start start 2024-05-27T09:27:27.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.240+0000 7f23ba600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23b41547d0 0x7f23b40c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.240+0000 7f23ba600640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f23b4074840 0x7f23b415ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.240+0000 7f23ba600640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23b406fe50 con 0x7f23b40741f0 2024-05-27T09:27:27.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.240+0000 7f23ba600640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f23b40703d0 con 0x7f23b4074840 2024-05-27T09:27:27.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.240+0000 7f23ba600640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f23b415b260 con 0x7f23b41547d0 2024-05-27T09:27:27.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.240+0000 7f23b9200640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f23b40741f0 0x7f23b406edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:46554/0 (socket says 172.21.15.114:46554) 2024-05-27T09:27:27.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.240+0000 7f23b9200640 1 -- 172.21.15.114:0/1425630518 learned_addr learned my addr 172.21.15.114:0/1425630518 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:27.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.240+0000 7f23b3e00640 1 --2- 172.21.15.114:0/1425630518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23b41547d0 0x7f23b40c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.240+0000 7f23b9c00640 1 --2- 172.21.15.114:0/1425630518 >> v2:172.21.15.163:3300/0 conn(0x7f23b4074840 0x7f23b415ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.241+0000 7f23b9c00640 1 -- 172.21.15.114:0/1425630518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23b41547d0 msgr2=0x7f23b40c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:27.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.241+0000 7f23b9c00640 1 --2- 172.21.15.114:0/1425630518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23b41547d0 0x7f23b40c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.241+0000 7f23b9c00640 1 -- 172.21.15.114:0/1425630518 >> v1:172.21.15.114:6789/0 conn(0x7f23b40741f0 legacy=0x7f23b406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.241+0000 7f23b9c00640 1 -- 172.21.15.114:0/1425630518 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b40700f0 con 0x7f23b4074840 2024-05-27T09:27:27.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.241+0000 7f23b3400640 1 -- 172.21.15.114:0/1425630518 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3283117123 0 0) 0x7f23b406fe50 con 0x7f23b40741f0 2024-05-27T09:27:27.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.241+0000 7f23b3e00640 1 --2- 172.21.15.114:0/1425630518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23b41547d0 0x7f23b40c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:27.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.242+0000 7f23b9c00640 1 --2- 172.21.15.114:0/1425630518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23b4074840 0x7f23b415ad20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009d90 tx=0x7f23ac02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25f136c9e6c39cf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.242+0000 7f23b3400640 1 -- 172.21.15.114:0/1425630518 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac007740 con 0x7f23b4074840 2024-05-27T09:27:27.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.242+0000 7f23b3400640 1 -- 172.21.15.114:0/1425630518 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23ac0078e0 con 0x7f23b4074840 2024-05-27T09:27:27.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.242+0000 7f23b3400640 1 -- 172.21.15.114:0/1425630518 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac007cb0 con 0x7f23b4074840 2024-05-27T09:27:27.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.243+0000 7f23ba600640 1 -- 172.21.15.114:0/1425630518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23b4074840 msgr2=0x7f23b415ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.243+0000 7f23ba600640 1 --2- 172.21.15.114:0/1425630518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23b4074840 0x7f23b415ad20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009d90 tx=0x7f23ac02a6f0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.243+0000 7f23ba600640 1 -- 172.21.15.114:0/1425630518 shutdown_connections 2024-05-27T09:27:27.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.243+0000 7f23ba600640 1 --2- 172.21.15.114:0/1425630518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23b4074840 0x7f23b415ad20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.243+0000 7f23ba600640 1 --2- 172.21.15.114:0/1425630518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23b41547d0 0x7f23b40c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.243+0000 7f23ba600640 1 -- 172.21.15.114:0/1425630518 >> 172.21.15.114:0/1425630518 conn(0x7f23b40ca320 msgr2=0x7f23b40ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:27.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.243+0000 7f23ba600640 1 -- 172.21.15.114:0/1425630518 shutdown_connections 2024-05-27T09:27:27.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.243+0000 7f23ba600640 1 -- 172.21.15.114:0/1425630518 wait complete. 2024-05-27T09:27:27.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.245+0000 7f23ba600640 1 Processor -- start 2024-05-27T09:27:27.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.245+0000 7f23ba600640 1 -- start start 2024-05-27T09:27:27.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.247+0000 7f23ba600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23b40741f0 0x7f23b415c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.247+0000 7f23ba600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23b4074840 0x7f23b41f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.247+0000 7f23ba600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23b41547d0 0x7f23b40c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.247+0000 7f23ba600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f23b4073790 con 0x7f23b40741f0 2024-05-27T09:27:27.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.247+0000 7f23ba600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f23b40734b0 con 0x7f23b41547d0 2024-05-27T09:27:27.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.247+0000 7f23ba600640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f23b406f500 con 0x7f23b4074840 2024-05-27T09:27:27.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.247+0000 7f23b9200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23b40741f0 0x7f23b415c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.247+0000 7f23b9200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23b40741f0 0x7f23b415c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35454/0 (socket says 172.21.15.114:35454) 2024-05-27T09:27:27.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.247+0000 7f23b9200640 1 -- 172.21.15.114:0/2327902815 learned_addr learned my addr 172.21.15.114:0/2327902815 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:27.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.247+0000 7f23b9c00640 1 --2- 172.21.15.114:0/2327902815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23b41547d0 0x7f23b40c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.247+0000 7f23b3e00640 1 --2- 172.21.15.114:0/2327902815 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23b4074840 0x7f23b41f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.247+0000 7f23b9200640 1 -- 172.21.15.114:0/2327902815 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23b4074840 msgr2=0x7f23b41f12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.247+0000 7f23b9200640 1 --2- 172.21.15.114:0/2327902815 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23b4074840 0x7f23b41f12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.247+0000 7f23b9200640 1 -- 172.21.15.114:0/2327902815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23b41547d0 msgr2=0x7f23b40c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.247+0000 7f23b9200640 1 --2- 172.21.15.114:0/2327902815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23b41547d0 0x7f23b40c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.247+0000 7f23b9200640 1 -- 172.21.15.114:0/2327902815 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b41f1830 con 0x7f23b40741f0 2024-05-27T09:27:27.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.248+0000 7f23b9200640 1 --2- 172.21.15.114:0/2327902815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23b40741f0 0x7f23b415c550 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23a400c430 tx=0x7f23a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.248+0000 7f23b0c00640 1 -- 172.21.15.114:0/2327902815 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a4017070 con 0x7f23b40741f0 2024-05-27T09:27:27.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.248+0000 7f23b0c00640 1 -- 172.21.15.114:0/2327902815 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23a4005ce0 con 0x7f23b40741f0 2024-05-27T09:27:27.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.248+0000 7f23ba600640 1 -- 172.21.15.114:0/2327902815 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b415e3a0 con 0x7f23b40741f0 2024-05-27T09:27:27.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.248+0000 7f23b0c00640 1 -- 172.21.15.114:0/2327902815 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a4007310 con 0x7f23b40741f0 2024-05-27T09:27:27.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.248+0000 7f23ba600640 1 -- 172.21.15.114:0/2327902815 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b415bd50 con 0x7f23b40741f0 2024-05-27T09:27:27.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.249+0000 7f23ba600640 1 -- 172.21.15.114:0/2327902815 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23b4157cb0 con 0x7f23b40741f0 2024-05-27T09:27:27.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.250+0000 7f23b0c00640 1 -- 172.21.15.114:0/2327902815 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f23a401a070 con 0x7f23b40741f0 2024-05-27T09:27:27.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.251+0000 7f23b0c00640 1 --2- 172.21.15.114:0/2327902815 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f23940762d0 0x7f23940787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.251+0000 7f23b0c00640 1 -- 172.21.15.114:0/2327902815 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f23a4096a30 con 0x7f23b40741f0 2024-05-27T09:27:27.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.252+0000 7f23b3e00640 1 --2- 172.21.15.114:0/2327902815 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f23940762d0 0x7f23940787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.253+0000 7f23b3e00640 1 --2- 172.21.15.114:0/2327902815 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f23940762d0 0x7f23940787e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f23b40c7ea0 tx=0x7f23a0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.256+0000 7f23b0c00640 1 -- 172.21.15.114:0/2327902815 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f23a405fc10 con 0x7f23b40741f0 2024-05-27T09:27:27.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.467+0000 7f23ba600640 1 -- 172.21.15.114:0/2327902815 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23b40764f0 con 0x7f23b40741f0 2024-05-27T09:27:27.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.469+0000 7f23b0c00640 1 -- 172.21.15.114:0/2327902815 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16645 (secure 0 0 0) 0x7f23a4063f10 con 0x7f23b40741f0 2024-05-27T09:27:27.471 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:27.471 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"fsid":"2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48","created":"2024-05-27T09:27:00.307278+0000","modified":"2024-05-27T09:27:19.530054+0000","last_up_change":"2024-05-27T09:27:11.463226+0000","last_in_change":"2024-05-27T09:27:02.972880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T09:27:10.796859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T09:27:16.063199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T09:27:17.064857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3561c750-1d63-42f6-a6ae-0ee63721e15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6817","nonce":1960550710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6819","nonce":1960550710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6824","nonce":1960550710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6821","nonce":1960550710}]},"public_addr":"172.21.15.114:6817/1960550710","cluster_addr":"172.21.15.114:6819/1960550710","heartbeat_back_addr":"172.21.15.114:6824/1960550710","heartbeat_front_addr":"172.21.15.114:6821/1960550710","state":["exists","up"]},{"osd":1,"uuid":"9a407d04-38c0-4676-928f-d04e8116bd19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6825","nonce":326580057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6827","nonce":326580057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6831","nonce":326580057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6829","nonce":326580057}]},"public_addr":"172.21.15.114:6825/326580057","cluster_addr":"172.21.15.114:6827/326580057","heartbeat_back_addr":"172.21.15.114:6831/326580057","heartbeat_front_addr":"172.21.15.114:6829/326580057","state":["exists","up"]},{"osd":2,"uuid":"255a1cc6-9357-43f7-9db5-d760f4806ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6809","nonce":865568619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6811","nonce":865568619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6815","nonce":865568619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6813","nonce":865568619}]},"public_addr":"172.21.15.114:6809/865568619","cluster_addr":"172.21.15.114:6811/865568619","heartbeat_back_addr":"172.21.15.114:6815/865568619","heartbeat_front_addr":"172.21.15.114:6813/865568619","state":["exists","up"]},{"osd":3,"uuid":"cb4d7692-9a17-41b9-927e-d8eac3e217c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6801","nonce":1601792164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6803","nonce":1601792164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6807","nonce":1601792164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6805","nonce":1601792164}]},"public_addr":"172.21.15.114:6801/1601792164","cluster_addr":"172.21.15.114:6803/1601792164","heartbeat_back_addr":"172.21.15.114:6807/1601792164","heartbeat_front_addr":"172.21.15.114:6805/1601792164","state":["exists","up"]},{"osd":4,"uuid":"cdef1940-3453-4fbd-b0f1-459f5f5c7142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6817","nonce":1982246015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6819","nonce":1982246015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6823","nonce":1982246015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6821","nonce":1982246015}]},"public_addr":"172.21.15.163:6817/1982246015","cluster_addr":"172.21.15.163:6819/1982246015","heartbeat_back_addr":"172.21.15.163:6823/1982246015","heartbeat_front_addr":"172.21.15.163:6821/1982246015","state":["exists","up"]},{"osd":5,"uuid":"220a7b32-abcc-492f-bca8-4615cc2ea111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6801","nonce":1695294004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6803","nonce":1695294004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6807","nonce":1695294004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6805","nonce":1695294004}]},"public_addr":"172.21.15.163:6801/1695294004","cluster_addr":"172.21.15.163:6803/1695294004","heartbeat_back_addr":"172.21.15.163:6807/1695294004","heartbeat_front_addr":"172.21.15.163:6805/1695294004","state":["exists","up"]},{"osd":6,"uuid":"43a74d13-d03f-493a-a6c0-e73b0a056c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6809","nonce":3649718505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6811","nonce":3649718505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6815","nonce":3649718505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6813","nonce":3649718505}]},"public_addr":"172.21.15.163:6809/3649718505","cluster_addr":"172.21.15.163:6811/3649718505","heartbeat_back_addr":"172.21.15.163:6815/3649718505","heartbeat_front_addr":"172.21.15.163:6813/3649718505","state":["exists","up"]},{"osd":7,"uuid":"134629c2-70af-4b84-83a6-7ddf53681baa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6825","nonce":2539451311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6827","nonce":2539451311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6831","nonce":2539451311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6829","nonce":2539451311}]},"public_addr":"172.21.15.163:6825/2539451311","cluster_addr":"172.21.15.163:6827/2539451311","heartbeat_back_addr":"172.21.15.163:6831/2539451311","heartbeat_front_addr":"172.21.15.163:6829/2539451311","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.388942+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.515675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.646761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.400212+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.220072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.079531+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.180419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.175008+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T09:27:27.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.472+0000 7f23ba600640 1 -- 172.21.15.114:0/2327902815 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f23940762d0 msgr2=0x7f23940787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.472+0000 7f23ba600640 1 --2- 172.21.15.114:0/2327902815 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f23940762d0 0x7f23940787e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f23b40c7ea0 tx=0x7f23a0009190 comp rx=0 tx=0).stop 2024-05-27T09:27:27.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.472+0000 7f23ba600640 1 -- 172.21.15.114:0/2327902815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23b40741f0 msgr2=0x7f23b415c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.472+0000 7f23ba600640 1 --2- 172.21.15.114:0/2327902815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23b40741f0 0x7f23b415c550 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23a400c430 tx=0x7f23a400c8f0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.472+0000 7f23ba600640 1 -- 172.21.15.114:0/2327902815 shutdown_connections 2024-05-27T09:27:27.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.472+0000 7f23ba600640 1 --2- 172.21.15.114:0/2327902815 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f23940762d0 0x7f23940787e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.472+0000 7f23ba600640 1 --2- 172.21.15.114:0/2327902815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23b41547d0 0x7f23b40c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.472+0000 7f23ba600640 1 --2- 172.21.15.114:0/2327902815 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23b4074840 0x7f23b41f12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.472+0000 7f23ba600640 1 --2- 172.21.15.114:0/2327902815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f23b40741f0 0x7f23b415c550 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.472+0000 7f23ba600640 1 -- 172.21.15.114:0/2327902815 >> 172.21.15.114:0/2327902815 conn(0x7f23b40ca320 msgr2=0x7f23b40c4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:27.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.472+0000 7f23ba600640 1 -- 172.21.15.114:0/2327902815 shutdown_connections 2024-05-27T09:27:27.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.472+0000 7f23ba600640 1 -- 172.21.15.114:0/2327902815 wait complete. 2024-05-27T09:27:27.499 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-27T09:27:27.499 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-27T09:27:27.500 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-27T09:27:27.500 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-27T09:27:27.500 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-27T09:27:27.501 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-27T09:27:27.501 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-27T09:27:27.502 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-27T09:27:27.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.907+0000 7f6fb0800640 1 Processor -- start 2024-05-27T09:27:27.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.907+0000 7f6fb0800640 1 -- start start 2024-05-27T09:27:27.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.909+0000 7fc4d3400640 1 Processor -- start 2024-05-27T09:27:27.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.909+0000 7fc4d3400640 1 -- start start 2024-05-27T09:27:27.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.910+0000 7f6fb0800640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6fa80c28b0 0x7f6fa815ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.910+0000 7f6fb0800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6fa8073ab0 0x7f6fa8070d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.910+0000 7f6fb0800640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fa8160ab0 con 0x7f6fa8070960 2024-05-27T09:27:27.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.910+0000 7f6fb0800640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f6fa81618d0 con 0x7f6fa80c28b0 2024-05-27T09:27:27.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.910+0000 7f6fb0800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa8160d50 con 0x7f6fa8073ab0 2024-05-27T09:27:27.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.911+0000 7f6fad400640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6fa8070960 0x7f6fa80733a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:44618/0 (socket says 172.21.15.114:44618) 2024-05-27T09:27:27.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.911+0000 7f6fad400640 1 -- 172.21.15.114:0/3615519273 learned_addr learned my addr 172.21.15.114:0/3615519273 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:27.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.911+0000 7f6fae800640 1 --2- 172.21.15.114:0/3615519273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6fa8073ab0 0x7f6fa8070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.912+0000 7f6faca00640 1 -- 172.21.15.114:0/3615519273 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2547521285 0 0) 0x7f6fa8160ab0 con 0x7f6fa8070960 2024-05-27T09:27:27.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.912+0000 7fc4d3400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc4c40d9830 0x7fc4c40073b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.912+0000 7fc4d3400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc4c404b2e0 0x7fc4c40078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.912+0000 7f6faca00640 1 -- 172.21.15.114:0/3615519273 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f90003520 con 0x7f6fa8070960 2024-05-27T09:27:27.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.912+0000 7fc4d3400640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4c40dfee0 con 0x7fc4c4050ab0 2024-05-27T09:27:27.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.912+0000 7fc4d3400640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc4c4049fd0 con 0x7fc4c404b2e0 2024-05-27T09:27:27.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.912+0000 7fc4d3400640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fc4c40e0180 con 0x7fc4c40d9830 2024-05-27T09:27:27.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.913+0000 7fc4d1400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc4c404b2e0 0x7fc4c40078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.913+0000 7fc4d1400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc4c404b2e0 0x7fc4c40078f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35470/0 (socket says 172.21.15.114:35470) 2024-05-27T09:27:27.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.913+0000 7fc4d1400640 1 -- 172.21.15.114:0/1781053875 learned_addr learned my addr 172.21.15.114:0/1781053875 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:27.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.913+0000 7fc4d0a00640 1 --2- 172.21.15.114:0/1781053875 >> v2:172.21.15.163:3300/0 conn(0x7fc4c40d9830 0x7fc4c40073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.912+0000 7f6fae800640 1 -- 172.21.15.114:0/3615519273 >> v2:172.21.15.163:3300/0 conn(0x7f6fa80c28b0 msgr2=0x7f6fa815ac00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T09:27:27.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.912+0000 7f6fae800640 1 --2- 172.21.15.114:0/3615519273 >> v2:172.21.15.163:3300/0 conn(0x7f6fa80c28b0 0x7f6fa815ac00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.912+0000 7f6fae800640 1 -- 172.21.15.114:0/3615519273 >> v1:172.21.15.163:6789/0 conn(0x7f6fa8070960 legacy=0x7f6fa80733a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.912+0000 7f6fae800640 1 -- 172.21.15.114:0/3615519273 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa8161030 con 0x7f6fa8073ab0 2024-05-27T09:27:27.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.914+0000 7fc4d0a00640 1 -- 172.21.15.114:0/1781053875 >> v2:172.21.15.114:3300/0 conn(0x7fc4c404b2e0 msgr2=0x7fc4c40078f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:27.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.914+0000 7f6fae800640 1 --2- 172.21.15.114:0/3615519273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6fa8073ab0 0x7f6fa8070d70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8161a50 tx=0x7f6fa402a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ed614489643a9cc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.914+0000 7f6faca00640 1 -- 172.21.15.114:0/3615519273 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4007740 con 0x7f6fa8073ab0 2024-05-27T09:27:27.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.914+0000 7f6faca00640 1 -- 172.21.15.114:0/3615519273 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fa40078e0 con 0x7f6fa8073ab0 2024-05-27T09:27:27.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.914+0000 7f6faca00640 1 -- 172.21.15.114:0/3615519273 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4007cb0 con 0x7f6fa8073ab0 2024-05-27T09:27:27.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.915+0000 7ff422200640 1 Processor -- start 2024-05-27T09:27:27.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.914+0000 7fc4d0a00640 1 --2- 172.21.15.114:0/1781053875 >> v2:172.21.15.114:3300/0 conn(0x7fc4c404b2e0 0x7fc4c40078f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.914+0000 7fc4d0a00640 1 -- 172.21.15.114:0/1781053875 >> v1:172.21.15.114:6789/0 conn(0x7fc4c4050ab0 legacy=0x7fc4c404abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:27.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.915+0000 7fc4d0a00640 1 -- 172.21.15.114:0/1781053875 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c404a6b0 con 0x7fc4c40d9830 2024-05-27T09:27:27.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.916+0000 7f6fb0800640 1 -- 172.21.15.114:0/3615519273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6fa8073ab0 msgr2=0x7f6fa8070d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.916+0000 7f6fb0800640 1 --2- 172.21.15.114:0/3615519273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6fa8073ab0 0x7f6fa8070d70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8161a50 tx=0x7f6fa402a680 comp rx=0 tx=0).stop 2024-05-27T09:27:27.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.917+0000 7f6fb0800640 1 -- 172.21.15.114:0/3615519273 shutdown_connections 2024-05-27T09:27:27.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.917+0000 7f6fb0800640 1 --2- 172.21.15.114:0/3615519273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6fa8073ab0 0x7f6fa8070d70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.917+0000 7f6fb0800640 1 --2- 172.21.15.114:0/3615519273 >> v2:172.21.15.163:3300/0 conn(0x7f6fa80c28b0 0x7f6fa815ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.915+0000 7ff422200640 1 -- start start 2024-05-27T09:27:27.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.917+0000 7f6fb0800640 1 -- 172.21.15.114:0/3615519273 >> 172.21.15.114:0/3615519273 conn(0x7f6fa80cd890 msgr2=0x7f6fa80cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:27.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.917+0000 7f01bbe00640 1 Processor -- start 2024-05-27T09:27:27.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.917+0000 7f01bbe00640 1 -- start start 2024-05-27T09:27:27.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.917+0000 7f6fb0800640 1 -- 172.21.15.114:0/3615519273 shutdown_connections 2024-05-27T09:27:27.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.918+0000 7ff422200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff41c0ce440 0x7ff41c0c8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.918+0000 7ff422200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff41c0c9240 con 0x7ff41c0710e0 2024-05-27T09:27:27.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.918+0000 7ff422200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff41c0c9800 con 0x7ff41c15ad10 2024-05-27T09:27:27.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.918+0000 7ff422200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7ff41c0c9ae0 con 0x7ff41c0ce440 2024-05-27T09:27:27.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.919+0000 7ff421800640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7ff41c0710e0 0x7ff41c16b220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:46576/0 (socket says 172.21.15.114:46576) 2024-05-27T09:27:27.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.919+0000 7ff421800640 1 -- 172.21.15.114:0/1730869083 learned_addr learned my addr 172.21.15.114:0/1730869083 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:27.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.917+0000 7f6fb0800640 1 -- 172.21.15.114:0/3615519273 wait complete. 2024-05-27T09:27:27.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.920+0000 7f01bbe00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f01b4160860 0x7f01b415d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.917+0000 7fc4d0a00640 1 --2- 172.21.15.114:0/1781053875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4c40d9830 0x7fc4c40073b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4055720 tx=0x7fc4c002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e20f81e1480bb78 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.918+0000 7fc4cb400640 1 -- 172.21.15.114:0/1781053875 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0037070 con 0x7fc4c40d9830 2024-05-27T09:27:27.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.918+0000 7fc4cb400640 1 -- 172.21.15.114:0/1781053875 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4c0032440 con 0x7fc4c40d9830 2024-05-27T09:27:27.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.918+0000 7fc4cb400640 1 -- 172.21.15.114:0/1781053875 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0032850 con 0x7fc4c40d9830 2024-05-27T09:27:27.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.919+0000 7fc4d3400640 1 -- 172.21.15.114:0/1781053875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4c40d9830 msgr2=0x7fc4c40073b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.919+0000 7fc4d3400640 1 --2- 172.21.15.114:0/1781053875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4c40d9830 0x7fc4c40073b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4055720 tx=0x7fc4c002a1c0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.919+0000 7fc4d3400640 1 -- 172.21.15.114:0/1781053875 shutdown_connections 2024-05-27T09:27:27.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.919+0000 7fc4d3400640 1 --2- 172.21.15.114:0/1781053875 >> v2:172.21.15.114:3300/0 conn(0x7fc4c404b2e0 0x7fc4c40078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.919+0000 7fc4d3400640 1 --2- 172.21.15.114:0/1781053875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4c40d9830 0x7fc4c40073b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.919+0000 7fc4d3400640 1 -- 172.21.15.114:0/1781053875 >> 172.21.15.114:0/1781053875 conn(0x7fc4c400e360 msgr2=0x7fc4c400e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:27.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.920+0000 7fc4d3400640 1 -- 172.21.15.114:0/1781053875 shutdown_connections 2024-05-27T09:27:27.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.920+0000 7fc4d3400640 1 -- 172.21.15.114:0/1781053875 wait complete. 2024-05-27T09:27:27.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.921+0000 7f01b9400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f01b4160860 0x7f01b415d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.920+0000 7fa080000640 1 Processor -- start 2024-05-27T09:27:27.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.920+0000 7fa080000640 1 -- start start 2024-05-27T09:27:27.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.921+0000 7f01b9400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f01b4160860 0x7f01b415d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35478/0 (socket says 172.21.15.114:35478) 2024-05-27T09:27:27.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.921+0000 7f01b9400640 1 -- 172.21.15.114:0/935514035 learned_addr learned my addr 172.21.15.114:0/935514035 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:27.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.920+0000 7ff420e00640 1 --2- 172.21.15.114:0/1730869083 >> v2:172.21.15.163:3300/0 conn(0x7ff41c0ce440 0x7ff41c0c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.921+0000 7f01bbe00640 1 -- 172.21.15.114:0/935514035 --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01b40cc6f0 con 0x7f01b40c73b0 2024-05-27T09:27:27.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.921+0000 7f6fb0800640 1 Processor -- start 2024-05-27T09:27:27.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.921+0000 7f01bbe00640 1 -- 172.21.15.114:0/935514035 --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01b40cc990 con 0x7f01b4156760 2024-05-27T09:27:27.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.921+0000 7f01bbe00640 1 -- 172.21.15.114:0/935514035 --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f01b40ccc30 con 0x7f01b4160860 2024-05-27T09:27:27.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.921+0000 7f6fb0800640 1 -- start start 2024-05-27T09:27:27.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.922+0000 7ff41b400640 1 -- 172.21.15.114:0/1730869083 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2686820070 0 0) 0x7ff41c0c9240 con 0x7ff41c0710e0 2024-05-27T09:27:27.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.922+0000 7ff41b400640 1 -- 172.21.15.114:0/1730869083 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff400003520 con 0x7ff41c0710e0 2024-05-27T09:27:27.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.922+0000 7ff420e00640 1 -- 172.21.15.114:0/1730869083 >> v1:172.21.15.163:6789/0 conn(0x7ff41c15ad10 legacy=0x7ff41c0709d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.922+0000 7ff420e00640 1 -- 172.21.15.114:0/1730869083 >> v1:172.21.15.114:6789/0 conn(0x7ff41c0710e0 legacy=0x7ff41c16b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.922+0000 7ff420e00640 1 -- 172.21.15.114:0/1730869083 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41c0c9dc0 con 0x7ff41c0ce440 2024-05-27T09:27:27.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.923+0000 7f01b3e00640 1 -- 172.21.15.114:0/935514035 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2387189031 0 0) 0x7f01b40cc990 con 0x7f01b4156760 2024-05-27T09:27:27.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.923+0000 7f01b3e00640 1 -- 172.21.15.114:0/935514035 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f019c003520 con 0x7f01b4156760 2024-05-27T09:27:27.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.923+0000 7f01b9400640 1 -- 172.21.15.114:0/935514035 >> v1:172.21.15.163:6789/0 conn(0x7f01b4156760 legacy=0x7f01b40b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.923+0000 7f01b9400640 1 -- 172.21.15.114:0/935514035 >> v1:172.21.15.114:6789/0 conn(0x7f01b40c73b0 legacy=0x7f01b40c77c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T09:27:27.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.923+0000 7f01b9400640 1 -- 172.21.15.114:0/935514035 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b415fd80 con 0x7f01b4160860 2024-05-27T09:27:27.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.924+0000 7f8fc0000640 1 Processor -- start 2024-05-27T09:27:27.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.924+0000 7f8fc0000640 1 -- start start 2024-05-27T09:27:27.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.924+0000 7fa080000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa07808d320 0x7fa078073c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.924+0000 7fa080000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa078169450 0x7fa078169860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.924+0000 7fa080000640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa078166ee0 con 0x7fa0780ca300 2024-05-27T09:27:27.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.924+0000 7fa080000640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fa0780c4720 con 0x7fa078169450 2024-05-27T09:27:27.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.924+0000 7fa080000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa07816d820 con 0x7fa07808d320 2024-05-27T09:27:27.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.924+0000 7f6fb0800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6fa8070960 0x7f6fa8161f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.924+0000 7ff420e00640 1 --2- 172.21.15.114:0/1730869083 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff41c0ce440 0x7ff41c0c8290 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff414009d90 tx=0x7ff41402a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4f8cff302582ba7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.924+0000 7fa07cc00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa078169450 0x7fa078169860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.925+0000 7ff41b400640 1 -- 172.21.15.114:0/1730869083 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414007740 con 0x7ff41c0ce440 2024-05-27T09:27:27.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.925+0000 7f6fade00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6fa8070960 0x7f6fa8161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.925+0000 7f6fade00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6fa8070960 0x7f6fa8161f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:60432/0 (socket says 172.21.15.114:60432) 2024-05-27T09:27:27.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.924+0000 7fa07d600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa07808d320 0x7fa078073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.925+0000 7f6fade00640 1 -- 172.21.15.114:0/2480549801 learned_addr learned my addr 172.21.15.114:0/2480549801 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:27.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.925+0000 7fa07cc00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa078169450 0x7fa078169860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37594/0 (socket says 172.21.15.114:37594) 2024-05-27T09:27:27.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.925+0000 7fa07cc00640 1 -- 172.21.15.114:0/1076803141 learned_addr learned my addr 172.21.15.114:0/1076803141 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:27.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.925+0000 7f6fb0800640 1 --2- 172.21.15.114:0/2480549801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fa8073ab0 0x7f6fa80c7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.925+0000 7fa07cc00640 1 -- 172.21.15.114:0/1076803141 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa07808d320 msgr2=0x7fa078073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:27.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.925+0000 7fa07cc00640 1 --2- 172.21.15.114:0/1076803141 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa07808d320 0x7fa078073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.925+0000 7f01b9400640 1 --2- 172.21.15.114:0/935514035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01b4160860 0x7f01b415d060 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009870 tx=0x7f01a802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1c16802ba50ebd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.925+0000 7fa07cc00640 1 -- 172.21.15.114:0/1076803141 >> v1:172.21.15.163:6789/0 conn(0x7fa0780ca300 legacy=0x7fa0780ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.925+0000 7fa07cc00640 1 -- 172.21.15.114:0/1076803141 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0780c49e0 con 0x7fa078169450 2024-05-27T09:27:27.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.925+0000 7ff41b400640 1 -- 172.21.15.114:0/1730869083 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4140078e0 con 0x7ff41c0ce440 2024-05-27T09:27:27.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.925+0000 7ff41b400640 1 -- 172.21.15.114:0/1730869083 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414007cb0 con 0x7ff41c0ce440 2024-05-27T09:27:27.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.925+0000 7f6fb0800640 1 --2- 172.21.15.114:0/2480549801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fa80c28b0 0x7f6fa80cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.925+0000 7f6fb0800640 1 -- 172.21.15.114:0/2480549801 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa80729d0 con 0x7f6fa8073ab0 2024-05-27T09:27:27.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.925+0000 7f6fb0800640 1 -- 172.21.15.114:0/2480549801 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa81615f0 con 0x7f6fa80c28b0 2024-05-27T09:27:27.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.925+0000 7f6fb0800640 1 -- 172.21.15.114:0/2480549801 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa8072b50 con 0x7f6fa8070960 2024-05-27T09:27:27.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.926+0000 7f6fad400640 1 --2- 172.21.15.114:0/2480549801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fa8073ab0 0x7f6fa80c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.926+0000 7f01b3e00640 1 -- 172.21.15.114:0/935514035 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8037070 con 0x7f01b4160860 2024-05-27T09:27:27.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.926+0000 7f01b3e00640 1 -- 172.21.15.114:0/935514035 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a8032440 con 0x7f01b4160860 2024-05-27T09:27:27.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.926+0000 7f01b3e00640 1 -- 172.21.15.114:0/935514035 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8032810 con 0x7f01b4160860 2024-05-27T09:27:27.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.926+0000 7f6fae800640 1 --2- 172.21.15.114:0/2480549801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fa80c28b0 0x7f6fa80cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.926+0000 7fc4d3400640 1 Processor -- start 2024-05-27T09:27:27.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.926+0000 7f6fade00640 1 -- 172.21.15.114:0/2480549801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fa80c28b0 msgr2=0x7f6fa80cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.926+0000 7f6fade00640 1 --2- 172.21.15.114:0/2480549801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fa80c28b0 0x7f6fa80cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.926+0000 7f6fade00640 1 -- 172.21.15.114:0/2480549801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fa8073ab0 msgr2=0x7f6fa80c7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.926+0000 7f6fade00640 1 --2- 172.21.15.114:0/2480549801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fa8073ab0 0x7f6fa80c7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.926+0000 7f6fade00640 1 -- 172.21.15.114:0/2480549801 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa8162450 con 0x7f6fa8070960 2024-05-27T09:27:27.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.926+0000 7fc4d3400640 1 -- start start 2024-05-27T09:27:27.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.926+0000 7f6fad400640 1 --2- 172.21.15.114:0/2480549801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fa8073ab0 0x7f6fa80c7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:27:27.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.926+0000 7f6fae800640 1 --2- 172.21.15.114:0/2480549801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fa80c28b0 0x7f6fa80cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:27:27.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.926+0000 7ff422200640 1 -- 172.21.15.114:0/1730869083 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff41c0ce440 msgr2=0x7ff41c0c8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.926+0000 7ff422200640 1 --2- 172.21.15.114:0/1730869083 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff41c0ce440 0x7ff41c0c8290 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff414009d90 tx=0x7ff41402a560 comp rx=0 tx=0).stop 2024-05-27T09:27:27.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.926+0000 7faf23600640 1 Processor -- start 2024-05-27T09:27:27.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.926+0000 7f6fade00640 1 --2- 172.21.15.114:0/2480549801 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6fa8070960 0x7f6fa8161f10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00b3c0 tx=0x7f6f9c00b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.927+0000 7f6fa2000640 1 -- 172.21.15.114:0/2480549801 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c005920 con 0x7f6fa8070960 2024-05-27T09:27:27.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.927+0000 7f8fc0000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8fb80c8740 0x7f8fb81421b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.927+0000 7f8fc0000640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8fb8142810 0x7f8fb80c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.927+0000 7f6fb0800640 1 -- 172.21.15.114:0/2480549801 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa81ff510 con 0x7f6fa8070960 2024-05-27T09:27:27.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.927+0000 7f8fc0000640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fb80c9280 con 0x7f8fb8159fe0 2024-05-27T09:27:27.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.927+0000 7f8fc0000640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f8fb815a4b0 con 0x7f8fb8142810 2024-05-27T09:27:27.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.927+0000 7f8fc0000640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f8fb8074610 con 0x7f8fb80c8740 2024-05-27T09:27:27.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.927+0000 7ff422200640 1 -- 172.21.15.114:0/1730869083 shutdown_connections 2024-05-27T09:27:27.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.927+0000 7ff422200640 1 --2- 172.21.15.114:0/1730869083 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff41c0ce440 0x7ff41c0c8290 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.927+0000 7ff422200640 1 -- 172.21.15.114:0/1730869083 >> 172.21.15.114:0/1730869083 conn(0x7ff41c0cd890 msgr2=0x7ff41c0cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:27.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.927+0000 7faf23600640 1 -- start start 2024-05-27T09:27:27.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.927+0000 7f6fb0800640 1 -- 172.21.15.114:0/2480549801 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa81ffad0 con 0x7f6fa8070960 2024-05-27T09:27:27.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.927+0000 7f6fa2000640 1 -- 172.21.15.114:0/2480549801 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f9c005ae0 con 0x7f6fa8070960 2024-05-27T09:27:27.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.927+0000 7f6fa2000640 1 -- 172.21.15.114:0/2480549801 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c006020 con 0x7f6fa8070960 2024-05-27T09:27:27.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.927+0000 7fa07cc00640 1 --2- 172.21.15.114:0/1076803141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa078169450 0x7fa078169860 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009870 tx=0x7fa06c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f42d6265ce7c7016 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.928+0000 7fa077e00640 1 -- 172.21.15.114:0/1076803141 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06c037070 con 0x7fa078169450 2024-05-27T09:27:27.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.928+0000 7ff422200640 1 -- 172.21.15.114:0/1730869083 shutdown_connections 2024-05-27T09:27:27.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.928+0000 7fa077e00640 1 -- 172.21.15.114:0/1076803141 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa06c032440 con 0x7fa078169450 2024-05-27T09:27:27.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.928+0000 7fa077e00640 1 -- 172.21.15.114:0/1076803141 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06c032810 con 0x7fa078169450 2024-05-27T09:27:27.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.928+0000 7fc4d3400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4c4050ab0 0x7fc4c40d4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.928+0000 7fc4d3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4c40d9830 0x7fc4c40d3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.928+0000 7fc4d3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c40d51f0 0x7fc4c40dd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.928+0000 7fc4d3400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c400f390 con 0x7fc4c40d51f0 2024-05-27T09:27:27.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.928+0000 7fc4d3400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c400f210 con 0x7fc4c40d9830 2024-05-27T09:27:27.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.928+0000 7fc4d3400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c40495e0 con 0x7fc4c4050ab0 2024-05-27T09:27:27.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.927+0000 7f01bbe00640 1 -- 172.21.15.114:0/935514035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01b4160860 msgr2=0x7f01b415d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.927+0000 7f01bbe00640 1 --2- 172.21.15.114:0/935514035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01b4160860 0x7f01b415d060 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009870 tx=0x7f01a802a250 comp rx=0 tx=0).stop 2024-05-27T09:27:27.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.928+0000 7f6fb0800640 1 -- 172.21.15.114:0/2480549801 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6f84001030 con 0x7f6fa8070960 2024-05-27T09:27:27.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.929+0000 7f01bbe00640 1 -- 172.21.15.114:0/935514035 shutdown_connections 2024-05-27T09:27:27.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.929+0000 7f01bbe00640 1 --2- 172.21.15.114:0/935514035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01b4160860 0x7f01b415d060 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.929+0000 7f01bbe00640 1 -- 172.21.15.114:0/935514035 >> 172.21.15.114:0/935514035 conn(0x7f01b406f2f0 msgr2=0x7f01b4068600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:27.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.929+0000 7fc4cbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4c40d9830 0x7fc4c40d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.929+0000 7f01bbe00640 1 -- 172.21.15.114:0/935514035 shutdown_connections 2024-05-27T09:27:27.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.930+0000 7fa080000640 1 -- 172.21.15.114:0/1076803141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa078169450 msgr2=0x7fa078169860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.930+0000 7fa080000640 1 --2- 172.21.15.114:0/1076803141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa078169450 0x7fa078169860 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009870 tx=0x7fa06c02a250 comp rx=0 tx=0).stop 2024-05-27T09:27:27.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.930+0000 7fa080000640 1 -- 172.21.15.114:0/1076803141 shutdown_connections 2024-05-27T09:27:27.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.930+0000 7fa080000640 1 --2- 172.21.15.114:0/1076803141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa078169450 0x7fa078169860 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.930+0000 7fa080000640 1 --2- 172.21.15.114:0/1076803141 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa07808d320 0x7fa078073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.930+0000 7fa080000640 1 -- 172.21.15.114:0/1076803141 >> 172.21.15.114:0/1076803141 conn(0x7fa07806f6e0 msgr2=0x7fa078070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:27.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.931+0000 7f6fa2000640 1 -- 172.21.15.114:0/2480549801 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6f9c018020 con 0x7f6fa8070960 2024-05-27T09:27:27.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.929+0000 7fc4cbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4c40d9830 0x7fc4c40d3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37622/0 (socket says 172.21.15.114:37622) 2024-05-27T09:27:27.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.929+0000 7fc4d1400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c40d51f0 0x7fc4c40dd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.929+0000 7fc4cbe00640 1 -- 172.21.15.114:0/4623052 learned_addr learned my addr 172.21.15.114:0/4623052 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:27.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.930+0000 7fc4d1400640 1 -- 172.21.15.114:0/4623052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4c4050ab0 msgr2=0x7fc4c40d4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.930+0000 7fc4d1400640 1 --2- 172.21.15.114:0/4623052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4c4050ab0 0x7fc4c40d4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.930+0000 7fc4d1400640 1 -- 172.21.15.114:0/4623052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4c40d9830 msgr2=0x7fc4c40d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.930+0000 7fc4d1400640 1 --2- 172.21.15.114:0/4623052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4c40d9830 0x7fc4c40d3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.930+0000 7fc4d1400640 1 -- 172.21.15.114:0/4623052 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c40dae30 con 0x7fc4c40d51f0 2024-05-27T09:27:27.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.930+0000 7fc4d1400640 1 --2- 172.21.15.114:0/4623052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c40d51f0 0x7fc4c40dd6a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc06bd90 tx=0x7fc4cc08dd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.931+0000 7fa080000640 1 -- 172.21.15.114:0/1076803141 shutdown_connections 2024-05-27T09:27:27.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.930+0000 7f01bbe00640 1 -- 172.21.15.114:0/935514035 wait complete. 2024-05-27T09:27:27.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.928+0000 7ff422200640 1 -- 172.21.15.114:0/1730869083 wait complete. 2024-05-27T09:27:27.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.931+0000 7faf23600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf1c15abc0 0x7faf1c0ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.931+0000 7faf23600640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf1c073ff0 con 0x7faf1c161800 2024-05-27T09:27:27.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.931+0000 7faf23600640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf1c074cc0 con 0x7faf1c0c39b0 2024-05-27T09:27:27.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.931+0000 7faf23600640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7faf1c073a30 con 0x7faf1c15abc0 2024-05-27T09:27:27.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.931+0000 7faf13e00640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7faf1c0c39b0 0x7faf1c164ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:44654/0 (socket says 172.21.15.114:44654) 2024-05-27T09:27:27.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.931+0000 7faf13e00640 1 -- 172.21.15.114:0/1020418385 learned_addr learned my addr 172.21.15.114:0/1020418385 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:27.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.931+0000 7faf20c00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7faf1c161800 0x7faf1c073260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:46598/0 (socket says 172.21.15.114:46598) 2024-05-27T09:27:27.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.932+0000 7faf21600640 1 --2- 172.21.15.114:0/1020418385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf1c15abc0 0x7faf1c0ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.932+0000 7faf1be00640 1 -- 172.21.15.114:0/1020418385 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 669712396 0 0) 0x7faf1c074cc0 con 0x7faf1c0c39b0 2024-05-27T09:27:27.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.932+0000 7faf1be00640 1 -- 172.21.15.114:0/1020418385 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf00003520 con 0x7faf1c0c39b0 2024-05-27T09:27:27.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.933+0000 7faf21600640 1 -- 172.21.15.114:0/1020418385 >> v1:172.21.15.163:6789/0 conn(0x7faf1c0c39b0 legacy=0x7faf1c164ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.933+0000 7faf21600640 1 -- 172.21.15.114:0/1020418385 >> v1:172.21.15.114:6789/0 conn(0x7faf1c161800 legacy=0x7faf1c073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.928+0000 7f8fbcc00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8fb80c8740 0x7f8fb81421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.928+0000 7f8fbcc00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8fb80c8740 0x7f8fb81421b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37614/0 (socket says 172.21.15.114:37614) 2024-05-27T09:27:27.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.928+0000 7f8fbcc00640 1 -- 172.21.15.114:0/3941406179 learned_addr learned my addr 172.21.15.114:0/3941406179 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:27.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.928+0000 7f8fbe000640 1 --2- 172.21.15.114:0/3941406179 >> v2:172.21.15.114:3300/0 conn(0x7f8fb8142810 0x7f8fb80c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.929+0000 7f8fbcc00640 1 -- 172.21.15.114:0/3941406179 >> v2:172.21.15.114:3300/0 conn(0x7f8fb8142810 msgr2=0x7f8fb80c2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:27.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.929+0000 7f8fbcc00640 1 --2- 172.21.15.114:0/3941406179 >> v2:172.21.15.114:3300/0 conn(0x7f8fb8142810 0x7f8fb80c2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.929+0000 7f8fbcc00640 1 -- 172.21.15.114:0/3941406179 >> v1:172.21.15.114:6789/0 conn(0x7f8fb8159fe0 legacy=0x7f8fb8073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.929+0000 7f8fbcc00640 1 -- 172.21.15.114:0/3941406179 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb80748f0 con 0x7f8fb80c8740 2024-05-27T09:27:27.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.931+0000 7f8fbe000640 1 --2- 172.21.15.114:0/3941406179 >> v2:172.21.15.114:3300/0 conn(0x7f8fb8142810 0x7f8fb80c2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:27.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.932+0000 7f8fbcc00640 1 --2- 172.21.15.114:0/3941406179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fb80c8740 0x7f8fb81421b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8fb815a630 tx=0x7f8fac02a380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a500e94129b5604 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.932+0000 7f8fb7e00640 1 -- 172.21.15.114:0/3941406179 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac037070 con 0x7f8fb80c8740 2024-05-27T09:27:27.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.932+0000 7f8fb7e00640 1 -- 172.21.15.114:0/3941406179 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fac032640 con 0x7f8fb80c8740 2024-05-27T09:27:27.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.932+0000 7f8fb7e00640 1 -- 172.21.15.114:0/3941406179 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac032a10 con 0x7f8fb80c8740 2024-05-27T09:27:27.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.933+0000 7f6fa2000640 1 --2- 172.21.15.114:0/2480549801 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6f8c0762d0 0x7f6f8c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.931+0000 7fc4c8c00640 1 -- 172.21.15.114:0/4623052 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc09b070 con 0x7fc4c40d51f0 2024-05-27T09:27:27.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.932+0000 7fc4c8c00640 1 -- 172.21.15.114:0/4623052 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4cc069c70 con 0x7fc4c40d51f0 2024-05-27T09:27:27.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.933+0000 7f6fa2000640 1 -- 172.21.15.114:0/2480549801 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6f9c0979e0 con 0x7f6fa8070960 2024-05-27T09:27:27.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.933+0000 7fc4c8c00640 1 -- 172.21.15.114:0/4623052 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc096630 con 0x7fc4c40d51f0 2024-05-27T09:27:27.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.933+0000 7fa080000640 1 -- 172.21.15.114:0/1076803141 wait complete. 2024-05-27T09:27:27.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.933+0000 7faf21600640 1 -- 172.21.15.114:0/1020418385 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1c073d10 con 0x7faf1c15abc0 2024-05-27T09:27:27.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.933+0000 7f6fa2000640 1 --2- 172.21.15.114:0/2480549801 >> [v2:172.21.15.114:6816/1960550710,v1:172.21.15.114:6817/1960550710] conn(0x7f6f8c07e5a0 0x7f6f8c0809f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.933+0000 7f6fa2000640 1 -- 172.21.15.114:0/2480549801 --> [v2:172.21.15.114:6816/1960550710,v1:172.21.15.114:6817/1960550710] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6f9c03ce50 con 0x7f6f8c07e5a0 2024-05-27T09:27:27.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.933+0000 7f6fa2000640 1 -- 172.21.15.114:0/2480549801 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6f9c097dd0 con 0x7f6fa8070960 2024-05-27T09:27:27.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.933+0000 7f6fad400640 1 --2- 172.21.15.114:0/2480549801 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6f8c0762d0 0x7f6f8c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.934+0000 7f8fc0000640 1 -- 172.21.15.114:0/3941406179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fb80c8740 msgr2=0x7f8fb81421b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.933+0000 7ff422200640 1 Processor -- start 2024-05-27T09:27:27.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.934+0000 7f8fc0000640 1 --2- 172.21.15.114:0/3941406179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fb80c8740 0x7f8fb81421b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8fb815a630 tx=0x7f8fac02a380 comp rx=0 tx=0).stop 2024-05-27T09:27:27.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.934+0000 7f8fc0000640 1 -- 172.21.15.114:0/3941406179 shutdown_connections 2024-05-27T09:27:27.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.934+0000 7f8fc0000640 1 --2- 172.21.15.114:0/3941406179 >> v2:172.21.15.114:3300/0 conn(0x7f8fb8142810 0x7f8fb80c2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.934+0000 7f8fc0000640 1 --2- 172.21.15.114:0/3941406179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fb80c8740 0x7f8fb81421b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.934+0000 7f8fc0000640 1 -- 172.21.15.114:0/3941406179 >> 172.21.15.114:0/3941406179 conn(0x7f8fb806f6e0 msgr2=0x7f8fb8070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:27.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.934+0000 7f6fae800640 1 --2- 172.21.15.114:0/2480549801 >> [v2:172.21.15.114:6816/1960550710,v1:172.21.15.114:6817/1960550710] conn(0x7f6f8c07e5a0 0x7f6f8c0809f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.934+0000 7f6fad400640 1 --2- 172.21.15.114:0/2480549801 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6f8c0762d0 0x7f6f8c0787e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6fa815a320 tx=0x7f6f98009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.934+0000 7ff422200640 1 -- start start 2024-05-27T09:27:27.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.935+0000 7fc4d3400640 1 -- 172.21.15.114:0/4623052 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c40dc5d0 con 0x7fc4c40d51f0 2024-05-27T09:27:27.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.935+0000 7fc4d3400640 1 -- 172.21.15.114:0/4623052 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c40da630 con 0x7fc4c40d51f0 2024-05-27T09:27:27.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.935+0000 7f8fc0000640 1 -- 172.21.15.114:0/3941406179 shutdown_connections 2024-05-27T09:27:27.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.935+0000 7f8fc0000640 1 -- 172.21.15.114:0/3941406179 wait complete. 2024-05-27T09:27:27.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.935+0000 7f6fae800640 1 --2- 172.21.15.114:0/2480549801 >> [v2:172.21.15.114:6816/1960550710,v1:172.21.15.114:6817/1960550710] conn(0x7f6f8c07e5a0 0x7f6f8c0809f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.934+0000 7f01bbe00640 1 Processor -- start 2024-05-27T09:27:27.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.934+0000 7f01bbe00640 1 -- start start 2024-05-27T09:27:27.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.936+0000 7faf21600640 1 --2- 172.21.15.114:0/1020418385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf1c15abc0 0x7faf1c0ca2d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faf140097d0 tx=0x7faf1402a2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4ebab5444ce38a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.936+0000 7fa080000640 1 Processor -- start 2024-05-27T09:27:27.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.936+0000 7fa080000640 1 -- start start 2024-05-27T09:27:27.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.936+0000 7f01bbe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01b40c73b0 0x7f01b41f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.936+0000 7f01bbe00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01b4156760 0x7f01b415f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.937+0000 7ff422200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff41c0710e0 0x7ff41c148530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.936+0000 7faf1be00640 1 -- 172.21.15.114:0/1020418385 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf14037070 con 0x7faf1c15abc0 2024-05-27T09:27:27.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.937+0000 7ff422200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff41c0ce440 0x7ff41c15e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.936+0000 7faf1be00640 1 -- 172.21.15.114:0/1020418385 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf14032440 con 0x7faf1c15abc0 2024-05-27T09:27:27.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.936+0000 7faf1be00640 1 -- 172.21.15.114:0/1020418385 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf14032810 con 0x7faf1c15abc0 2024-05-27T09:27:27.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.937+0000 7ff422200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff41c15ad10 0x7ff41c14c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.937+0000 7ff422200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c0c2a30 con 0x7ff41c0710e0 2024-05-27T09:27:27.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.937+0000 7ff422200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c0c28b0 con 0x7ff41c0ce440 2024-05-27T09:27:27.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.937+0000 7ff422200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7ff41c074b40 con 0x7ff41c15ad10 2024-05-27T09:27:27.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.937+0000 7ff41be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff41c0ce440 0x7ff41c15e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.938+0000 7ff41be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff41c0ce440 0x7ff41c15e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37638/0 (socket says 172.21.15.114:37638) 2024-05-27T09:27:27.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.938+0000 7ff41be00640 1 -- 172.21.15.114:0/3348275336 learned_addr learned my addr 172.21.15.114:0/3348275336 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:27.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.938+0000 7faf23600640 1 -- 172.21.15.114:0/1020418385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf1c15abc0 msgr2=0x7faf1c0ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.938+0000 7faf23600640 1 --2- 172.21.15.114:0/1020418385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf1c15abc0 0x7faf1c0ca2d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faf140097d0 tx=0x7faf1402a2a0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.938+0000 7f8fc0000640 1 Processor -- start 2024-05-27T09:27:27.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.938+0000 7ff420e00640 1 --2- 172.21.15.114:0/3348275336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff41c0710e0 0x7ff41c148530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.938+0000 7ff421800640 1 --2- 172.21.15.114:0/3348275336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff41c15ad10 0x7ff41c14c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.938+0000 7f8fc0000640 1 -- start start 2024-05-27T09:27:27.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.938+0000 7faf23600640 1 -- 172.21.15.114:0/1020418385 shutdown_connections 2024-05-27T09:27:27.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.938+0000 7faf23600640 1 --2- 172.21.15.114:0/1020418385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf1c15abc0 0x7faf1c0ca2d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.938+0000 7faf23600640 1 -- 172.21.15.114:0/1020418385 >> 172.21.15.114:0/1020418385 conn(0x7faf1c06f6e0 msgr2=0x7faf1c070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:27.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.938+0000 7f6fa2000640 1 -- 172.21.15.114:0/2480549801 <== osd.0 v2:172.21.15.114:6816/1960550710 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f9c03ce50 con 0x7f6f8c07e5a0 2024-05-27T09:27:27.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.938+0000 7f01b9400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01b40c73b0 0x7f01b41f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.938+0000 7fa080000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa07808d320 0x7fa0781fd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.938+0000 7f01b9400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01b40c73b0 0x7f01b41f64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35510/0 (socket says 172.21.15.114:35510) 2024-05-27T09:27:27.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7f01b9400640 1 -- 172.21.15.114:0/272347691 learned_addr learned my addr 172.21.15.114:0/272347691 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:27.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7fa080000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0780ca300 0x7fa0781639b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7fc4d3400640 1 -- 172.21.15.114:0/4623052 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc4c40059c0 con 0x7fc4c40d51f0 2024-05-27T09:27:27.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7f01b8a00640 1 --2- 172.21.15.114:0/272347691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01b4156760 0x7f01b415f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7faf23600640 1 -- 172.21.15.114:0/1020418385 shutdown_connections 2024-05-27T09:27:27.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7f01bbe00640 1 --2- 172.21.15.114:0/272347691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01b4160860 0x7f01b41f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7fa080000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa078169450 0x7fa0781625d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7fa080000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa07816d020 con 0x7fa0780ca300 2024-05-27T09:27:27.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7fa080000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa07816cd40 con 0x7fa07808d320 2024-05-27T09:27:27.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7fa080000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa07816d300 con 0x7fa078169450 2024-05-27T09:27:27.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7fa07d600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa07808d320 0x7fa0781fd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7fa07d600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa07808d320 0x7fa0781fd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37652/0 (socket says 172.21.15.114:37652) 2024-05-27T09:27:27.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7fa07d600640 1 -- 172.21.15.114:0/1588870705 learned_addr learned my addr 172.21.15.114:0/1588870705 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:27.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7fa07e000640 1 --2- 172.21.15.114:0/1588870705 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa078169450 0x7fa0781625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7fa07cc00640 1 --2- 172.21.15.114:0/1588870705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0780ca300 0x7fa0781639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7fa07d600640 1 -- 172.21.15.114:0/1588870705 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa078169450 msgr2=0x7fa0781625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7fa07d600640 1 --2- 172.21.15.114:0/1588870705 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa078169450 0x7fa0781625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7fa07d600640 1 -- 172.21.15.114:0/1588870705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0780ca300 msgr2=0x7fa0781639b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7fa07d600640 1 --2- 172.21.15.114:0/1588870705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa0780ca300 0x7fa0781639b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7fa07d600640 1 -- 172.21.15.114:0/1588870705 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0780713c0 con 0x7fa07808d320 2024-05-27T09:27:27.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.940+0000 7f01b9e00640 1 --2- 172.21.15.114:0/272347691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01b4160860 0x7f01b41f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7ff41be00640 1 -- 172.21.15.114:0/3348275336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff41c15ad10 msgr2=0x7ff41c14c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7ff41be00640 1 --2- 172.21.15.114:0/3348275336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff41c15ad10 0x7ff41c14c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7ff41be00640 1 -- 172.21.15.114:0/3348275336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff41c0710e0 msgr2=0x7ff41c148530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7ff41be00640 1 --2- 172.21.15.114:0/3348275336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff41c0710e0 0x7ff41c148530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7ff41be00640 1 -- 172.21.15.114:0/3348275336 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41c14c7f0 con 0x7ff41c0ce440 2024-05-27T09:27:27.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7ff41be00640 1 --2- 172.21.15.114:0/3348275336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff41c0ce440 0x7ff41c15e770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00c730 tx=0x7ff40c00cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.940+0000 7ff418c00640 1 -- 172.21.15.114:0/3348275336 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c017070 con 0x7ff41c0ce440 2024-05-27T09:27:27.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.940+0000 7fa07d600640 1 --2- 172.21.15.114:0/1588870705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa07808d320 0x7fa0781fd2a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa06400b9f0 tx=0x7fa06400beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.940+0000 7ff418c00640 1 -- 172.21.15.114:0/3348275336 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff40c005270 con 0x7ff41c0ce440 2024-05-27T09:27:27.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.940+0000 7ff418c00640 1 -- 172.21.15.114:0/3348275336 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c00a640 con 0x7ff41c0ce440 2024-05-27T09:27:27.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.940+0000 7ff422200640 1 -- 172.21.15.114:0/3348275336 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41c14d3c0 con 0x7ff41c0ce440 2024-05-27T09:27:27.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.940+0000 7ff422200640 1 -- 172.21.15.114:0/3348275336 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff41c1faed0 con 0x7ff41c0ce440 2024-05-27T09:27:27.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.940+0000 7fa075600640 1 -- 172.21.15.114:0/1588870705 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06401b070 con 0x7fa07808d320 2024-05-27T09:27:27.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.940+0000 7fa075600640 1 -- 172.21.15.114:0/1588870705 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa06400f890 con 0x7fa07808d320 2024-05-27T09:27:27.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.940+0000 7fa075600640 1 -- 172.21.15.114:0/1588870705 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06400fc80 con 0x7fa07808d320 2024-05-27T09:27:27.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.940+0000 7fa080000640 1 -- 172.21.15.114:0/1588870705 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa078163ef0 con 0x7fa07808d320 2024-05-27T09:27:27.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.940+0000 7f01bbe00640 1 -- 172.21.15.114:0/272347691 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f01b40b4a00 con 0x7f01b40c73b0 2024-05-27T09:27:27.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.940+0000 7f01bbe00640 1 -- 172.21.15.114:0/272347691 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f01b4160340 con 0x7f01b4160860 2024-05-27T09:27:27.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.940+0000 7f01bbe00640 1 -- 172.21.15.114:0/272347691 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f01b41f2950 con 0x7f01b4156760 2024-05-27T09:27:27.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.940+0000 7fa080000640 1 -- 172.21.15.114:0/1588870705 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0781fb4f0 con 0x7fa07808d320 2024-05-27T09:27:27.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.940+0000 7ff420e00640 1 --2- 172.21.15.114:0/3348275336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff41c0710e0 0x7ff41c148530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:27.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.941+0000 7f8fc0000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fb80c8740 0x7f8fb81ffb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.941+0000 7fc4c8c00640 1 -- 172.21.15.114:0/4623052 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc4cc09e070 con 0x7fc4c40d51f0 2024-05-27T09:27:27.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.941+0000 7f01b9400640 1 -- 172.21.15.114:0/272347691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01b4156760 msgr2=0x7f01b415f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.941+0000 7f01b9400640 1 --2- 172.21.15.114:0/272347691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01b4156760 0x7f01b415f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.941+0000 7f01b9400640 1 -- 172.21.15.114:0/272347691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01b4160860 msgr2=0x7f01b41f2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.941+0000 7f01b9400640 1 --2- 172.21.15.114:0/272347691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01b4160860 0x7f01b41f2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.941+0000 7f01b9400640 1 -- 172.21.15.114:0/272347691 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b41f6a00 con 0x7f01b40c73b0 2024-05-27T09:27:27.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.941+0000 7f01b9e00640 1 --2- 172.21.15.114:0/272347691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01b4160860 0x7f01b41f2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:27:27.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.939+0000 7faf23600640 1 -- 172.21.15.114:0/1020418385 wait complete. 2024-05-27T09:27:27.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.942+0000 7fa080000640 1 -- 172.21.15.114:0/1588870705 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa054001030 con 0x7fa07808d320 2024-05-27T09:27:27.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.942+0000 7f01b9400640 1 --2- 172.21.15.114:0/272347691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01b40c73b0 0x7f01b41f64c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009990 tx=0x7f01a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.942+0000 7f8fc0000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fb8142810 0x7f8fb81e1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.942+0000 7f8fc0000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fb8159fe0 0x7f8fb82003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.942+0000 7f8fc0000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb80735e0 con 0x7f8fb8159fe0 2024-05-27T09:27:27.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.942+0000 7f8fc0000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb8073300 con 0x7f8fb8142810 2024-05-27T09:27:27.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.942+0000 7f8fc0000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb80c4b80 con 0x7f8fb80c8740 2024-05-27T09:27:27.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.943+0000 7fc4c8c00640 1 --2- 172.21.15.114:0/4623052 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc4ac076320 0x7fc4ac078830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.943+0000 7f01b1600640 1 -- 172.21.15.114:0/272347691 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8046070 con 0x7f01b40c73b0 2024-05-27T09:27:27.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.943+0000 7faf23600640 1 Processor -- start 2024-05-27T09:27:27.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.943+0000 7f8fbe000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fb8159fe0 0x7f8fb82003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.943+0000 7f8fbe000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fb8159fe0 0x7f8fb82003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35520/0 (socket says 172.21.15.114:35520) 2024-05-27T09:27:27.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.943+0000 7f8fbe000640 1 -- 172.21.15.114:0/3784657571 learned_addr learned my addr 172.21.15.114:0/3784657571 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:27.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.943+0000 7fc4c8c00640 1 -- 172.21.15.114:0/4623052 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc4cc113be0 con 0x7fc4c40d51f0 2024-05-27T09:27:27.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.943+0000 7faf23600640 1 -- start start 2024-05-27T09:27:27.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.943+0000 7fc4d0a00640 1 --2- 172.21.15.114:0/4623052 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc4ac076320 0x7fc4ac078830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.943+0000 7fc4c8c00640 1 --2- 172.21.15.114:0/4623052 >> [v2:172.21.15.114:6823/326580057,v1:172.21.15.114:6825/326580057] conn(0x7fc4ac07e5f0 0x7fc4ac080a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.943+0000 7fc4c8c00640 1 -- 172.21.15.114:0/4623052 --> [v2:172.21.15.114:6823/326580057,v1:172.21.15.114:6825/326580057] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc4cc06fe80 con 0x7fc4ac07e5f0 2024-05-27T09:27:27.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.943+0000 7fc4c8c00640 1 -- 172.21.15.114:0/4623052 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc4cc116330 con 0x7fc4c40d51f0 2024-05-27T09:27:27.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.944+0000 7f8fbcc00640 1 --2- 172.21.15.114:0/3784657571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fb8142810 0x7f8fb81e1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.944+0000 7f8fbd600640 1 --2- 172.21.15.114:0/3784657571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fb80c8740 0x7f8fb81ffb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.944+0000 7fc4cbe00640 1 --2- 172.21.15.114:0/4623052 >> [v2:172.21.15.114:6823/326580057,v1:172.21.15.114:6825/326580057] conn(0x7fc4ac07e5f0 0x7fc4ac080a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.943+0000 7f01bbe00640 1 -- 172.21.15.114:0/272347691 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b40c9a30 con 0x7f01b40c73b0 2024-05-27T09:27:27.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.943+0000 7f01b1600640 1 -- 172.21.15.114:0/272347691 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a8040d20 con 0x7f01b40c73b0 2024-05-27T09:27:27.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.943+0000 7f01b1600640 1 -- 172.21.15.114:0/272347691 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8046070 con 0x7f01b40c73b0 2024-05-27T09:27:27.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.943+0000 7f01bbe00640 1 -- 172.21.15.114:0/272347691 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b40c9280 con 0x7f01b40c73b0 2024-05-27T09:27:27.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.944+0000 7fc4d0a00640 1 --2- 172.21.15.114:0/4623052 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc4ac076320 0x7fc4ac078830 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0034000 tx=0x7fc4c002c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.945+0000 7f8fbe000640 1 -- 172.21.15.114:0/3784657571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fb80c8740 msgr2=0x7f8fb81ffb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.945+0000 7f8fbe000640 1 --2- 172.21.15.114:0/3784657571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fb80c8740 0x7f8fb81ffb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.945+0000 7f8fbe000640 1 -- 172.21.15.114:0/3784657571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fb8142810 msgr2=0x7f8fb81e1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.945+0000 7f8fbe000640 1 --2- 172.21.15.114:0/3784657571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fb8142810 0x7f8fb81e1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.945+0000 7f8fbe000640 1 -- 172.21.15.114:0/3784657571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb81e1560 con 0x7f8fb8159fe0 2024-05-27T09:27:27.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.945+0000 7ff422200640 1 -- 172.21.15.114:0/3348275336 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff41c050860 con 0x7ff41c0ce440 2024-05-27T09:27:27.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.945+0000 7f8fbcc00640 1 --2- 172.21.15.114:0/3784657571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fb8142810 0x7f8fb81e1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:27:27.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.945+0000 7f8fbd600640 1 --2- 172.21.15.114:0/3784657571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fb80c8740 0x7f8fb81ffb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:27:27.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.946+0000 7faf23600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf1c0c39b0 0x7faf1c1ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.946+0000 7fc4cbe00640 1 --2- 172.21.15.114:0/4623052 >> [v2:172.21.15.114:6823/326580057,v1:172.21.15.114:6825/326580057] conn(0x7fc4ac07e5f0 0x7fc4ac080a40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.946+0000 7faf23600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf1c15abc0 0x7faf1c15d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.946+0000 7fa075600640 1 -- 172.21.15.114:0/1588870705 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa06401e360 con 0x7fa07808d320 2024-05-27T09:27:27.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.947+0000 7faf23600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf1c161800 0x7faf1c151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.947+0000 7faf23600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c1612e0 con 0x7faf1c15abc0 2024-05-27T09:27:27.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.947+0000 7faf20c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf1c0c39b0 0x7faf1c1ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.947+0000 7faf23600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c161000 con 0x7faf1c0c39b0 2024-05-27T09:27:27.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.947+0000 7f01bbe00640 1 -- 172.21.15.114:0/272347691 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f01b406a380 con 0x7f01b40c73b0 2024-05-27T09:27:27.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.947+0000 7faf13e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf1c15abc0 0x7faf1c15d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.947+0000 7faf23600640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7faf1c0c3dc0 con 0x7faf1c161800 2024-05-27T09:27:27.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.947+0000 7faf21600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf1c161800 0x7faf1c151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.948+0000 7faf20c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf1c0c39b0 0x7faf1c1ffdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37676/0 (socket says 172.21.15.114:37676) 2024-05-27T09:27:27.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.948+0000 7faf20c00640 1 -- 172.21.15.114:0/129815001 learned_addr learned my addr 172.21.15.114:0/129815001 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:27.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.947+0000 7faf13e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf1c15abc0 0x7faf1c15d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35530/0 (socket says 172.21.15.114:35530) 2024-05-27T09:27:27.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.947+0000 7f8fbe000640 1 --2- 172.21.15.114:0/3784657571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fb8159fe0 0x7f8fb82003a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009f30 tx=0x7f8fa400e600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.948+0000 7f8fb5600640 1 -- 172.21.15.114:0/3784657571 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa400f040 con 0x7f8fb8159fe0 2024-05-27T09:27:27.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.948+0000 7f01b1600640 1 -- 172.21.15.114:0/272347691 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f01a8051060 con 0x7f01b40c73b0 2024-05-27T09:27:27.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.948+0000 7f8fb5600640 1 -- 172.21.15.114:0/3784657571 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa40133f0 con 0x7f8fb8159fe0 2024-05-27T09:27:27.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.948+0000 7f8fb5600640 1 -- 172.21.15.114:0/3784657571 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4004020 con 0x7f8fb8159fe0 2024-05-27T09:27:27.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.948+0000 7f8fc0000640 1 -- 172.21.15.114:0/3784657571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb81df9f0 con 0x7f8fb8159fe0 2024-05-27T09:27:27.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.948+0000 7f4218800640 1 Processor -- start 2024-05-27T09:27:27.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.948+0000 7fc4c8c00640 1 -- 172.21.15.114:0/4623052 <== osd.1 v2:172.21.15.114:6823/326580057 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4cc06fe80 con 0x7fc4ac07e5f0 2024-05-27T09:27:27.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.948+0000 7fa075600640 1 --2- 172.21.15.114:0/1588870705 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa0580763d0 0x7fa0580788e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.948+0000 7fa075600640 1 -- 172.21.15.114:0/1588870705 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa064096ea0 con 0x7fa07808d320 2024-05-27T09:27:27.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.949+0000 7fa07cc00640 1 --2- 172.21.15.114:0/1588870705 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa0580763d0 0x7fa0580788e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.949+0000 7fa075600640 1 --2- 172.21.15.114:0/1588870705 >> [v2:172.21.15.114:6808/865568619,v1:172.21.15.114:6809/865568619] conn(0x7fa05807e6a0 0x7fa058080af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.949+0000 7fa075600640 1 -- 172.21.15.114:0/1588870705 --> [v2:172.21.15.114:6808/865568619,v1:172.21.15.114:6809/865568619] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa064008e50 con 0x7fa05807e6a0 2024-05-27T09:27:27.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.949+0000 7fa075600640 1 -- 172.21.15.114:0/1588870705 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa064097290 con 0x7fa07808d320 2024-05-27T09:27:27.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.949+0000 7fa07cc00640 1 --2- 172.21.15.114:0/1588870705 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa0580763d0 0x7fa0580788e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa06c034000 tx=0x7fa06c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.949+0000 7fa07e000640 1 --2- 172.21.15.114:0/1588870705 >> [v2:172.21.15.114:6808/865568619,v1:172.21.15.114:6809/865568619] conn(0x7fa05807e6a0 0x7fa058080af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.949+0000 7f01b1600640 1 --2- 172.21.15.114:0/272347691 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f01940764d0 0x7f01940789e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.948+0000 7f8fc0000640 1 -- 172.21.15.114:0/3784657571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb81dffb0 con 0x7f8fb8159fe0 2024-05-27T09:27:27.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.948+0000 7f4218800640 1 -- start start 2024-05-27T09:27:27.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.950+0000 7f01b8a00640 1 --2- 172.21.15.114:0/272347691 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f01940764d0 0x7f01940789e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.947+0000 7ff418c00640 1 -- 172.21.15.114:0/3348275336 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff40c01a070 con 0x7ff41c0ce440 2024-05-27T09:27:27.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.950+0000 7faf20c00640 1 -- 172.21.15.114:0/129815001 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf1c161800 msgr2=0x7faf1c151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.950+0000 7faf20c00640 1 --2- 172.21.15.114:0/129815001 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf1c161800 0x7faf1c151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.950+0000 7faf20c00640 1 -- 172.21.15.114:0/129815001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf1c15abc0 msgr2=0x7faf1c15d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.950+0000 7ff418c00640 1 --2- 172.21.15.114:0/3348275336 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff3f8076320 0x7ff3f8078830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.950+0000 7faf20c00640 1 --2- 172.21.15.114:0/129815001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf1c15abc0 0x7faf1c15d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.950+0000 7faf13e00640 1 --2- 172.21.15.114:0/129815001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf1c15abc0 0x7faf1c15d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:27:27.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.950+0000 7f01b1600640 1 -- 172.21.15.114:0/272347691 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f01a80bf5f0 con 0x7f01b40c73b0 2024-05-27T09:27:27.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.950+0000 7f01b1600640 1 --2- 172.21.15.114:0/272347691 >> [v2:172.21.15.114:6800/1601792164,v1:172.21.15.114:6801/1601792164] conn(0x7f019407e6c0 0x7f0194080b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.950+0000 7f01b1600640 1 -- 172.21.15.114:0/272347691 --> [v2:172.21.15.114:6800/1601792164,v1:172.21.15.114:6801/1601792164] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01a802eec0 con 0x7f019407e6c0 2024-05-27T09:27:27.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.950+0000 7f01b1600640 1 -- 172.21.15.114:0/272347691 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f01a80c6050 con 0x7f01b40c73b0 2024-05-27T09:27:27.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.951+0000 7f01b9e00640 1 --2- 172.21.15.114:0/272347691 >> [v2:172.21.15.114:6800/1601792164,v1:172.21.15.114:6801/1601792164] conn(0x7f019407e6c0 0x7f0194080b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.951+0000 7f01b8a00640 1 --2- 172.21.15.114:0/272347691 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f01940764d0 0x7f01940789e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f01b41f6d00 tx=0x7f01a0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.951+0000 7ff418c00640 1 -- 172.21.15.114:0/3348275336 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff40c097320 con 0x7ff41c0ce440 2024-05-27T09:27:27.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.951+0000 7ff418c00640 1 --2- 172.21.15.114:0/3348275336 >> [v2:172.21.15.163:6816/1982246015,v1:172.21.15.163:6817/1982246015] conn(0x7ff3f807e5f0 0x7ff3f8080a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.951+0000 7ff418c00640 1 -- 172.21.15.114:0/3348275336 --> [v2:172.21.15.163:6816/1982246015,v1:172.21.15.163:6817/1982246015] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff40c01fe70 con 0x7ff3f807e5f0 2024-05-27T09:27:27.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.951+0000 7ff418c00640 1 -- 172.21.15.114:0/3348275336 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff40c0976d0 con 0x7ff41c0ce440 2024-05-27T09:27:27.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.951+0000 7faf20c00640 1 -- 172.21.15.114:0/129815001 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1c200300 con 0x7faf1c0c39b0 2024-05-27T09:27:27.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.952+0000 7fa07e000640 1 --2- 172.21.15.114:0/1588870705 >> [v2:172.21.15.114:6808/865568619,v1:172.21.15.114:6809/865568619] conn(0x7fa05807e6a0 0x7fa058080af0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.951+0000 7ff421800640 1 --2- 172.21.15.114:0/3348275336 >> [v2:172.21.15.163:6816/1982246015,v1:172.21.15.163:6817/1982246015] conn(0x7ff3f807e5f0 0x7ff3f8080a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.951+0000 7ff420e00640 1 --2- 172.21.15.114:0/3348275336 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff3f8076320 0x7ff3f8078830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.952+0000 7f8fc0000640 1 -- 172.21.15.114:0/3784657571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8fb806a380 con 0x7f8fb8159fe0 2024-05-27T09:27:27.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.952+0000 7f4218800640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f42100cac20 0x7f4210154770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.952+0000 7f4218800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f421015df70 0x7f421015e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.952+0000 7f4218800640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f421015b560 con 0x7f42100ca5a0 2024-05-27T09:27:27.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.952+0000 7f4218800640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f42100c54b0 con 0x7f421015df70 2024-05-27T09:27:27.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.952+0000 7f4218800640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f42100c5790 con 0x7f42100cac20 2024-05-27T09:27:27.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.952+0000 7f01b9e00640 1 --2- 172.21.15.114:0/272347691 >> [v2:172.21.15.114:6800/1601792164,v1:172.21.15.114:6801/1601792164] conn(0x7f019407e6c0 0x7f0194080b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.952+0000 7ff421800640 1 --2- 172.21.15.114:0/3348275336 >> [v2:172.21.15.163:6816/1982246015,v1:172.21.15.163:6817/1982246015] conn(0x7ff3f807e5f0 0x7ff3f8080a40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.952+0000 7faf20c00640 1 --2- 172.21.15.114:0/129815001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf1c0c39b0 0x7faf1c1ffdc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00c800 tx=0x7faf0c00ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.952+0000 7faf19600640 1 -- 172.21.15.114:0/129815001 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c017070 con 0x7faf1c0c39b0 2024-05-27T09:27:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.952+0000 7faf19600640 1 -- 172.21.15.114:0/129815001 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0c00a640 con 0x7faf1c0c39b0 2024-05-27T09:27:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.952+0000 7faf19600640 1 -- 172.21.15.114:0/129815001 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c00a9f0 con 0x7faf1c0c39b0 2024-05-27T09:27:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.952+0000 7f4215400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f42100cac20 0x7f4210154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.953+0000 7f4215400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f42100cac20 0x7f4210154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37692/0 (socket says 172.21.15.114:37692) 2024-05-27T09:27:27.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.953+0000 7f4215400640 1 -- 172.21.15.114:0/449497141 learned_addr learned my addr 172.21.15.114:0/449497141 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:27.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.953+0000 7f4216800640 1 --2- 172.21.15.114:0/449497141 >> v2:172.21.15.114:3300/0 conn(0x7f421015df70 0x7f421015e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.953+0000 7ff418c00640 1 -- 172.21.15.114:0/3348275336 <== osd.4 v2:172.21.15.163:6816/1982246015 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff40c01fe70 con 0x7ff3f807e5f0 2024-05-27T09:27:27.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.953+0000 7f4216800640 1 -- 172.21.15.114:0/449497141 >> v2:172.21.15.163:3300/0 conn(0x7f42100cac20 msgr2=0x7f4210154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:27.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.953+0000 7f4216800640 1 --2- 172.21.15.114:0/449497141 >> v2:172.21.15.163:3300/0 conn(0x7f42100cac20 0x7f4210154770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:27.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.953+0000 7f4216800640 1 -- 172.21.15.114:0/449497141 >> v1:172.21.15.114:6789/0 conn(0x7f42100ca5a0 legacy=0x7f42100cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:27.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.954+0000 7fa075600640 1 -- 172.21.15.114:0/1588870705 <== osd.2 v2:172.21.15.114:6808/865568619 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa064008e50 con 0x7fa05807e6a0 2024-05-27T09:27:27.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.953+0000 7faf23600640 1 -- 172.21.15.114:0/129815001 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1c203770 con 0x7faf1c0c39b0 2024-05-27T09:27:27.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.953+0000 7faf23600640 1 -- 172.21.15.114:0/129815001 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf1c2034a0 con 0x7faf1c0c39b0 2024-05-27T09:27:27.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.954+0000 7f4216800640 1 -- 172.21.15.114:0/449497141 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42100c5a70 con 0x7f421015df70 2024-05-27T09:27:27.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.954+0000 7f01b1600640 1 -- 172.21.15.114:0/272347691 <== osd.3 v2:172.21.15.114:6800/1601792164 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f01a802eec0 con 0x7f019407e6c0 2024-05-27T09:27:27.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.954+0000 7ff420e00640 1 --2- 172.21.15.114:0/3348275336 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff3f8076320 0x7ff3f8078830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff414034000 tx=0x7ff41402f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.953+0000 7f8fb5600640 1 -- 172.21.15.114:0/3784657571 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8fa40041c0 con 0x7f8fb8159fe0 2024-05-27T09:27:27.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.955+0000 7f8fb5600640 1 --2- 172.21.15.114:0/3784657571 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f8f980763d0 0x7f8f980788e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.955+0000 7f8fb5600640 1 -- 172.21.15.114:0/3784657571 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8fa4097aa0 con 0x7f8fb8159fe0 2024-05-27T09:27:27.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.955+0000 7f8fb5600640 1 --2- 172.21.15.114:0/3784657571 >> [v2:172.21.15.163:6824/2539451311,v1:172.21.15.163:6825/2539451311] conn(0x7f8f9807e6a0 0x7f8f98080af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:27.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.955+0000 7f8fb5600640 1 -- 172.21.15.114:0/3784657571 --> [v2:172.21.15.163:6824/2539451311,v1:172.21.15.163:6825/2539451311] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8fa40044b0 con 0x7f8f9807e6a0 2024-05-27T09:27:27.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.955+0000 7f8fb5600640 1 -- 172.21.15.114:0/3784657571 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8fa409c2a0 con 0x7f8fb8159fe0 2024-05-27T09:27:27.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.956+0000 7f8fbcc00640 1 --2- 172.21.15.114:0/3784657571 >> [v2:172.21.15.163:6824/2539451311,v1:172.21.15.163:6825/2539451311] conn(0x7f8f9807e6a0 0x7f8f98080af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.957+0000 7f8fbcc00640 1 --2- 172.21.15.114:0/3784657571 >> [v2:172.21.15.163:6824/2539451311,v1:172.21.15.163:6825/2539451311] conn(0x7f8f9807e6a0 0x7f8f98080af0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.955+0000 7faf23600640 1 -- 172.21.15.114:0/129815001 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faf1c0692a0 con 0x7faf1c0c39b0 2024-05-27T09:27:27.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.957+0000 7f8fbd600640 1 --2- 172.21.15.114:0/3784657571 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f8f980763d0 0x7f8f980788e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:27.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.957+0000 7f4216800640 1 --2- 172.21.15.114:0/449497141 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f421015df70 0x7f421015e380 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f42100d0a90 tx=0x7f420c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a999234ee3585cfe server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.958+0000 7f8fb5600640 1 -- 172.21.15.114:0/3784657571 <== osd.7 v2:172.21.15.163:6824/2539451311 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8fa40044b0 con 0x7f8f9807e6a0 2024-05-27T09:27:27.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.958+0000 7f4214a00640 1 -- 172.21.15.114:0/449497141 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c037070 con 0x7f421015df70 2024-05-27T09:27:27.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.958+0000 7f4214a00640 1 -- 172.21.15.114:0/449497141 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f420c032440 con 0x7f421015df70 2024-05-27T09:27:27.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.958+0000 7f4214a00640 1 -- 172.21.15.114:0/449497141 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c032850 con 0x7f421015df70 2024-05-27T09:27:27.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.959+0000 7f8fbd600640 1 --2- 172.21.15.114:0/3784657571 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f8f980763d0 0x7f8f980788e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0001e80 tx=0x7f8fb0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:27.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.959+0000 7f4218800640 1 -- 172.21.15.114:0/449497141 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f421015df70 msgr2=0x7f421015e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:27.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.959+0000 7f4218800640 1 --2- 172.21.15.114:0/449497141 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f421015df70 0x7f421015e380 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f42100d0a90 tx=0x7f420c02a130 comp rx=0 tx=0).stop 2024-05-27T09:27:27.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.961+0000 7faf19600640 1 -- 172.21.15.114:0/129815001 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7faf0c01a070 con 0x7faf1c0c39b0 2024-05-27T09:27:28.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.961+0000 7f4218800640 1 -- 172.21.15.114:0/449497141 shutdown_connections 2024-05-27T09:27:28.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.961+0000 7f4218800640 1 --2- 172.21.15.114:0/449497141 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f421015df70 0x7f421015e380 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.961+0000 7f4218800640 1 --2- 172.21.15.114:0/449497141 >> v2:172.21.15.163:3300/0 conn(0x7f42100cac20 0x7f4210154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.961+0000 7f4218800640 1 -- 172.21.15.114:0/449497141 >> 172.21.15.114:0/449497141 conn(0x7f4210071660 msgr2=0x7f4210068600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.961+0000 7f4218800640 1 -- 172.21.15.114:0/449497141 shutdown_connections 2024-05-27T09:27:28.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.961+0000 7f4218800640 1 -- 172.21.15.114:0/449497141 wait complete. 2024-05-27T09:27:28.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.962+0000 7faf19600640 1 --2- 172.21.15.114:0/129815001 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7faef80763d0 0x7faef80788e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.962+0000 7faf19600640 1 -- 172.21.15.114:0/129815001 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7faf0c096e20 con 0x7faf1c0c39b0 2024-05-27T09:27:28.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.963+0000 7faf13e00640 1 --2- 172.21.15.114:0/129815001 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7faef80763d0 0x7faef80788e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.963+0000 7faf19600640 1 --2- 172.21.15.114:0/129815001 >> [v2:172.21.15.163:6800/1695294004,v1:172.21.15.163:6801/1695294004] conn(0x7faef807e640 0x7faef8080a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.963+0000 7faf19600640 1 -- 172.21.15.114:0/129815001 --> [v2:172.21.15.163:6800/1695294004,v1:172.21.15.163:6801/1695294004] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faef8081160 con 0x7faef807e640 2024-05-27T09:27:28.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.963+0000 7faf19600640 1 -- 172.21.15.114:0/129815001 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7faf0c097210 con 0x7faf1c0c39b0 2024-05-27T09:27:28.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.963+0000 7faf13e00640 1 --2- 172.21.15.114:0/129815001 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7faef80763d0 0x7faef80788e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faf1c160920 tx=0x7faf04009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.963+0000 7f4218800640 1 Processor -- start 2024-05-27T09:27:28.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.963+0000 7faf21600640 1 --2- 172.21.15.114:0/129815001 >> [v2:172.21.15.163:6800/1695294004,v1:172.21.15.163:6801/1695294004] conn(0x7faef807e640 0x7faef8080a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.964+0000 7faf21600640 1 --2- 172.21.15.114:0/129815001 >> [v2:172.21.15.163:6800/1695294004,v1:172.21.15.163:6801/1695294004] conn(0x7faef807e640 0x7faef8080a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.963+0000 7f4218800640 1 -- start start 2024-05-27T09:27:28.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.965+0000 7f4218800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42100ca5a0 0x7f42100cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.965+0000 7f4218800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42100cac20 0x7f4210145790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.965+0000 7f4218800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f421015df70 0x7f42100c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.966+0000 7f4218800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f421015e8c0 con 0x7f421015df70 2024-05-27T09:27:28.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.966+0000 7f4218800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f42100c60c0 con 0x7f42100ca5a0 2024-05-27T09:27:28.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.966+0000 7f4218800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f42100c6ad0 con 0x7f42100cac20 2024-05-27T09:27:28.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.966+0000 7faf19600640 1 -- 172.21.15.114:0/129815001 <== osd.5 v2:172.21.15.163:6800/1695294004 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faef8081160 con 0x7faef807e640 2024-05-27T09:27:28.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.966+0000 7f4216800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f421015df70 0x7f42100c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.966+0000 7f4216800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f421015df70 0x7f42100c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35550/0 (socket says 172.21.15.114:35550) 2024-05-27T09:27:28.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.966+0000 7f4216800640 1 -- 172.21.15.114:0/3306782595 learned_addr learned my addr 172.21.15.114:0/3306782595 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:28.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.966+0000 7f4216800640 1 -- 172.21.15.114:0/3306782595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42100cac20 msgr2=0x7f4210145790 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T09:27:28.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.966+0000 7f4216800640 1 --2- 172.21.15.114:0/3306782595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42100cac20 0x7f4210145790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.966+0000 7f4216800640 1 -- 172.21.15.114:0/3306782595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42100ca5a0 msgr2=0x7f42100cfaf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T09:27:28.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.966+0000 7f4216800640 1 --2- 172.21.15.114:0/3306782595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42100ca5a0 0x7f42100cfaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.966+0000 7f4216800640 1 -- 172.21.15.114:0/3306782595 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42100d0030 con 0x7f421015df70 2024-05-27T09:27:28.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.967+0000 7f4216800640 1 --2- 172.21.15.114:0/3306782595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f421015df70 0x7f42100c6dd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f420c02a720 tx=0x7f420c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.967+0000 7f420a000640 1 -- 172.21.15.114:0/3306782595 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c037050 con 0x7f421015df70 2024-05-27T09:27:28.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.967+0000 7f420a000640 1 -- 172.21.15.114:0/3306782595 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f420c038040 con 0x7f421015df70 2024-05-27T09:27:28.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.967+0000 7f420a000640 1 -- 172.21.15.114:0/3306782595 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c030aa0 con 0x7f421015df70 2024-05-27T09:27:28.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.967+0000 7f4218800640 1 -- 172.21.15.114:0/3306782595 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42101f6d20 con 0x7f421015df70 2024-05-27T09:27:28.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.968+0000 7f4218800640 1 -- 172.21.15.114:0/3306782595 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42101f72e0 con 0x7f421015df70 2024-05-27T09:27:28.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.970+0000 7f4218800640 1 -- 172.21.15.114:0/3306782595 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f421006a380 con 0x7f421015df70 2024-05-27T09:27:28.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.970+0000 7f420a000640 1 -- 172.21.15.114:0/3306782595 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f420c045020 con 0x7f421015df70 2024-05-27T09:27:28.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.972+0000 7f420a000640 1 --2- 172.21.15.114:0/3306782595 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f41f4076320 0x7f41f4078830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.972+0000 7f420a000640 1 -- 172.21.15.114:0/3306782595 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f420c0b7bf0 con 0x7f421015df70 2024-05-27T09:27:28.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.972+0000 7f420a000640 1 --2- 172.21.15.114:0/3306782595 >> [v2:172.21.15.163:6808/3649718505,v1:172.21.15.163:6809/3649718505] conn(0x7f41f407e590 0x7f41f40809e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.972+0000 7f420a000640 1 -- 172.21.15.114:0/3306782595 --> [v2:172.21.15.163:6808/3649718505,v1:172.21.15.163:6809/3649718505] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f420c02ce50 con 0x7f41f407e590 2024-05-27T09:27:28.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.972+0000 7f420a000640 1 -- 172.21.15.114:0/3306782595 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f420c0bc2a0 con 0x7f421015df70 2024-05-27T09:27:28.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.975+0000 7f4215e00640 1 --2- 172.21.15.114:0/3306782595 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f41f4076320 0x7f41f4078830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.975+0000 7f4215400640 1 --2- 172.21.15.114:0/3306782595 >> [v2:172.21.15.163:6808/3649718505,v1:172.21.15.163:6809/3649718505] conn(0x7f41f407e590 0x7f41f40809e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.976+0000 7f4215400640 1 --2- 172.21.15.114:0/3306782595 >> [v2:172.21.15.163:6808/3649718505,v1:172.21.15.163:6809/3649718505] conn(0x7f41f407e590 0x7f41f40809e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.976+0000 7f4215e00640 1 --2- 172.21.15.114:0/3306782595 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f41f4076320 0x7f41f4078830 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4204001b80 tx=0x7f4204007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.977+0000 7f420a000640 1 -- 172.21.15.114:0/3306782595 <== osd.6 v2:172.21.15.163:6808/3649718505 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f420c02ce50 con 0x7f41f407e590 2024-05-27T09:27:28.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.979+0000 7f6fb0800640 1 -- 172.21.15.114:0/2480549801 --> [v2:172.21.15.114:6816/1960550710,v1:172.21.15.114:6817/1960550710] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6f84002dd0 con 0x7f6f8c07e5a0 2024-05-27T09:27:28.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.979+0000 7f6fa2000640 1 -- 172.21.15.114:0/2480549801 <== osd.0 v2:172.21.15.114:6816/1960550710 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6f84002dd0 con 0x7f6f8c07e5a0 2024-05-27T09:27:28.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.980+0000 7f6fb0800640 1 -- 172.21.15.114:0/2480549801 >> [v2:172.21.15.114:6816/1960550710,v1:172.21.15.114:6817/1960550710] conn(0x7f6f8c07e5a0 msgr2=0x7f6f8c0809f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.980+0000 7f6fb0800640 1 --2- 172.21.15.114:0/2480549801 >> [v2:172.21.15.114:6816/1960550710,v1:172.21.15.114:6817/1960550710] conn(0x7f6f8c07e5a0 0x7f6f8c0809f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.981+0000 7f6fb0800640 1 -- 172.21.15.114:0/2480549801 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6f8c0762d0 msgr2=0x7f6f8c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.981+0000 7f6fb0800640 1 --2- 172.21.15.114:0/2480549801 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6f8c0762d0 0x7f6f8c0787e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6fa815a320 tx=0x7f6f98009040 comp rx=0 tx=0).stop 2024-05-27T09:27:28.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.981+0000 7f6fb0800640 1 -- 172.21.15.114:0/2480549801 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6fa8070960 msgr2=0x7f6fa8161f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.981+0000 7f6fb0800640 1 --2- 172.21.15.114:0/2480549801 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6fa8070960 0x7f6fa8161f10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00b3c0 tx=0x7f6f9c00b880 comp rx=0 tx=0).stop 2024-05-27T09:27:28.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.981+0000 7f6fae800640 1 -- 172.21.15.114:0/2480549801 reap_dead start 2024-05-27T09:27:28.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.981+0000 7fc4d3400640 1 -- 172.21.15.114:0/4623052 --> [v2:172.21.15.114:6823/326580057,v1:172.21.15.114:6825/326580057] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc4c40d5b00 con 0x7fc4ac07e5f0 2024-05-27T09:27:28.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.982+0000 7f6fb0800640 1 -- 172.21.15.114:0/2480549801 shutdown_connections 2024-05-27T09:27:28.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.982+0000 7f6fb0800640 1 -- 172.21.15.114:0/2480549801 >> 172.21.15.114:0/2480549801 conn(0x7f6fa80cd890 msgr2=0x7f6fa81567c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.982+0000 7fc4c8c00640 1 -- 172.21.15.114:0/4623052 <== osd.1 v2:172.21.15.114:6823/326580057 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc4c40d5b00 con 0x7fc4ac07e5f0 2024-05-27T09:27:28.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.982+0000 7fc4d3400640 1 -- 172.21.15.114:0/4623052 >> [v2:172.21.15.114:6823/326580057,v1:172.21.15.114:6825/326580057] conn(0x7fc4ac07e5f0 msgr2=0x7fc4ac080a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.982+0000 7fc4d3400640 1 --2- 172.21.15.114:0/4623052 >> [v2:172.21.15.114:6823/326580057,v1:172.21.15.114:6825/326580057] conn(0x7fc4ac07e5f0 0x7fc4ac080a40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.983+0000 7f6fb0800640 1 -- 172.21.15.114:0/2480549801 shutdown_connections 2024-05-27T09:27:28.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.983+0000 7fc4d3400640 1 -- 172.21.15.114:0/4623052 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc4ac076320 msgr2=0x7fc4ac078830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.983+0000 7fc4d3400640 1 --2- 172.21.15.114:0/4623052 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc4ac076320 0x7fc4ac078830 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0034000 tx=0x7fc4c002c000 comp rx=0 tx=0).stop 2024-05-27T09:27:28.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.983+0000 7fc4d3400640 1 -- 172.21.15.114:0/4623052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c40d51f0 msgr2=0x7fc4c40dd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.983+0000 7fc4d3400640 1 --2- 172.21.15.114:0/4623052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c40d51f0 0x7fc4c40dd6a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc06bd90 tx=0x7fc4cc08dd30 comp rx=0 tx=0).stop 2024-05-27T09:27:28.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.983+0000 7fc4d1400640 1 -- 172.21.15.114:0/4623052 reap_dead start 2024-05-27T09:27:28.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.983+0000 7fc4d3400640 1 -- 172.21.15.114:0/4623052 shutdown_connections 2024-05-27T09:27:28.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.983+0000 7fc4d3400640 1 -- 172.21.15.114:0/4623052 >> 172.21.15.114:0/4623052 conn(0x7fc4c400e360 msgr2=0x7fc4c4005bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.983+0000 7fc4d3400640 1 -- 172.21.15.114:0/4623052 shutdown_connections 2024-05-27T09:27:28.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.983+0000 7fc4d3400640 1 -- 172.21.15.114:0/4623052 wait complete. 2024-05-27T09:27:28.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.986+0000 7f6fb0800640 1 -- 172.21.15.114:0/2480549801 wait complete. 2024-05-27T09:27:28.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.988+0000 7f01bbe00640 1 -- 172.21.15.114:0/272347691 --> [v2:172.21.15.114:6800/1601792164,v1:172.21.15.114:6801/1601792164] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f01b40cf450 con 0x7f019407e6c0 2024-05-27T09:27:28.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.990+0000 7f01b1600640 1 -- 172.21.15.114:0/272347691 <== osd.3 v2:172.21.15.114:6800/1601792164 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f01b40cf450 con 0x7f019407e6c0 2024-05-27T09:27:28.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.991+0000 7f01bbe00640 1 -- 172.21.15.114:0/272347691 >> [v2:172.21.15.114:6800/1601792164,v1:172.21.15.114:6801/1601792164] conn(0x7f019407e6c0 msgr2=0x7f0194080b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.991+0000 7f01bbe00640 1 --2- 172.21.15.114:0/272347691 >> [v2:172.21.15.114:6800/1601792164,v1:172.21.15.114:6801/1601792164] conn(0x7f019407e6c0 0x7f0194080b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.992+0000 7f01bbe00640 1 -- 172.21.15.114:0/272347691 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f01940764d0 msgr2=0x7f01940789e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.992+0000 7f01bbe00640 1 --2- 172.21.15.114:0/272347691 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f01940764d0 0x7f01940789e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f01b41f6d00 tx=0x7f01a0009040 comp rx=0 tx=0).stop 2024-05-27T09:27:28.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.992+0000 7f01bbe00640 1 -- 172.21.15.114:0/272347691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01b40c73b0 msgr2=0x7f01b41f64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.992+0000 7f01bbe00640 1 --2- 172.21.15.114:0/272347691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01b40c73b0 0x7f01b41f64c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009990 tx=0x7f01a80046c0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.993+0000 7f01b9e00640 1 -- 172.21.15.114:0/272347691 reap_dead start 2024-05-27T09:27:28.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.993+0000 7f01bbe00640 1 -- 172.21.15.114:0/272347691 shutdown_connections 2024-05-27T09:27:28.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.993+0000 7f01bbe00640 1 -- 172.21.15.114:0/272347691 >> 172.21.15.114:0/272347691 conn(0x7f01b406f2f0 msgr2=0x7f01b40cdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.994+0000 7f8fc0000640 1 -- 172.21.15.114:0/3784657571 --> [v2:172.21.15.163:6824/2539451311,v1:172.21.15.163:6825/2539451311] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8fb81df7a0 con 0x7f8f9807e6a0 2024-05-27T09:27:28.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.994+0000 7f01bbe00640 1 -- 172.21.15.114:0/272347691 shutdown_connections 2024-05-27T09:27:28.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.994+0000 7fa080000640 1 -- 172.21.15.114:0/1588870705 --> [v2:172.21.15.114:6808/865568619,v1:172.21.15.114:6809/865568619] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa054002dd0 con 0x7fa05807e6a0 2024-05-27T09:27:28.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.994+0000 7f01bbe00640 1 -- 172.21.15.114:0/272347691 wait complete. 2024-05-27T09:27:28.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.994+0000 7fa075600640 1 -- 172.21.15.114:0/1588870705 <== osd.2 v2:172.21.15.114:6808/865568619 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa054002dd0 con 0x7fa05807e6a0 2024-05-27T09:27:28.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.995+0000 7f8fb5600640 1 -- 172.21.15.114:0/3784657571 <== osd.7 v2:172.21.15.163:6824/2539451311 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8fb81df7a0 con 0x7f8f9807e6a0 2024-05-27T09:27:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.995+0000 7fa080000640 1 -- 172.21.15.114:0/1588870705 >> [v2:172.21.15.114:6808/865568619,v1:172.21.15.114:6809/865568619] conn(0x7fa05807e6a0 msgr2=0x7fa058080af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.995+0000 7fa080000640 1 --2- 172.21.15.114:0/1588870705 >> [v2:172.21.15.114:6808/865568619,v1:172.21.15.114:6809/865568619] conn(0x7fa05807e6a0 0x7fa058080af0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.995+0000 7fa080000640 1 -- 172.21.15.114:0/1588870705 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa0580763d0 msgr2=0x7fa0580788e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.996+0000 7fa080000640 1 --2- 172.21.15.114:0/1588870705 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa0580763d0 0x7fa0580788e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa06c034000 tx=0x7fa06c02e000 comp rx=0 tx=0).stop 2024-05-27T09:27:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.996+0000 7fa080000640 1 -- 172.21.15.114:0/1588870705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa07808d320 msgr2=0x7fa0781fd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.996+0000 7fa080000640 1 --2- 172.21.15.114:0/1588870705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa07808d320 0x7fa0781fd2a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa06400b9f0 tx=0x7fa06400beb0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.996+0000 7f8faaa00640 1 -- 172.21.15.114:0/3784657571 >> [v2:172.21.15.163:6824/2539451311,v1:172.21.15.163:6825/2539451311] conn(0x7f8f9807e6a0 msgr2=0x7f8f98080af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.996+0000 7f8faaa00640 1 --2- 172.21.15.114:0/3784657571 >> [v2:172.21.15.163:6824/2539451311,v1:172.21.15.163:6825/2539451311] conn(0x7f8f9807e6a0 0x7f8f98080af0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.996+0000 7f8faaa00640 1 -- 172.21.15.114:0/3784657571 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f8f980763d0 msgr2=0x7f8f980788e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.996+0000 7fa07e000640 1 -- 172.21.15.114:0/1588870705 reap_dead start 2024-05-27T09:27:28.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.996+0000 7fa080000640 1 -- 172.21.15.114:0/1588870705 shutdown_connections 2024-05-27T09:27:28.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.996+0000 7fa080000640 1 -- 172.21.15.114:0/1588870705 >> 172.21.15.114:0/1588870705 conn(0x7fa07806f6e0 msgr2=0x7fa0781651f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.996+0000 7f8faaa00640 1 --2- 172.21.15.114:0/3784657571 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f8f980763d0 0x7f8f980788e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0001e80 tx=0x7f8fb0009040 comp rx=0 tx=0).stop 2024-05-27T09:27:28.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.996+0000 7fa080000640 1 -- 172.21.15.114:0/1588870705 shutdown_connections 2024-05-27T09:27:28.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.996+0000 7f8faaa00640 1 -- 172.21.15.114:0/3784657571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fb8159fe0 msgr2=0x7f8fb82003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.996+0000 7f8faaa00640 1 --2- 172.21.15.114:0/3784657571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8fb8159fe0 0x7f8fb82003a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009f30 tx=0x7f8fa400e600 comp rx=0 tx=0).stop 2024-05-27T09:27:28.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.996+0000 7f8fbe000640 1 -- 172.21.15.114:0/3784657571 reap_dead start 2024-05-27T09:27:28.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.996+0000 7f8faaa00640 1 -- 172.21.15.114:0/3784657571 shutdown_connections 2024-05-27T09:27:28.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.997+0000 7f8faaa00640 1 -- 172.21.15.114:0/3784657571 >> 172.21.15.114:0/3784657571 conn(0x7f8fb806f6e0 msgr2=0x7f8fb80c9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.997+0000 7fa080000640 1 -- 172.21.15.114:0/1588870705 wait complete. 2024-05-27T09:27:28.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.997+0000 7ff422200640 1 -- 172.21.15.114:0/3348275336 --> [v2:172.21.15.163:6816/1982246015,v1:172.21.15.163:6817/1982246015] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff41c161840 con 0x7ff3f807e5f0 2024-05-27T09:27:28.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.997+0000 7ff418c00640 1 -- 172.21.15.114:0/3348275336 <== osd.4 v2:172.21.15.163:6816/1982246015 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff41c161840 con 0x7ff3f807e5f0 2024-05-27T09:27:28.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.998+0000 7faf23600640 1 -- 172.21.15.114:0/129815001 --> [v2:172.21.15.163:6800/1695294004,v1:172.21.15.163:6801/1695294004] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faf1c06a380 con 0x7faef807e640 2024-05-27T09:27:28.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.998+0000 7ff422200640 1 -- 172.21.15.114:0/3348275336 >> [v2:172.21.15.163:6816/1982246015,v1:172.21.15.163:6817/1982246015] conn(0x7ff3f807e5f0 msgr2=0x7ff3f8080a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.998+0000 7ff422200640 1 --2- 172.21.15.114:0/3348275336 >> [v2:172.21.15.163:6816/1982246015,v1:172.21.15.163:6817/1982246015] conn(0x7ff3f807e5f0 0x7ff3f8080a40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.999+0000 7ff422200640 1 -- 172.21.15.114:0/3348275336 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff3f8076320 msgr2=0x7ff3f8078830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.999+0000 7ff422200640 1 --2- 172.21.15.114:0/3348275336 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff3f8076320 0x7ff3f8078830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff414034000 tx=0x7ff41402f000 comp rx=0 tx=0).stop 2024-05-27T09:27:28.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.999+0000 7ff422200640 1 -- 172.21.15.114:0/3348275336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff41c0ce440 msgr2=0x7ff41c15e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.999+0000 7ff422200640 1 --2- 172.21.15.114:0/3348275336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff41c0ce440 0x7ff41c15e770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00c730 tx=0x7ff40c00cbf0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.999+0000 7faf19600640 1 -- 172.21.15.114:0/129815001 <== osd.5 v2:172.21.15.163:6800/1695294004 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7faf1c06a380 con 0x7faef807e640 2024-05-27T09:27:28.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:27.999+0000 7ff421800640 1 -- 172.21.15.114:0/3348275336 reap_dead start 2024-05-27T09:27:28.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.000+0000 7ff422200640 1 -- 172.21.15.114:0/3348275336 shutdown_connections 2024-05-27T09:27:28.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.000+0000 7ff422200640 1 -- 172.21.15.114:0/3348275336 >> 172.21.15.114:0/3348275336 conn(0x7ff41c0cd890 msgr2=0x7ff41c150b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.000+0000 7ff422200640 1 -- 172.21.15.114:0/3348275336 shutdown_connections 2024-05-27T09:27:28.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.000+0000 7ff422200640 1 -- 172.21.15.114:0/3348275336 wait complete. 2024-05-27T09:27:28.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.000+0000 7faf23600640 1 -- 172.21.15.114:0/129815001 >> [v2:172.21.15.163:6800/1695294004,v1:172.21.15.163:6801/1695294004] conn(0x7faef807e640 msgr2=0x7faef8080a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.000+0000 7faf23600640 1 --2- 172.21.15.114:0/129815001 >> [v2:172.21.15.163:6800/1695294004,v1:172.21.15.163:6801/1695294004] conn(0x7faef807e640 0x7faef8080a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.000+0000 7f8faaa00640 1 -- 172.21.15.114:0/3784657571 shutdown_connections 2024-05-27T09:27:28.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.000+0000 7f8faaa00640 1 -- 172.21.15.114:0/3784657571 wait complete. 2024-05-27T09:27:28.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.001+0000 7faf23600640 1 -- 172.21.15.114:0/129815001 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7faef80763d0 msgr2=0x7faef80788e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.001+0000 7faf23600640 1 --2- 172.21.15.114:0/129815001 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7faef80763d0 0x7faef80788e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faf1c160920 tx=0x7faf04009040 comp rx=0 tx=0).stop 2024-05-27T09:27:28.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.001+0000 7faf23600640 1 -- 172.21.15.114:0/129815001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf1c0c39b0 msgr2=0x7faf1c1ffdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.001+0000 7faf23600640 1 --2- 172.21.15.114:0/129815001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf1c0c39b0 0x7faf1c1ffdc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00c800 tx=0x7faf0c00ccc0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.001+0000 7faf21600640 1 -- 172.21.15.114:0/129815001 reap_dead start 2024-05-27T09:27:28.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.001+0000 7faf23600640 1 -- 172.21.15.114:0/129815001 shutdown_connections 2024-05-27T09:27:28.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.001+0000 7faf23600640 1 -- 172.21.15.114:0/129815001 >> 172.21.15.114:0/129815001 conn(0x7faf1c06f6e0 msgr2=0x7faf1c0c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.002+0000 7faf23600640 1 -- 172.21.15.114:0/129815001 shutdown_connections 2024-05-27T09:27:28.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.002+0000 7faf23600640 1 -- 172.21.15.114:0/129815001 wait complete. 2024-05-27T09:27:28.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.009+0000 7f4218800640 1 -- 172.21.15.114:0/3306782595 --> [v2:172.21.15.163:6808/3649718505,v1:172.21.15.163:6809/3649718505] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f42100caa00 con 0x7f41f407e590 2024-05-27T09:27:28.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.010+0000 7f420a000640 1 -- 172.21.15.114:0/3306782595 <== osd.6 v2:172.21.15.163:6808/3649718505 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f42100caa00 con 0x7f41f407e590 2024-05-27T09:27:28.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.010+0000 7f4218800640 1 -- 172.21.15.114:0/3306782595 >> [v2:172.21.15.163:6808/3649718505,v1:172.21.15.163:6809/3649718505] conn(0x7f41f407e590 msgr2=0x7f41f40809e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.010+0000 7f4218800640 1 --2- 172.21.15.114:0/3306782595 >> [v2:172.21.15.163:6808/3649718505,v1:172.21.15.163:6809/3649718505] conn(0x7f41f407e590 0x7f41f40809e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.012+0000 7f4218800640 1 -- 172.21.15.114:0/3306782595 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f41f4076320 msgr2=0x7f41f4078830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.012+0000 7f4218800640 1 --2- 172.21.15.114:0/3306782595 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f41f4076320 0x7f41f4078830 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4204001b80 tx=0x7f4204007d30 comp rx=0 tx=0).stop 2024-05-27T09:27:28.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.012+0000 7f4218800640 1 -- 172.21.15.114:0/3306782595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f421015df70 msgr2=0x7f42100c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.012+0000 7f4218800640 1 --2- 172.21.15.114:0/3306782595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f421015df70 0x7f42100c6dd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f420c02a720 tx=0x7f420c0046c0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.012+0000 7f4216800640 1 -- 172.21.15.114:0/3306782595 reap_dead start 2024-05-27T09:27:28.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.012+0000 7f4218800640 1 -- 172.21.15.114:0/3306782595 shutdown_connections 2024-05-27T09:27:28.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.012+0000 7f4218800640 1 -- 172.21.15.114:0/3306782595 >> 172.21.15.114:0/3306782595 conn(0x7f4210071660 msgr2=0x7f4210159cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.012+0000 7f4218800640 1 -- 172.21.15.114:0/3306782595 shutdown_connections 2024-05-27T09:27:28.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.013+0000 7f4218800640 1 -- 172.21.15.114:0/3306782595 wait complete. 2024-05-27T09:27:28.027 INFO:teuthology.orchestra.run.smithi114.stdout:64424509447 2024-05-27T09:27:28.027 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-27T09:27:28.028 INFO:teuthology.orchestra.run.smithi114.stdout:64424509447 2024-05-27T09:27:28.028 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-27T09:27:28.039 INFO:teuthology.orchestra.run.smithi114.stdout:64424509447 2024-05-27T09:27:28.039 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-27T09:27:28.043 INFO:teuthology.orchestra.run.smithi114.stdout:64424509447 2024-05-27T09:27:28.044 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-27T09:27:28.044 INFO:teuthology.orchestra.run.smithi114.stdout:60129542150 2024-05-27T09:27:28.045 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-27T09:27:28.048 INFO:teuthology.orchestra.run.smithi114.stdout:60129542150 2024-05-27T09:27:28.048 INFO:teuthology.orchestra.run.smithi114.stdout:60129542150 2024-05-27T09:27:28.048 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-27T09:27:28.049 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-27T09:27:28.051 INFO:teuthology.orchestra.run.smithi114.stdout:60129542150 2024-05-27T09:27:28.051 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-27T09:27:28.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.426+0000 7f868aa00640 1 Processor -- start 2024-05-27T09:27:28.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.426+0000 7f2be6c00640 1 Processor -- start 2024-05-27T09:27:28.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.426+0000 7f2be6c00640 1 -- start start 2024-05-27T09:27:28.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.429+0000 7f2be6c00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2bd8045350 0x7f2bd8006090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.427+0000 7f868aa00640 1 -- start start 2024-05-27T09:27:28.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.430+0000 7f2be6c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bd8036070 0x7f2bd8100c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.430+0000 7f2be6c00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bd80f7600 con 0x7f2bd804c8c0 2024-05-27T09:27:28.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.429+0000 7f868aa00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f86840cde50 0x7f86840c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.429+0000 7f868aa00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8684156550 0x7f86840cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.430+0000 7f2be5800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2bd8045350 0x7f2bd8006090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.430+0000 7f2be6c00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2bd8033150 con 0x7f2bd8045350 2024-05-27T09:27:28.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.431+0000 7f2be6c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd8035be0 con 0x7f2bd8036070 2024-05-27T09:27:28.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.430+0000 7f2be6200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bd8036070 0x7f2bd8100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.431+0000 7f2be6200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bd8036070 0x7f2bd8100c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:60520/0 (socket says 172.21.15.114:60520) 2024-05-27T09:27:28.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.431+0000 7f2be6200640 1 -- 172.21.15.114:0/3671796029 learned_addr learned my addr 172.21.15.114:0/3671796029 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:28.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.431+0000 7f8688c00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8684156550 0x7f86840cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.431+0000 7f8688c00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8684156550 0x7f86840cd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35568/0 (socket says 172.21.15.114:35568) 2024-05-27T09:27:28.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.430+0000 7f2be4e00640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2bd804c8c0 0x7f2bd81004f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:44670/0 (socket says 172.21.15.114:44670) 2024-05-27T09:27:28.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.431+0000 7f8689600640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f86840cde50 0x7f86840c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.431+0000 7fedafe00640 1 Processor -- start 2024-05-27T09:27:28.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.431+0000 7f8688c00640 1 -- 172.21.15.114:0/317954005 learned_addr learned my addr 172.21.15.114:0/317954005 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:28.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.431+0000 7f2be6200640 1 -- 172.21.15.114:0/3671796029 >> v2:172.21.15.114:3300/0 conn(0x7f2bd8045350 msgr2=0x7f2bd8006090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:28.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.432+0000 7fedafe00640 1 -- start start 2024-05-27T09:27:28.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.432+0000 7f2bdfe00640 1 -- 172.21.15.114:0/3671796029 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1147380546 0 0) 0x7f2bd80f7600 con 0x7f2bd804c8c0 2024-05-27T09:27:28.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.432+0000 7f2be6200640 1 --2- 172.21.15.114:0/3671796029 >> v2:172.21.15.114:3300/0 conn(0x7f2bd8045350 0x7f2bd8006090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.432+0000 7f2be6200640 1 -- 172.21.15.114:0/3671796029 >> v1:172.21.15.163:6789/0 conn(0x7f2bd804c8c0 legacy=0x7f2bd81004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.432+0000 7f2be6200640 1 -- 172.21.15.114:0/3671796029 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd8034fd0 con 0x7f2bd8036070 2024-05-27T09:27:28.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.432+0000 7f2be5800640 1 --2- 172.21.15.114:0/3671796029 >> v2:172.21.15.114:3300/0 conn(0x7f2bd8045350 0x7f2bd8006090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:27:28.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.432+0000 7f868aa00640 1 -- 172.21.15.114:0/317954005 --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f868415c6e0 con 0x7f86840cd630 2024-05-27T09:27:28.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.432+0000 7f868aa00640 1 -- 172.21.15.114:0/317954005 --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f86840b5c00 con 0x7f8684156550 2024-05-27T09:27:28.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.433+0000 7f868aa00640 1 -- 172.21.15.114:0/317954005 --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f868415c980 con 0x7f86840cde50 2024-05-27T09:27:28.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.433+0000 7f2be6200640 1 --2- 172.21.15.114:0/3671796029 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bd8036070 0x7f2bd8100c00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2be006c4a0 tx=0x7f2be0096a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9509378fcc1b240b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.434+0000 7f2bdfe00640 1 -- 172.21.15.114:0/3671796029 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be009a6b0 con 0x7f2bd8036070 2024-05-27T09:27:28.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.434+0000 7f8688c00640 1 -- 172.21.15.114:0/317954005 >> v2:172.21.15.163:3300/0 conn(0x7f86840cde50 msgr2=0x7f86840c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:28.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.434+0000 7f2bdfe00640 1 -- 172.21.15.114:0/3671796029 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2be009a850 con 0x7f2bd8036070 2024-05-27T09:27:28.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.434+0000 7f2bdfe00640 1 -- 172.21.15.114:0/3671796029 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be009a6b0 con 0x7f2bd8036070 2024-05-27T09:27:28.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.434+0000 7f8688c00640 1 --2- 172.21.15.114:0/317954005 >> v2:172.21.15.163:3300/0 conn(0x7f86840cde50 0x7f86840c69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.435+0000 7f8688c00640 1 -- 172.21.15.114:0/317954005 >> v1:172.21.15.163:6789/0 conn(0x7f86840cd630 legacy=0x7f868415fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.435+0000 7f8688c00640 1 -- 172.21.15.114:0/317954005 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868415cc60 con 0x7f8684156550 2024-05-27T09:27:28.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.435+0000 7f8689600640 1 --2- 172.21.15.114:0/317954005 >> v2:172.21.15.163:3300/0 conn(0x7f86840cde50 0x7f86840c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:28.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.435+0000 7f2be6c00640 1 -- 172.21.15.114:0/3671796029 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bd8036070 msgr2=0x7f2bd8100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.435+0000 7f2be6c00640 1 --2- 172.21.15.114:0/3671796029 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bd8036070 0x7f2bd8100c00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2be006c4a0 tx=0x7f2be0096a40 comp rx=0 tx=0).stop 2024-05-27T09:27:28.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.435+0000 7f2be6c00640 1 -- 172.21.15.114:0/3671796029 shutdown_connections 2024-05-27T09:27:28.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.435+0000 7f2be6c00640 1 --2- 172.21.15.114:0/3671796029 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bd8036070 0x7f2bd8100c00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.435+0000 7f2be6c00640 1 --2- 172.21.15.114:0/3671796029 >> v2:172.21.15.114:3300/0 conn(0x7f2bd8045350 0x7f2bd8006090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.435+0000 7f2be6c00640 1 -- 172.21.15.114:0/3671796029 >> 172.21.15.114:0/3671796029 conn(0x7f2bd80503b0 msgr2=0x7f2bd80507c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.436+0000 7fedafe00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feda8152890 0x7feda80c28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.436+0000 7f8688c00640 1 --2- 172.21.15.114:0/317954005 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8684156550 0x7f86840cd0f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f868408e750 tx=0x7f867002a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8317ae1302ee5c81 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.436+0000 7fedafe00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda80c8c20 0x7feda80c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.437+0000 7f2be6c00640 1 -- 172.21.15.114:0/3671796029 shutdown_connections 2024-05-27T09:27:28.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.437+0000 7fedad400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feda8152890 0x7feda80c28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.437+0000 7fedad400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feda8152890 0x7feda80c28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37704/0 (socket says 172.21.15.114:37704) 2024-05-27T09:27:28.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.437+0000 7fedad400640 1 -- 172.21.15.114:0/552170237 learned_addr learned my addr 172.21.15.114:0/552170237 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:28.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.437+0000 7f8683e00640 1 -- 172.21.15.114:0/317954005 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8670036070 con 0x7f8684156550 2024-05-27T09:27:28.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.437+0000 7f8683e00640 1 -- 172.21.15.114:0/317954005 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8670031600 con 0x7f8684156550 2024-05-27T09:27:28.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.437+0000 7fedafe00640 1 -- 172.21.15.114:0/552170237 --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feda80c8660 con 0x7feda80c2df0 2024-05-27T09:27:28.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.437+0000 7fedafe00640 1 -- 172.21.15.114:0/552170237 --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7feda80c8940 con 0x7feda8152890 2024-05-27T09:27:28.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.437+0000 7feda7e00640 1 -- 172.21.15.114:0/552170237 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 170359185 0 0) 0x7feda80c8660 con 0x7feda80c2df0 2024-05-27T09:27:28.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.437+0000 7fedafe00640 1 -- 172.21.15.114:0/552170237 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7feda80744d0 con 0x7feda80c8c20 2024-05-27T09:27:28.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.438+0000 7feda7e00640 1 -- 172.21.15.114:0/552170237 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed90004630 con 0x7feda80c2df0 2024-05-27T09:27:28.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.437+0000 7f8683e00640 1 -- 172.21.15.114:0/317954005 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86700319d0 con 0x7f8684156550 2024-05-27T09:27:28.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.438+0000 7f868aa00640 1 -- 172.21.15.114:0/317954005 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8684156550 msgr2=0x7f86840cd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.438+0000 7f868aa00640 1 --2- 172.21.15.114:0/317954005 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8684156550 0x7f86840cd0f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f868408e750 tx=0x7f867002a570 comp rx=0 tx=0).stop 2024-05-27T09:27:28.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.437+0000 7fedade00640 1 --2- 172.21.15.114:0/552170237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda80c8c20 0x7feda80c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.438+0000 7feda7e00640 1 -- 172.21.15.114:0/552170237 <== mon.1 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4281081598 0 0) 0x7fed90004630 con 0x7feda80c2df0 2024-05-27T09:27:28.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.439+0000 7f868aa00640 1 -- 172.21.15.114:0/317954005 shutdown_connections 2024-05-27T09:27:28.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.439+0000 7feda7e00640 1 -- 172.21.15.114:0/552170237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda80c8c20 msgr2=0x7feda80c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:28.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.439+0000 7feda7e00640 1 --2- 172.21.15.114:0/552170237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda80c8c20 0x7feda80c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.439+0000 7feda7e00640 1 -- 172.21.15.114:0/552170237 >> v2:172.21.15.163:3300/0 conn(0x7feda8152890 msgr2=0x7feda80c28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:28.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.439+0000 7feda7e00640 1 --2- 172.21.15.114:0/552170237 >> v2:172.21.15.163:3300/0 conn(0x7feda8152890 0x7feda80c28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.439+0000 7feda7e00640 1 -- 172.21.15.114:0/552170237 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda80747b0 con 0x7feda80c2df0 2024-05-27T09:27:28.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.439+0000 7f868aa00640 1 --2- 172.21.15.114:0/317954005 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8684156550 0x7f86840cd0f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.439+0000 7f868aa00640 1 --2- 172.21.15.114:0/317954005 >> v2:172.21.15.163:3300/0 conn(0x7f86840cde50 0x7f86840c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.439+0000 7f868aa00640 1 -- 172.21.15.114:0/317954005 >> 172.21.15.114:0/317954005 conn(0x7f868408d960 msgr2=0x7f868408dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.439+0000 7feda7e00640 1 -- 172.21.15.114:0/552170237 <== mon.1 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3867083960 0 0) 0x7fed940016f0 con 0x7feda80c2df0 2024-05-27T09:27:28.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.439+0000 7feda7e00640 1 -- 172.21.15.114:0/552170237 >> v1:172.21.15.163:6789/0 conn(0x7feda80c2df0 legacy=0x7feda80b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.439+0000 7f868aa00640 1 -- 172.21.15.114:0/317954005 shutdown_connections 2024-05-27T09:27:28.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.439+0000 7f868aa00640 1 -- 172.21.15.114:0/317954005 wait complete. 2024-05-27T09:27:28.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.439+0000 7feda7e00640 1 --2- 172.21.15.114:0/552170237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed90004b10 0x7fed90004f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.439+0000 7feda7e00640 1 --2- 172.21.15.114:0/552170237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed90005910 0x7fed90005d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.439+0000 7feda7e00640 1 --2- 172.21.15.114:0/552170237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed90006260 0x7fed90003420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.439+0000 7feda7e00640 1 -- 172.21.15.114:0/552170237 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7feda80c8940 con 0x7fed90004b10 2024-05-27T09:27:28.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.439+0000 7feda7e00640 1 -- 172.21.15.114:0/552170237 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7feda80744d0 con 0x7fed90006260 2024-05-27T09:27:28.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.437+0000 7f2be6c00640 1 -- 172.21.15.114:0/3671796029 wait complete. 2024-05-27T09:27:28.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.439+0000 7feda7e00640 1 -- 172.21.15.114:0/552170237 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fed90003b10 con 0x7fed90005910 2024-05-27T09:27:28.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.440+0000 7f2be6c00640 1 Processor -- start 2024-05-27T09:27:28.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.441+0000 7f2be6c00640 1 -- start start 2024-05-27T09:27:28.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.441+0000 7fedad400640 1 --2- 172.21.15.114:0/552170237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed90004b10 0x7fed90004f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.441+0000 7fedad400640 1 -- 172.21.15.114:0/552170237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed90005910 msgr2=0x7fed90005d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T09:27:28.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.443+0000 7f868aa00640 1 Processor -- start 2024-05-27T09:27:28.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.444+0000 7f2be6c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bd8036070 0x7f2bd80ff9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.443+0000 7f868aa00640 1 -- start start 2024-05-27T09:27:28.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.444+0000 7f2be6c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bd8045350 0x7f2bd81a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.444+0000 7f2be6c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bd804c8c0 0x7f2bd8006a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.445+0000 7f2be6c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8035870 con 0x7f2bd804c8c0 2024-05-27T09:27:28.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.445+0000 7f2be6c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8035590 con 0x7f2bd8045350 2024-05-27T09:27:28.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.445+0000 7f2be6200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bd804c8c0 0x7f2bd8006a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.445+0000 7f2be6200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bd804c8c0 0x7f2bd8006a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35584/0 (socket says 172.21.15.114:35584) 2024-05-27T09:27:28.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.445+0000 7f2be6200640 1 -- 172.21.15.114:0/2658050711 learned_addr learned my addr 172.21.15.114:0/2658050711 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:28.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.445+0000 7f2be6c00640 1 -- 172.21.15.114:0/2658050711 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd8036480 con 0x7f2bd8036070 2024-05-27T09:27:28.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.441+0000 7fedad400640 1 --2- 172.21.15.114:0/552170237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed90005910 0x7fed90005d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.441+0000 7fedad400640 1 -- 172.21.15.114:0/552170237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed90006260 msgr2=0x7fed90003420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T09:27:28.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.441+0000 7fedad400640 1 --2- 172.21.15.114:0/552170237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed90006260 0x7fed90003420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.441+0000 7fedad400640 1 -- 172.21.15.114:0/552170237 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda80747b0 con 0x7fed90004b10 2024-05-27T09:27:28.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.443+0000 7fedad400640 1 --2- 172.21.15.114:0/552170237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed90004b10 0x7fed90004f80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7feda00098c0 tx=0x7fed90006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781cdb32fc843931 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.444+0000 7feda7e00640 1 -- 172.21.15.114:0/552170237 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0038070 con 0x7fed90004b10 2024-05-27T09:27:28.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.444+0000 7feda7e00640 1 -- 172.21.15.114:0/552170237 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda0030640 con 0x7fed90004b10 2024-05-27T09:27:28.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.445+0000 7feda7e00640 1 -- 172.21.15.114:0/552170237 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0038070 con 0x7fed90004b10 2024-05-27T09:27:28.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.445+0000 7fedafe00640 1 -- 172.21.15.114:0/552170237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed90004b10 msgr2=0x7fed90004f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.445+0000 7fedafe00640 1 --2- 172.21.15.114:0/552170237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed90004b10 0x7fed90004f80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7feda00098c0 tx=0x7fed90006940 comp rx=0 tx=0).stop 2024-05-27T09:27:28.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.446+0000 7fedade00640 1 -- 172.21.15.114:0/552170237 reap_dead start 2024-05-27T09:27:28.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.446+0000 7f7d9b800640 1 Processor -- start 2024-05-27T09:27:28.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.446+0000 7f2be6200640 1 -- 172.21.15.114:0/2658050711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bd8036070 msgr2=0x7f2bd80ff9b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T09:27:28.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.445+0000 7f868aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86840cd630 0x7f868415e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.446+0000 7f868aa00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f86840cde50 0x7f868414ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.446+0000 7f7d9b800640 1 -- start start 2024-05-27T09:27:28.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.446+0000 7fedafe00640 1 -- 172.21.15.114:0/552170237 shutdown_connections 2024-05-27T09:27:28.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.446+0000 7fedafe00640 1 -- 172.21.15.114:0/552170237 >> 172.21.15.114:0/552170237 conn(0x7feda80cd890 msgr2=0x7feda80cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.446+0000 7f2be6200640 1 --2- 172.21.15.114:0/2658050711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bd8036070 0x7f2bd80ff9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.446+0000 7f2be6200640 1 -- 172.21.15.114:0/2658050711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bd8045350 msgr2=0x7f2bd81a4a50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T09:27:28.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.446+0000 7f2be6200640 1 --2- 172.21.15.114:0/2658050711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bd8045350 0x7f2bd81a4a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.446+0000 7f3183e00640 1 Processor -- start 2024-05-27T09:27:28.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.446+0000 7fedafe00640 1 -- 172.21.15.114:0/552170237 shutdown_connections 2024-05-27T09:27:28.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.446+0000 7fedafe00640 1 -- 172.21.15.114:0/552170237 wait complete. 2024-05-27T09:27:28.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.446+0000 7f2be6200640 1 -- 172.21.15.114:0/2658050711 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd80f1ff0 con 0x7f2bd804c8c0 2024-05-27T09:27:28.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.446+0000 7f2be6200640 1 --2- 172.21.15.114:0/2658050711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bd804c8c0 0x7f2bd8006a00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2be002e1b0 tx=0x7f2be006ab30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.447+0000 7f3183e00640 1 -- start start 2024-05-27T09:27:28.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.447+0000 7f868aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8684156550 0x7f86841f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.447+0000 7f868aa00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8684156960 con 0x7f86840cd630 2024-05-27T09:27:28.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.447+0000 7f868aa00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f868415d220 con 0x7f8684156550 2024-05-27T09:27:28.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.447+0000 7f2bdd600640 1 -- 172.21.15.114:0/2658050711 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be00b2070 con 0x7f2bd804c8c0 2024-05-27T09:27:28.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.447+0000 7f2bdd600640 1 -- 172.21.15.114:0/2658050711 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2be00ad3f0 con 0x7f2bd804c8c0 2024-05-27T09:27:28.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.447+0000 7f2bdd600640 1 -- 172.21.15.114:0/2658050711 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be009a6b0 con 0x7f2bd804c8c0 2024-05-27T09:27:28.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.448+0000 7f8689600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86840cd630 0x7f868415e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.448+0000 7f8689600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86840cd630 0x7f868415e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35586/0 (socket says 172.21.15.114:35586) 2024-05-27T09:27:28.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.448+0000 7f8689600640 1 -- 172.21.15.114:0/3941003987 learned_addr learned my addr 172.21.15.114:0/3941003987 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:28.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.448+0000 7f8688c00640 1 --2- 172.21.15.114:0/3941003987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f86840cde50 0x7f868414ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.447+0000 7f868aa00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f86840b4580 con 0x7f86840cde50 2024-05-27T09:27:28.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.447+0000 7f2be6c00640 1 -- 172.21.15.114:0/2658050711 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd81a7840 con 0x7f2bd804c8c0 2024-05-27T09:27:28.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.447+0000 7f2be6c00640 1 -- 172.21.15.114:0/2658050711 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd81a7e00 con 0x7f2bd804c8c0 2024-05-27T09:27:28.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.449+0000 7f7d9b800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7d94152890 0x7f7d940c28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.450+0000 7f8688c00640 1 -- 172.21.15.114:0/3941003987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8684156550 msgr2=0x7f86841f56f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T09:27:28.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.450+0000 7f868a000640 1 --2- 172.21.15.114:0/3941003987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8684156550 0x7f86841f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.450+0000 7f3183e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f31840c44b0 0x7f3184152990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.450+0000 7f7d98e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7d94152890 0x7f7d940c28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.450+0000 7f7d98e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7d94152890 0x7f7d940c28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35590/0 (socket says 172.21.15.114:35590) 2024-05-27T09:27:28.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.450+0000 7f3183e00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f31840cae40 0x7f3184162e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.450+0000 7f3183e00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31840c97b0 con 0x7f31840c2840 2024-05-27T09:27:28.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.450+0000 7f3183e00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f31840c73a0 con 0x7f31840c44b0 2024-05-27T09:27:28.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.450+0000 7f3183e00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f31840c7660 con 0x7f31840cae40 2024-05-27T09:27:28.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.451+0000 7f7d98e00640 1 -- 172.21.15.114:0/3284769196 learned_addr learned my addr 172.21.15.114:0/3284769196 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:28.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.451+0000 7f7d9b800640 1 --2- 172.21.15.114:0/3284769196 >> v2:172.21.15.163:3300/0 conn(0x7f7d940ce3b0 0x7f7d940b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.451+0000 7f3182a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f31840c44b0 0x7f3184152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.451+0000 7f3182a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f31840c44b0 0x7f3184152990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35594/0 (socket says 172.21.15.114:35594) 2024-05-27T09:27:28.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.451+0000 7f3182a00640 1 -- 172.21.15.114:0/4163564550 learned_addr learned my addr 172.21.15.114:0/4163564550 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:28.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.450+0000 7fedafe00640 1 Processor -- start 2024-05-27T09:27:28.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.451+0000 7fedafe00640 1 -- start start 2024-05-27T09:27:28.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.451+0000 7f7d9b800640 1 -- 172.21.15.114:0/3284769196 --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d940c8660 con 0x7f7d940c2df0 2024-05-27T09:27:28.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.451+0000 7f7d9b800640 1 -- 172.21.15.114:0/3284769196 --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f7d940c8940 con 0x7f7d94152890 2024-05-27T09:27:28.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.450+0000 7f8688c00640 1 --2- 172.21.15.114:0/3941003987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8684156550 0x7f86841f56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.450+0000 7f8688c00640 1 -- 172.21.15.114:0/3941003987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86840cd630 msgr2=0x7f868415e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.450+0000 7f8688c00640 1 --2- 172.21.15.114:0/3941003987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86840cd630 0x7f868415e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.450+0000 7f8688c00640 1 -- 172.21.15.114:0/3941003987 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868415ec30 con 0x7f86840cde50 2024-05-27T09:27:28.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.451+0000 7f7d9b800640 1 -- 172.21.15.114:0/3284769196 --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f7d940744d0 con 0x7f7d940ce3b0 2024-05-27T09:27:28.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.452+0000 7f8689600640 1 --2- 172.21.15.114:0/3941003987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86840cd630 0x7f868415e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:27:28.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.452+0000 7fd35d600640 1 Processor -- start 2024-05-27T09:27:28.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.452+0000 7fd35d600640 1 -- start start 2024-05-27T09:27:28.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.452+0000 7f8688c00640 1 --2- 172.21.15.114:0/3941003987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f86840cde50 0x7f868414ea00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f86840cf550 tx=0x7f86700046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.452+0000 7f7d93e00640 1 --2- 172.21.15.114:0/3284769196 >> v2:172.21.15.163:3300/0 conn(0x7f7d940ce3b0 0x7f7d940b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.451+0000 7f2bdd600640 1 -- 172.21.15.114:0/2658050711 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2be00ad7a0 con 0x7f2bd804c8c0 2024-05-27T09:27:28.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.452+0000 7f2bdd600640 1 --2- 172.21.15.114:0/2658050711 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2bc00762d0 0x7f2bc00787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.452+0000 7f2be6c00640 1 -- 172.21.15.114:0/2658050711 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bbc005290 con 0x7f2bd804c8c0 2024-05-27T09:27:28.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.452+0000 7f2bdd600640 1 -- 172.21.15.114:0/2658050711 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2be01242c0 con 0x7f2bd804c8c0 2024-05-27T09:27:28.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.453+0000 7f2be5800640 1 --2- 172.21.15.114:0/2658050711 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2bc00762d0 0x7f2bc00787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.451+0000 7f3182a00640 1 -- 172.21.15.114:0/4163564550 >> v2:172.21.15.163:3300/0 conn(0x7f31840cae40 msgr2=0x7f3184162e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T09:27:28.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.451+0000 7f3182a00640 1 --2- 172.21.15.114:0/4163564550 >> v2:172.21.15.163:3300/0 conn(0x7f31840cae40 0x7f3184162e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.451+0000 7f3182a00640 1 -- 172.21.15.114:0/4163564550 >> v1:172.21.15.163:6789/0 conn(0x7f31840c2840 legacy=0x7f31840c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:28.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.451+0000 7f3182a00640 1 -- 172.21.15.114:0/4163564550 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31840c7940 con 0x7f31840c44b0 2024-05-27T09:27:28.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.453+0000 7fedafe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed90004b10 0x7feda8159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.453+0000 7fedafe00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda815c7b0 0x7feda8155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.453+0000 7fedafe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feda81fff70 0x7feda80c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.453+0000 7fedafe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7feda80c8410 con 0x7feda81fff70 2024-05-27T09:27:28.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.453+0000 7fedafe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7feda80c8290 con 0x7fed90004b10 2024-05-27T09:27:28.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.453+0000 7fedafe00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7feda80b38d0 con 0x7feda815c7b0 2024-05-27T09:27:28.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.453+0000 7fedad400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed90004b10 0x7feda8159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.453+0000 7fedaca00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda815c7b0 0x7feda8155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.453+0000 7f8681600640 1 -- 172.21.15.114:0/3941003987 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8670040020 con 0x7f86840cde50 2024-05-27T09:27:28.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.453+0000 7f8681600640 1 -- 172.21.15.114:0/3941003987 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f867003a810 con 0x7f86840cde50 2024-05-27T09:27:28.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.453+0000 7f8681600640 1 -- 172.21.15.114:0/3941003987 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8670036040 con 0x7f86840cde50 2024-05-27T09:27:28.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.454+0000 7f868aa00640 1 -- 172.21.15.114:0/3941003987 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f868415f640 con 0x7f86840cde50 2024-05-27T09:27:28.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.453+0000 7f7d98e00640 1 -- 172.21.15.114:0/3284769196 >> v2:172.21.15.163:3300/0 conn(0x7f7d940ce3b0 msgr2=0x7f7d940b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:28.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.453+0000 7f7d98e00640 1 --2- 172.21.15.114:0/3284769196 >> v2:172.21.15.163:3300/0 conn(0x7f7d940ce3b0 0x7f7d940b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.453+0000 7f7d98e00640 1 -- 172.21.15.114:0/3284769196 >> v1:172.21.15.114:6789/0 conn(0x7f7d940c2df0 legacy=0x7f7d940c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.453+0000 7f7d98e00640 1 -- 172.21.15.114:0/3284769196 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d940747b0 con 0x7f7d94152890 2024-05-27T09:27:28.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.454+0000 7f868aa00640 1 -- 172.21.15.114:0/3941003987 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f868415ee40 con 0x7f86840cde50 2024-05-27T09:27:28.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.455+0000 7fd35d600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd358164db0 0x7fd3580c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.454+0000 7f7d93e00640 1 --2- 172.21.15.114:0/3284769196 >> v2:172.21.15.163:3300/0 conn(0x7f7d940ce3b0 0x7f7d940b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:27:28.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.454+0000 7f3182a00640 1 --2- 172.21.15.114:0/4163564550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31840c44b0 0x7f3184152990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f316c009870 tx=0x7f316c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8a91bcbca445cd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.454+0000 7f7cf5e00640 1 Processor -- start 2024-05-27T09:27:28.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.454+0000 7f7cf5e00640 1 -- start start 2024-05-27T09:27:28.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.455+0000 7fd356a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd358164db0 0x7fd3580c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.453+0000 7fedad400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed90004b10 0x7feda8159820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37764/0 (socket says 172.21.15.114:37764) 2024-05-27T09:27:28.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.453+0000 7fedad400640 1 -- 172.21.15.114:0/2794436561 learned_addr learned my addr 172.21.15.114:0/2794436561 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:28.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.455+0000 7fd356a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd358164db0 0x7fd3580c3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:60560/0 (socket says 172.21.15.114:60560) 2024-05-27T09:27:28.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.455+0000 7fd356a00640 1 -- 172.21.15.114:0/1907478743 learned_addr learned my addr 172.21.15.114:0/1907478743 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:28.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.455+0000 7fd35d600640 1 --2- 172.21.15.114:0/1907478743 >> v2:172.21.15.163:3300/0 conn(0x7fd358074860 0x7fd35815aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.454+0000 7fedade00640 1 --2- 172.21.15.114:0/2794436561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feda81fff70 0x7feda80c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.454+0000 7fedaca00640 1 -- 172.21.15.114:0/2794436561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed90004b10 msgr2=0x7feda8159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.454+0000 7fedaca00640 1 --2- 172.21.15.114:0/2794436561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed90004b10 0x7feda8159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.454+0000 7fedaca00640 1 -- 172.21.15.114:0/2794436561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feda81fff70 msgr2=0x7feda80c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.454+0000 7fedaca00640 1 --2- 172.21.15.114:0/2794436561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feda81fff70 0x7feda80c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.455+0000 7fedaca00640 1 -- 172.21.15.114:0/2794436561 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda815c010 con 0x7feda815c7b0 2024-05-27T09:27:28.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.455+0000 7fedaca00640 1 --2- 172.21.15.114:0/2794436561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda815c7b0 0x7feda8155a10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fed9400c760 tx=0x7fed9400cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.455+0000 7f2be5800640 1 --2- 172.21.15.114:0/2658050711 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2bc00762d0 0x7f2bc00787e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400a810 tx=0x7f2bd4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.455+0000 7f7d98e00640 1 --2- 172.21.15.114:0/3284769196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d94152890 0x7f7d940c28b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7d940694a0 tx=0x7f7d8002aa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0da497012c90c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.456+0000 7feda5600640 1 -- 172.21.15.114:0/2794436561 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94005270 con 0x7feda815c7b0 2024-05-27T09:27:28.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.456+0000 7feda5600640 1 -- 172.21.15.114:0/2794436561 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed94005410 con 0x7feda815c7b0 2024-05-27T09:27:28.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.456+0000 7feda5600640 1 -- 172.21.15.114:0/2794436561 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9400a860 con 0x7feda815c7b0 2024-05-27T09:27:28.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.455+0000 7f868aa00640 1 -- 172.21.15.114:0/3941003987 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8684050860 con 0x7f86840cde50 2024-05-27T09:27:28.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.456+0000 7fd35d600640 1 -- 172.21.15.114:0/1907478743 --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd35815b000 con 0x7fd358160b40 2024-05-27T09:27:28.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.456+0000 7fd356000640 1 --2- 172.21.15.114:0/1907478743 >> v2:172.21.15.163:3300/0 conn(0x7fd358074860 0x7fd35815aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.456+0000 7fedafe00640 1 -- 172.21.15.114:0/2794436561 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda8155f50 con 0x7feda815c7b0 2024-05-27T09:27:28.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.456+0000 7fd35d600640 1 -- 172.21.15.114:0/1907478743 --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fd3580cade0 con 0x7fd358074860 2024-05-27T09:27:28.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.456+0000 7fd35d600640 1 -- 172.21.15.114:0/1907478743 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fd3580c3d60 con 0x7fd358164db0 2024-05-27T09:27:28.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.456+0000 7fedafe00640 1 -- 172.21.15.114:0/2794436561 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feda8153540 con 0x7feda815c7b0 2024-05-27T09:27:28.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.457+0000 7fd356a00640 1 -- 172.21.15.114:0/1907478743 >> v2:172.21.15.163:3300/0 conn(0x7fd358074860 msgr2=0x7fd35815aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:28.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.457+0000 7fd356a00640 1 --2- 172.21.15.114:0/1907478743 >> v2:172.21.15.163:3300/0 conn(0x7fd358074860 0x7fd35815aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.457+0000 7fd356a00640 1 -- 172.21.15.114:0/1907478743 >> v1:172.21.15.114:6789/0 conn(0x7fd358160b40 legacy=0x7fd358073310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.457+0000 7fd356a00640 1 -- 172.21.15.114:0/1907478743 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3581615d0 con 0x7fd358164db0 2024-05-27T09:27:28.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.458+0000 7fed9aa00640 1 -- 172.21.15.114:0/2794436561 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed7c005290 con 0x7feda815c7b0 2024-05-27T09:27:28.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.458+0000 7f8681600640 1 -- 172.21.15.114:0/3941003987 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8670043020 con 0x7f86840cde50 2024-05-27T09:27:28.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.458+0000 7f7cf5e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cf0073260 0x7f7cf00c4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.458+0000 7f7cf5e00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cf0073ff0 con 0x7f7cf0073760 2024-05-27T09:27:28.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.458+0000 7f7cf5e00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cf0158780 con 0x7f7cf00ca9d0 2024-05-27T09:27:28.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.458+0000 7f7cf5e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f7cf0158a60 con 0x7f7cf0073260 2024-05-27T09:27:28.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.458+0000 7f7cef400640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f7cf00ca9d0 0x7f7cf0162ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:44716/0 (socket says 172.21.15.114:44716) 2024-05-27T09:27:28.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.458+0000 7f7cef400640 1 -- 172.21.15.114:0/2271493176 learned_addr learned my addr 172.21.15.114:0/2271493176 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:28.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.458+0000 7f7ceea00640 1 --2- 172.21.15.114:0/2271493176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cf0073260 0x7f7cf00c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.459+0000 7fd356a00640 1 --2- 172.21.15.114:0/1907478743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd358164db0 0x7fd3580c3820 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd344009870 tx=0x7fd34402a2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62c48e02a17e01c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.459+0000 7f7ceea00640 1 -- 172.21.15.114:0/2271493176 >> v1:172.21.15.163:6789/0 conn(0x7f7cf00ca9d0 legacy=0x7f7cf0162ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.459+0000 7f7ceea00640 1 -- 172.21.15.114:0/2271493176 >> v1:172.21.15.114:6789/0 conn(0x7f7cf0073760 legacy=0x7f7cf00ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.459+0000 7f7ceea00640 1 -- 172.21.15.114:0/2271493176 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf0158d40 con 0x7f7cf0073260 2024-05-27T09:27:28.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.459+0000 7f8681600640 1 --2- 172.21.15.114:0/3941003987 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f86640763a0 0x7f86640788b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.460+0000 7f8681600640 1 -- 172.21.15.114:0/3941003987 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f86700b78c0 con 0x7f86840cde50 2024-05-27T09:27:28.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.460+0000 7feda5600640 1 -- 172.21.15.114:0/2794436561 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fed9400aa00 con 0x7feda815c7b0 2024-05-27T09:27:28.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.460+0000 7f8689600640 1 --2- 172.21.15.114:0/3941003987 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f86640763a0 0x7f86640788b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.460+0000 7f7ceea00640 1 --2- 172.21.15.114:0/2271493176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cf0073260 0x7f7cf00c4720 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7ce00097d0 tx=0x7f7ce002a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87e9708f3e160d4a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.461+0000 7feda5600640 1 --2- 172.21.15.114:0/2794436561 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fed88076350 0x7fed88078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.461+0000 7f8689600640 1 --2- 172.21.15.114:0/3941003987 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f86640763a0 0x7f86640788b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f867c0074d0 tx=0x7f867c00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.461+0000 7feda5600640 1 -- 172.21.15.114:0/2794436561 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fed94097ea0 con 0x7feda815c7b0 2024-05-27T09:27:28.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.461+0000 7fedad400640 1 --2- 172.21.15.114:0/2794436561 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fed88076350 0x7fed88078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.462+0000 7fedad400640 1 --2- 172.21.15.114:0/2794436561 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fed88076350 0x7fed88078860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feda0009160 tx=0x7feda0033000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.462+0000 7f3182000640 1 -- 172.21.15.114:0/4163564550 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c037070 con 0x7f31840c44b0 2024-05-27T09:27:28.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.463+0000 7f3182000640 1 -- 172.21.15.114:0/4163564550 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f316c032440 con 0x7f31840c44b0 2024-05-27T09:27:28.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.463+0000 7f3182000640 1 -- 172.21.15.114:0/4163564550 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c032810 con 0x7f31840c44b0 2024-05-27T09:27:28.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.463+0000 7f7d93400640 1 -- 172.21.15.114:0/3284769196 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80033020 con 0x7f7d94152890 2024-05-27T09:27:28.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.463+0000 7f2bdd600640 1 -- 172.21.15.114:0/2658050711 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2be009e030 con 0x7f2bd804c8c0 2024-05-27T09:27:28.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.463+0000 7f7d93400640 1 -- 172.21.15.114:0/3284769196 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d80007ce0 con 0x7f7d94152890 2024-05-27T09:27:28.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.463+0000 7f7d93400640 1 -- 172.21.15.114:0/3284769196 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d800092f0 con 0x7f7d94152890 2024-05-27T09:27:28.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.464+0000 7f7d9b800640 1 -- 172.21.15.114:0/3284769196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d94152890 msgr2=0x7f7d940c28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.464+0000 7f7d9b800640 1 --2- 172.21.15.114:0/3284769196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d94152890 0x7f7d940c28b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7d940694a0 tx=0x7f7d8002aa20 comp rx=0 tx=0).stop 2024-05-27T09:27:28.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.464+0000 7f3183e00640 1 -- 172.21.15.114:0/4163564550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31840c44b0 msgr2=0x7f3184152990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.464+0000 7f3183e00640 1 --2- 172.21.15.114:0/4163564550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31840c44b0 0x7f3184152990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f316c009870 tx=0x7f316c02a2c0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.464+0000 7f8681600640 1 -- 172.21.15.114:0/3941003987 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f867004a030 con 0x7f86840cde50 2024-05-27T09:27:28.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.464+0000 7f3183e00640 1 -- 172.21.15.114:0/4163564550 shutdown_connections 2024-05-27T09:27:28.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.464+0000 7f3183e00640 1 --2- 172.21.15.114:0/4163564550 >> v2:172.21.15.163:3300/0 conn(0x7f31840cae40 0x7f3184162e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.464+0000 7f3183e00640 1 --2- 172.21.15.114:0/4163564550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31840c44b0 0x7f3184152990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.464+0000 7f3183e00640 1 -- 172.21.15.114:0/4163564550 >> 172.21.15.114:0/4163564550 conn(0x7f31840ca320 msgr2=0x7f31840ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.464+0000 7f7d9b800640 1 -- 172.21.15.114:0/3284769196 shutdown_connections 2024-05-27T09:27:28.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.464+0000 7f7d9b800640 1 --2- 172.21.15.114:0/3284769196 >> v2:172.21.15.163:3300/0 conn(0x7f7d940ce3b0 0x7f7d940b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.464+0000 7f7d9b800640 1 --2- 172.21.15.114:0/3284769196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d94152890 0x7f7d940c28b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.464+0000 7f7d9b800640 1 -- 172.21.15.114:0/3284769196 >> 172.21.15.114:0/3284769196 conn(0x7f7d940cd890 msgr2=0x7f7d940cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.464+0000 7f3183e00640 1 -- 172.21.15.114:0/4163564550 shutdown_connections 2024-05-27T09:27:28.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.464+0000 7f3183e00640 1 -- 172.21.15.114:0/4163564550 wait complete. 2024-05-27T09:27:28.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.464+0000 7f7d9b800640 1 -- 172.21.15.114:0/3284769196 shutdown_connections 2024-05-27T09:27:28.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.466+0000 7f7d9b800640 1 -- 172.21.15.114:0/3284769196 wait complete. 2024-05-27T09:27:28.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.468+0000 7f3183e00640 1 Processor -- start 2024-05-27T09:27:28.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.468+0000 7f3183e00640 1 -- start start 2024-05-27T09:27:28.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.468+0000 7fd355600640 1 -- 172.21.15.114:0/1907478743 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344037070 con 0x7fd358164db0 2024-05-27T09:27:28.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.469+0000 7fd355600640 1 -- 172.21.15.114:0/1907478743 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd344032640 con 0x7fd358164db0 2024-05-27T09:27:28.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.469+0000 7fd355600640 1 -- 172.21.15.114:0/1907478743 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344032a10 con 0x7fd358164db0 2024-05-27T09:27:28.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.469+0000 7feda5600640 1 -- 172.21.15.114:0/2794436561 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fed94061750 con 0x7feda815c7b0 2024-05-27T09:27:28.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.470+0000 7fd35d600640 1 -- 172.21.15.114:0/1907478743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd358164db0 msgr2=0x7fd3580c3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.470+0000 7fd35d600640 1 --2- 172.21.15.114:0/1907478743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd358164db0 0x7fd3580c3820 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd344009870 tx=0x7fd34402a2f0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.470+0000 7f7d9b800640 1 Processor -- start 2024-05-27T09:27:28.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.469+0000 7f7ced600640 1 -- 172.21.15.114:0/2271493176 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0037070 con 0x7f7cf0073260 2024-05-27T09:27:28.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.469+0000 7f7ced600640 1 -- 172.21.15.114:0/2271493176 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ce0032440 con 0x7f7cf0073260 2024-05-27T09:27:28.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.469+0000 7f7ced600640 1 -- 172.21.15.114:0/2271493176 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0032810 con 0x7f7cf0073260 2024-05-27T09:27:28.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.470+0000 7f7cf5e00640 1 -- 172.21.15.114:0/2271493176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cf0073260 msgr2=0x7f7cf00c4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.470+0000 7f7cf5e00640 1 --2- 172.21.15.114:0/2271493176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cf0073260 0x7f7cf00c4720 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7ce00097d0 tx=0x7f7ce002a230 comp rx=0 tx=0).stop 2024-05-27T09:27:28.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.470+0000 7fd35d600640 1 -- 172.21.15.114:0/1907478743 shutdown_connections 2024-05-27T09:27:28.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.470+0000 7f7d9b800640 1 -- start start 2024-05-27T09:27:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.470+0000 7fd35d600640 1 --2- 172.21.15.114:0/1907478743 >> v2:172.21.15.163:3300/0 conn(0x7fd358074860 0x7fd35815aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.470+0000 7fd35d600640 1 --2- 172.21.15.114:0/1907478743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd358164db0 0x7fd3580c3820 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.470+0000 7fd35d600640 1 -- 172.21.15.114:0/1907478743 >> 172.21.15.114:0/1907478743 conn(0x7fd35806f6e0 msgr2=0x7fd358070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.471+0000 7fd35d600640 1 -- 172.21.15.114:0/1907478743 shutdown_connections 2024-05-27T09:27:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.471+0000 7fd35d600640 1 -- 172.21.15.114:0/1907478743 wait complete. 2024-05-27T09:27:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.474+0000 7f7d9b800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d940c2df0 0x7f7d941f4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.474+0000 7f7d9b800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d940ce3b0 0x7f7d941f0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.474+0000 7f7d9b800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d94152890 0x7f7d94140a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.474+0000 7f7d93e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d940ce3b0 0x7f7d941f0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.470+0000 7f3183e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31840c2840 0x7f31841579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.471+0000 7f3183e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31840cae40 0x7f318414e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.471+0000 7f3183e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31841f6b10 0x7f31841f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.471+0000 7f3183e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f31840c7c20 con 0x7f31841f6b10 2024-05-27T09:27:28.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.471+0000 7f3183e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f31840c9d00 con 0x7f31840c2840 2024-05-27T09:27:28.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.471+0000 7f3183e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f31840c2c50 con 0x7f31840cae40 2024-05-27T09:27:28.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.471+0000 7f317be00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31840cae40 0x7f318414e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.471+0000 7f3182a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31840c2840 0x7f31841579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.471+0000 7f3182a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31840c2840 0x7f31841579d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37784/0 (socket says 172.21.15.114:37784) 2024-05-27T09:27:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.471+0000 7f317be00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31840cae40 0x7f318414e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:60582/0 (socket says 172.21.15.114:60582) 2024-05-27T09:27:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.471+0000 7f3182a00640 1 -- 172.21.15.114:0/2657996259 learned_addr learned my addr 172.21.15.114:0/2657996259 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.471+0000 7f3182a00640 1 -- 172.21.15.114:0/2657996259 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31840cae40 msgr2=0x7f318414e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.471+0000 7f3182a00640 1 --2- 172.21.15.114:0/2657996259 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31840cae40 0x7f318414e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.471+0000 7f3182a00640 1 -- 172.21.15.114:0/2657996259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31841f6b10 msgr2=0x7f31841f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.471+0000 7f3182a00640 1 --2- 172.21.15.114:0/2657996259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31841f6b10 0x7f31841f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.471+0000 7f3182a00640 1 -- 172.21.15.114:0/2657996259 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3184157f10 con 0x7f31840c2840 2024-05-27T09:27:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.471+0000 7f317be00640 1 --2- 172.21.15.114:0/2657996259 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31840cae40 0x7f318414e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.472+0000 7f3182a00640 1 --2- 172.21.15.114:0/2657996259 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31840c2840 0x7f31841579d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f316c000c00 tx=0x7f316c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.472+0000 7f317aa00640 1 -- 172.21.15.114:0/2657996259 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c041070 con 0x7f31840c2840 2024-05-27T09:27:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.472+0000 7f317aa00640 1 -- 172.21.15.114:0/2657996259 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f316c03c870 con 0x7f31840c2840 2024-05-27T09:27:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.472+0000 7f317aa00640 1 -- 172.21.15.114:0/2657996259 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c037040 con 0x7f31840c2840 2024-05-27T09:27:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.472+0000 7f3183e00640 1 -- 172.21.15.114:0/2657996259 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3184156930 con 0x7f31840c2840 2024-05-27T09:27:28.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.472+0000 7f3183e00640 1 -- 172.21.15.114:0/2657996259 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3184156ef0 con 0x7f31840c2840 2024-05-27T09:27:28.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.472+0000 7f7cf5e00640 1 -- 172.21.15.114:0/2271493176 shutdown_connections 2024-05-27T09:27:28.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.472+0000 7f7cf5e00640 1 --2- 172.21.15.114:0/2271493176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cf0073260 0x7f7cf00c4720 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.472+0000 7f7cf5e00640 1 -- 172.21.15.114:0/2271493176 >> 172.21.15.114:0/2271493176 conn(0x7f7cf006f6e0 msgr2=0x7f7cf0070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.472+0000 7f7cf5e00640 1 -- 172.21.15.114:0/2271493176 shutdown_connections 2024-05-27T09:27:28.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.472+0000 7f7cf5e00640 1 -- 172.21.15.114:0/2271493176 wait complete. 2024-05-27T09:27:28.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.474+0000 7f7d93e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d940ce3b0 0x7f7d941f0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35636/0 (socket says 172.21.15.114:35636) 2024-05-27T09:27:28.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.475+0000 7f7d93e00640 1 -- 172.21.15.114:0/1363775970 learned_addr learned my addr 172.21.15.114:0/1363775970 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:28.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.475+0000 7f7d9b800640 1 -- 172.21.15.114:0/1363775970 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7d940c8410 con 0x7f7d940ce3b0 2024-05-27T09:27:28.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.475+0000 7f7d9b800640 1 -- 172.21.15.114:0/1363775970 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7d940c8290 con 0x7f7d94152890 2024-05-27T09:27:28.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.475+0000 7f7d9b800640 1 -- 172.21.15.114:0/1363775970 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f7d94152ca0 con 0x7f7d940c2df0 2024-05-27T09:27:28.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.475+0000 7fd35d600640 1 Processor -- start 2024-05-27T09:27:28.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.475+0000 7f317aa00640 1 -- 172.21.15.114:0/2657996259 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f316c044070 con 0x7f31840c2840 2024-05-27T09:27:28.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.475+0000 7f3183e00640 1 -- 172.21.15.114:0/2657996259 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31841580e0 con 0x7f31840c2840 2024-05-27T09:27:28.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.476+0000 7fd35d600640 1 -- start start 2024-05-27T09:27:28.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.476+0000 7f7d93e00640 1 -- 172.21.15.114:0/1363775970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d940c2df0 msgr2=0x7f7d941f4350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T09:27:28.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.476+0000 7f7d93e00640 1 --2- 172.21.15.114:0/1363775970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d940c2df0 0x7f7d941f4350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.476+0000 7f7d93e00640 1 -- 172.21.15.114:0/1363775970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d94152890 msgr2=0x7f7d94140a20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T09:27:28.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.476+0000 7f317aa00640 1 --2- 172.21.15.114:0/2657996259 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f31600762d0 0x7f31600787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.476+0000 7f7d93e00640 1 --2- 172.21.15.114:0/1363775970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d94152890 0x7f7d94140a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.476+0000 7f317aa00640 1 -- 172.21.15.114:0/2657996259 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f316c0b7ff0 con 0x7f31840c2840 2024-05-27T09:27:28.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.476+0000 7f7d93e00640 1 -- 172.21.15.114:0/1363775970 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d941f11e0 con 0x7f7d940ce3b0 2024-05-27T09:27:28.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.477+0000 7f317be00640 1 --2- 172.21.15.114:0/2657996259 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f31600762d0 0x7f31600787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.477+0000 7f7d93e00640 1 --2- 172.21.15.114:0/1363775970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d940ce3b0 0x7f7d941f0ca0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800c820 tx=0x7f7d8800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.477+0000 7f7cf5e00640 1 Processor -- start 2024-05-27T09:27:28.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.477+0000 7f7cf5e00640 1 -- start start 2024-05-27T09:27:28.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.477+0000 7f317be00640 1 --2- 172.21.15.114:0/2657996259 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f31600762d0 0x7f31600787e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f31840cc430 tx=0x7f3174009060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.477+0000 7f7d90c00640 1 -- 172.21.15.114:0/1363775970 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d88015070 con 0x7f7d940ce3b0 2024-05-27T09:27:28.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.478+0000 7f7d90c00640 1 -- 172.21.15.114:0/1363775970 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d88007a30 con 0x7f7d940ce3b0 2024-05-27T09:27:28.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.478+0000 7f7d9b800640 1 -- 172.21.15.114:0/1363775970 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d941f04e0 con 0x7f7d940ce3b0 2024-05-27T09:27:28.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.478+0000 7f7d9b800640 1 -- 172.21.15.114:0/1363775970 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d9415c730 con 0x7f7d940ce3b0 2024-05-27T09:27:28.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.479+0000 7fd35d600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd358074860 0x7fd3580ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.478+0000 7f7d90c00640 1 -- 172.21.15.114:0/1363775970 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d88007de0 con 0x7f7d940ce3b0 2024-05-27T09:27:28.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.480+0000 7fd35d600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd358160b40 0x7fd358149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.480+0000 7fd35d600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd358164db0 0x7fd3581ec410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.480+0000 7f7d9b800640 1 -- 172.21.15.114:0/1363775970 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d70005290 con 0x7f7d940ce3b0 2024-05-27T09:27:28.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.480+0000 7fd35d600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd3581645b0 con 0x7fd358074860 2024-05-27T09:27:28.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.480+0000 7fd356a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd358074860 0x7fd3580ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.480+0000 7fd356a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd358074860 0x7fd3580ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35640/0 (socket says 172.21.15.114:35640) 2024-05-27T09:27:28.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.480+0000 7fd356a00640 1 -- 172.21.15.114:0/1446290160 learned_addr learned my addr 172.21.15.114:0/1446290160 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:28.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.480+0000 7fd35d600640 1 -- 172.21.15.114:0/1446290160 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd3581642d0 con 0x7fd358164db0 2024-05-27T09:27:28.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.480+0000 7fd35d600640 1 -- 172.21.15.114:0/1446290160 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fd358164890 con 0x7fd358160b40 2024-05-27T09:27:28.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.480+0000 7fd357400640 1 --2- 172.21.15.114:0/1446290160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd358164db0 0x7fd3581ec410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.481+0000 7fd356000640 1 --2- 172.21.15.114:0/1446290160 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd358160b40 0x7fd358149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.481+0000 7fd357400640 1 -- 172.21.15.114:0/1446290160 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd358160b40 msgr2=0x7fd358149f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.482+0000 7fd357400640 1 --2- 172.21.15.114:0/1446290160 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd358160b40 0x7fd358149f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.482+0000 7fd357400640 1 -- 172.21.15.114:0/1446290160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd358074860 msgr2=0x7fd3580ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.482+0000 7fd357400640 1 --2- 172.21.15.114:0/1446290160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd358074860 0x7fd3580ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.482+0000 7fd357400640 1 -- 172.21.15.114:0/1446290160 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3580cd190 con 0x7fd358164db0 2024-05-27T09:27:28.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.482+0000 7fd356000640 1 --2- 172.21.15.114:0/1446290160 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd358160b40 0x7fd358149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:27:28.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.482+0000 7fd356a00640 1 --2- 172.21.15.114:0/1446290160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd358074860 0x7fd3580ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:28.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.482+0000 7fd357400640 1 --2- 172.21.15.114:0/1446290160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd358164db0 0x7fd3581ec410 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd35815b1d0 tx=0x7fd35000cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.483+0000 7fd34ea00640 1 -- 172.21.15.114:0/1446290160 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350016020 con 0x7fd358164db0 2024-05-27T09:27:28.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.483+0000 7f317aa00640 1 -- 172.21.15.114:0/2657996259 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f316c04b030 con 0x7f31840c2840 2024-05-27T09:27:28.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.483+0000 7fd35d600640 1 -- 172.21.15.114:0/1446290160 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3581ed0d0 con 0x7fd358164db0 2024-05-27T09:27:28.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.483+0000 7fd35d600640 1 -- 172.21.15.114:0/1446290160 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd358152fb0 con 0x7fd358164db0 2024-05-27T09:27:28.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.483+0000 7fd34ea00640 1 -- 172.21.15.114:0/1446290160 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd350006d90 con 0x7fd358164db0 2024-05-27T09:27:28.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.483+0000 7fd34ea00640 1 -- 172.21.15.114:0/1446290160 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350003bd0 con 0x7fd358164db0 2024-05-27T09:27:28.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.483+0000 7f7d90c00640 1 -- 172.21.15.114:0/1363775970 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7d88018070 con 0x7f7d940ce3b0 2024-05-27T09:27:28.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.485+0000 7f7d90c00640 1 --2- 172.21.15.114:0/1363775970 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7d74076300 0x7f7d74078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.485+0000 7f7d90c00640 1 -- 172.21.15.114:0/1363775970 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7d88097a10 con 0x7f7d940ce3b0 2024-05-27T09:27:28.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.485+0000 7f7d98e00640 1 --2- 172.21.15.114:0/1363775970 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7d74076300 0x7f7d74078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.484+0000 7f7cf5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cf0073260 0x7f7cf00c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.484+0000 7f7cf5e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cf0073760 0x7f7cf01480f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.484+0000 7f7cf5e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf00ca9d0 0x7f7cf0148630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.484+0000 7f7ceea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cf0073260 0x7f7cf00c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.484+0000 7f7cee000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cf0073760 0x7f7cf01480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.485+0000 7f7cf5e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf00744a0 con 0x7f7cf00ca9d0 2024-05-27T09:27:28.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.485+0000 7f7cf5e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf0159540 con 0x7f7cf0073260 2024-05-27T09:27:28.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.485+0000 7f7cf5e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f7cf0074620 con 0x7f7cf0073760 2024-05-27T09:27:28.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.485+0000 7f7cee000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cf0073760 0x7f7cf01480f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:60612/0 (socket says 172.21.15.114:60612) 2024-05-27T09:27:28.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.485+0000 7f7cee000640 1 -- 172.21.15.114:0/1297270551 learned_addr learned my addr 172.21.15.114:0/1297270551 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:28.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.485+0000 7f7cef400640 1 --2- 172.21.15.114:0/1297270551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf00ca9d0 0x7f7cf0148630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.485+0000 7f7cee000640 1 -- 172.21.15.114:0/1297270551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cf0073260 msgr2=0x7f7cf00c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.485+0000 7f7cee000640 1 --2- 172.21.15.114:0/1297270551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cf0073260 0x7f7cf00c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.485+0000 7f7cee000640 1 -- 172.21.15.114:0/1297270551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf00ca9d0 msgr2=0x7f7cf0148630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.486+0000 7f7cee000640 1 --2- 172.21.15.114:0/1297270551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf00ca9d0 0x7f7cf0148630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.486+0000 7f7cee000640 1 -- 172.21.15.114:0/1297270551 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf00c5fc0 con 0x7f7cf0073760 2024-05-27T09:27:28.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.486+0000 7f7cef400640 1 --2- 172.21.15.114:0/1297270551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf00ca9d0 0x7f7cf0148630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:27:28.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.486+0000 7f7cee000640 1 --2- 172.21.15.114:0/1297270551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cf0073760 0x7f7cf01480f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00c390 tx=0x7f7cdc00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.486+0000 7f7ce7400640 1 -- 172.21.15.114:0/1297270551 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc017070 con 0x7f7cf0073760 2024-05-27T09:27:28.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.486+0000 7f7ce7400640 1 -- 172.21.15.114:0/1297270551 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cdc0052d0 con 0x7f7cf0073760 2024-05-27T09:27:28.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.488+0000 7f7ce7400640 1 -- 172.21.15.114:0/1297270551 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc011890 con 0x7f7cf0073760 2024-05-27T09:27:28.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.488+0000 7f7cf5e00640 1 -- 172.21.15.114:0/1297270551 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cf0155e30 con 0x7f7cf0073760 2024-05-27T09:27:28.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.488+0000 7f7cf5e00640 1 -- 172.21.15.114:0/1297270551 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cf0155b50 con 0x7f7cf0073760 2024-05-27T09:27:28.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.488+0000 7fd35d600640 1 -- 172.21.15.114:0/1446290160 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd330005290 con 0x7fd358164db0 2024-05-27T09:27:28.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.490+0000 7fd34ea00640 1 -- 172.21.15.114:0/1446290160 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd350005070 con 0x7fd358164db0 2024-05-27T09:27:28.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.490+0000 7f7d98e00640 1 --2- 172.21.15.114:0/1363775970 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7d74076300 0x7f7d74078810 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7d80034000 tx=0x7f7d80009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.491+0000 7fd34ea00640 1 --2- 172.21.15.114:0/1446290160 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd3380763d0 0x7fd3380788e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.492+0000 7fd34ea00640 1 -- 172.21.15.114:0/1446290160 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd350097040 con 0x7fd358164db0 2024-05-27T09:27:28.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.492+0000 7fd356a00640 1 --2- 172.21.15.114:0/1446290160 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd3380763d0 0x7fd3380788e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.493+0000 7fd356a00640 1 --2- 172.21.15.114:0/1446290160 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd3380763d0 0x7fd3380788e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd34402a330 tx=0x7fd34402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.493+0000 7f7d90c00640 1 -- 172.21.15.114:0/1363775970 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7d880612c0 con 0x7f7d940ce3b0 2024-05-27T09:27:28.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.490+0000 7f7ce4c00640 1 -- 172.21.15.114:0/1297270551 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cc4005290 con 0x7f7cf0073760 2024-05-27T09:27:28.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.492+0000 7f7ce7400640 1 -- 172.21.15.114:0/1297270551 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7cdc01a070 con 0x7f7cf0073760 2024-05-27T09:27:28.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.494+0000 7f7ce7400640 1 --2- 172.21.15.114:0/1297270551 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7cd00762d0 0x7f7cd00787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.494+0000 7f7ce7400640 1 -- 172.21.15.114:0/1297270551 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7cdc0966a0 con 0x7f7cf0073760 2024-05-27T09:27:28.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.494+0000 7f7ceea00640 1 --2- 172.21.15.114:0/1297270551 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7cd00762d0 0x7f7cd00787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.495+0000 7f7ceea00640 1 --2- 172.21.15.114:0/1297270551 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7cd00762d0 0x7f7cd00787e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0034000 tx=0x7f7ce002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.498+0000 7fd34ea00640 1 -- 172.21.15.114:0/1446290160 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd350020020 con 0x7fd358164db0 2024-05-27T09:27:28.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.501+0000 7f7ce7400640 1 -- 172.21.15.114:0/1297270551 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7cdc05f9a0 con 0x7f7cf0073760 2024-05-27T09:27:28.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.504+0000 7f058b600640 1 Processor -- start 2024-05-27T09:27:28.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.504+0000 7f058b600640 1 -- start start 2024-05-27T09:27:28.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.507+0000 7f058b600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0584074a60 0x7f0584154820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.507+0000 7f058b600640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0584073260 0x7f0584073670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.508+0000 7f0583e00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0584073260 0x7f0584073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.508+0000 7f0583e00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0584073260 0x7f0584073670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37812/0 (socket says 172.21.15.114:37812) 2024-05-27T09:27:28.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.508+0000 7f0583e00640 1 -- 172.21.15.114:0/2858181464 learned_addr learned my addr 172.21.15.114:0/2858181464 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:28.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.508+0000 7f058b600640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f058415a810 con 0x7f05840ca2d0 2024-05-27T09:27:28.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.508+0000 7f058b600640 1 -- 172.21.15.114:0/2858181464 --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f058415db60 con 0x7f0584073260 2024-05-27T09:27:28.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.508+0000 7f058b600640 1 -- 172.21.15.114:0/2858181464 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f058415aaf0 con 0x7f0584074a60 2024-05-27T09:27:28.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.508+0000 7f0588c00640 1 --2- 172.21.15.114:0/2858181464 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0584074a60 0x7f0584154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.509+0000 7f0583400640 1 -- 172.21.15.114:0/2858181464 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3360686874 0 0) 0x7f058415a810 con 0x7f05840ca2d0 2024-05-27T09:27:28.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.509+0000 7f0583400640 1 -- 172.21.15.114:0/2858181464 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0568003520 con 0x7f05840ca2d0 2024-05-27T09:27:28.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.509+0000 7f0583e00640 1 -- 172.21.15.114:0/2858181464 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0584074a60 msgr2=0x7f0584154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:28.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.509+0000 7f0583e00640 1 --2- 172.21.15.114:0/2858181464 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0584074a60 0x7f0584154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.509+0000 7f0583e00640 1 -- 172.21.15.114:0/2858181464 >> v1:172.21.15.114:6789/0 conn(0x7f05840ca2d0 legacy=0x7f05840ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.509+0000 7f0583e00640 1 -- 172.21.15.114:0/2858181464 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058415add0 con 0x7f0584073260 2024-05-27T09:27:28.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.511+0000 7f0583e00640 1 --2- 172.21.15.114:0/2858181464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0584073260 0x7f0584073670 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0570009870 tx=0x7f057002a2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=863ad2bd66d9f6dc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.512+0000 7f0583400640 1 -- 172.21.15.114:0/2858181464 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570037070 con 0x7f0584073260 2024-05-27T09:27:28.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.512+0000 7f0583400640 1 -- 172.21.15.114:0/2858181464 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0570032640 con 0x7f0584073260 2024-05-27T09:27:28.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.513+0000 7f0583400640 1 -- 172.21.15.114:0/2858181464 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570037070 con 0x7f0584073260 2024-05-27T09:27:28.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.513+0000 7f058b600640 1 -- 172.21.15.114:0/2858181464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0584073260 msgr2=0x7f0584073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.513+0000 7f058b600640 1 --2- 172.21.15.114:0/2858181464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0584073260 0x7f0584073670 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0570009870 tx=0x7f057002a2f0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.514+0000 7f058b600640 1 -- 172.21.15.114:0/2858181464 shutdown_connections 2024-05-27T09:27:28.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.514+0000 7f058b600640 1 --2- 172.21.15.114:0/2858181464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0584073260 0x7f0584073670 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.514+0000 7f058b600640 1 --2- 172.21.15.114:0/2858181464 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0584074a60 0x7f0584154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.514+0000 7f058b600640 1 -- 172.21.15.114:0/2858181464 >> 172.21.15.114:0/2858181464 conn(0x7f058406f6e0 msgr2=0x7f0584070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.516+0000 7f058b600640 1 -- 172.21.15.114:0/2858181464 shutdown_connections 2024-05-27T09:27:28.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.516+0000 7f058b600640 1 -- 172.21.15.114:0/2858181464 wait complete. 2024-05-27T09:27:28.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.520+0000 7f058b600640 1 Processor -- start 2024-05-27T09:27:28.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.520+0000 7f058b600640 1 -- start start 2024-05-27T09:27:28.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.523+0000 7f058b600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0584073260 0x7f0584144a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.524+0000 7f058b600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0584074a60 0x7f058415cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.524+0000 7f0588c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0584073260 0x7f0584144a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.524+0000 7f0588c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0584073260 0x7f0584144a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37814/0 (socket says 172.21.15.114:37814) 2024-05-27T09:27:28.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.524+0000 7f0588c00640 1 -- 172.21.15.114:0/3009367558 learned_addr learned my addr 172.21.15.114:0/3009367558 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:28.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.524+0000 7f058b600640 1 --2- 172.21.15.114:0/3009367558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05840ca2d0 0x7f0584146260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.525+0000 7f058b600640 1 -- 172.21.15.114:0/3009367558 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f058415a4e0 con 0x7f05840ca2d0 2024-05-27T09:27:28.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.525+0000 7f058b600640 1 -- 172.21.15.114:0/3009367558 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f058415b390 con 0x7f0584073260 2024-05-27T09:27:28.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.525+0000 7f058b600640 1 -- 172.21.15.114:0/3009367558 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f058415dff0 con 0x7f0584074a60 2024-05-27T09:27:28.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.525+0000 7f0583e00640 1 --2- 172.21.15.114:0/3009367558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0584074a60 0x7f058415cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.525+0000 7f0589600640 1 --2- 172.21.15.114:0/3009367558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05840ca2d0 0x7f0584146260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.527+0000 7f0589600640 1 -- 172.21.15.114:0/3009367558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0584074a60 msgr2=0x7f058415cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.527+0000 7f0589600640 1 --2- 172.21.15.114:0/3009367558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0584074a60 0x7f058415cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.527+0000 7f0589600640 1 -- 172.21.15.114:0/3009367558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0584073260 msgr2=0x7f0584144a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.527+0000 7f0589600640 1 --2- 172.21.15.114:0/3009367558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0584073260 0x7f0584144a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.527+0000 7f0589600640 1 -- 172.21.15.114:0/3009367558 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0584144fb0 con 0x7f05840ca2d0 2024-05-27T09:27:28.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.527+0000 7f0589600640 1 --2- 172.21.15.114:0/3009367558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05840ca2d0 0x7f0584146260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0574001dc0 tx=0x7f057400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.531+0000 7f0580c00640 1 -- 172.21.15.114:0/3009367558 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05740035f0 con 0x7f05840ca2d0 2024-05-27T09:27:28.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.531+0000 7f0580c00640 1 -- 172.21.15.114:0/3009367558 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0574015030 con 0x7f05840ca2d0 2024-05-27T09:27:28.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.531+0000 7f0580c00640 1 -- 172.21.15.114:0/3009367558 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574003a10 con 0x7f05840ca2d0 2024-05-27T09:27:28.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.531+0000 7f058b600640 1 -- 172.21.15.114:0/3009367558 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0584153de0 con 0x7f05840ca2d0 2024-05-27T09:27:28.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.531+0000 7f058b600640 1 -- 172.21.15.114:0/3009367558 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0584151e00 con 0x7f05840ca2d0 2024-05-27T09:27:28.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.533+0000 7f058b600640 1 -- 172.21.15.114:0/3009367558 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f058406a380 con 0x7f05840ca2d0 2024-05-27T09:27:28.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.536+0000 7f0580c00640 1 -- 172.21.15.114:0/3009367558 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0574003bb0 con 0x7f05840ca2d0 2024-05-27T09:27:28.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.538+0000 7f0580c00640 1 --2- 172.21.15.114:0/3009367558 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f05640762d0 0x7f05640787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:28.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.538+0000 7f0580c00640 1 -- 172.21.15.114:0/3009367558 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f057409ab60 con 0x7f05840ca2d0 2024-05-27T09:27:28.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.539+0000 7f0588c00640 1 --2- 172.21.15.114:0/3009367558 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f05640762d0 0x7f05640787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:28.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.542+0000 7f0588c00640 1 --2- 172.21.15.114:0/3009367558 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f05640762d0 0x7f05640787e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f057c003b80 tx=0x7f057c0098d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:28.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.549+0000 7f0580c00640 1 -- 172.21.15.114:0/3009367558 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0574063e60 con 0x7f05840ca2d0 2024-05-27T09:27:28.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.911+0000 7f2be6c00640 1 -- 172.21.15.114:0/2658050711 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f2bbc005580 con 0x7f2bd804c8c0 2024-05-27T09:27:28.915 INFO:teuthology.orchestra.run.smithi114.stdout:64424509445 2024-05-27T09:27:28.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.912+0000 7f2bdd600640 1 -- 172.21.15.114:0/2658050711 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2be00ed4a0 con 0x7f2bd804c8c0 2024-05-27T09:27:28.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.916+0000 7f2be6c00640 1 -- 172.21.15.114:0/2658050711 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2bc00762d0 msgr2=0x7f2bc00787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.916+0000 7f2be6c00640 1 --2- 172.21.15.114:0/2658050711 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2bc00762d0 0x7f2bc00787e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400a810 tx=0x7f2bd4008040 comp rx=0 tx=0).stop 2024-05-27T09:27:28.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.916+0000 7f2be6c00640 1 -- 172.21.15.114:0/2658050711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bd804c8c0 msgr2=0x7f2bd8006a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.916+0000 7f2be6c00640 1 --2- 172.21.15.114:0/2658050711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bd804c8c0 0x7f2bd8006a00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2be002e1b0 tx=0x7f2be006ab30 comp rx=0 tx=0).stop 2024-05-27T09:27:28.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.917+0000 7f2be6c00640 1 -- 172.21.15.114:0/2658050711 shutdown_connections 2024-05-27T09:27:28.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.917+0000 7f2be6c00640 1 --2- 172.21.15.114:0/2658050711 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2bc00762d0 0x7f2bc00787e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.917+0000 7f2be6c00640 1 --2- 172.21.15.114:0/2658050711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bd804c8c0 0x7f2bd8006a00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.917+0000 7f2be6c00640 1 --2- 172.21.15.114:0/2658050711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bd8045350 0x7f2bd81a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.917+0000 7f2be6c00640 1 --2- 172.21.15.114:0/2658050711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bd8036070 0x7f2bd80ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.917+0000 7f2be6c00640 1 -- 172.21.15.114:0/2658050711 >> 172.21.15.114:0/2658050711 conn(0x7f2bd80503b0 msgr2=0x7f2bd804f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.918+0000 7f2be6c00640 1 -- 172.21.15.114:0/2658050711 shutdown_connections 2024-05-27T09:27:28.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.918+0000 7f2be6c00640 1 -- 172.21.15.114:0/2658050711 wait complete. 2024-05-27T09:27:28.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.923+0000 7fed9aa00640 1 -- 172.21.15.114:0/2794436561 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fed7c005850 con 0x7feda815c7b0 2024-05-27T09:27:28.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.924+0000 7feda5600640 1 -- 172.21.15.114:0/2794436561 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fed94061050 con 0x7feda815c7b0 2024-05-27T09:27:28.925 INFO:teuthology.orchestra.run.smithi114.stdout:64424509445 2024-05-27T09:27:28.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.927+0000 7fed9aa00640 1 -- 172.21.15.114:0/2794436561 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fed88076350 msgr2=0x7fed88078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.927+0000 7fed9aa00640 1 --2- 172.21.15.114:0/2794436561 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fed88076350 0x7fed88078860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feda0009160 tx=0x7feda0033000 comp rx=0 tx=0).stop 2024-05-27T09:27:28.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.927+0000 7fed9aa00640 1 -- 172.21.15.114:0/2794436561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda815c7b0 msgr2=0x7feda8155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.927+0000 7fed9aa00640 1 --2- 172.21.15.114:0/2794436561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda815c7b0 0x7feda8155a10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fed9400c760 tx=0x7fed9400cc20 comp rx=0 tx=0).stop 2024-05-27T09:27:28.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.928+0000 7fed9aa00640 1 -- 172.21.15.114:0/2794436561 shutdown_connections 2024-05-27T09:27:28.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.928+0000 7fed9aa00640 1 --2- 172.21.15.114:0/2794436561 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fed88076350 0x7fed88078860 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.928+0000 7fed9aa00640 1 --2- 172.21.15.114:0/2794436561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feda81fff70 0x7feda80c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.928+0000 7fed9aa00640 1 --2- 172.21.15.114:0/2794436561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda815c7b0 0x7feda8155a10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.928+0000 7fed9aa00640 1 --2- 172.21.15.114:0/2794436561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed90004b10 0x7feda8159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.928+0000 7fed9aa00640 1 -- 172.21.15.114:0/2794436561 >> 172.21.15.114:0/2794436561 conn(0x7feda80cd890 msgr2=0x7feda80c9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.929+0000 7fed9aa00640 1 -- 172.21.15.114:0/2794436561 shutdown_connections 2024-05-27T09:27:28.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.929+0000 7fed9aa00640 1 -- 172.21.15.114:0/2794436561 wait complete. 2024-05-27T09:27:28.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.944+0000 7f868aa00640 1 -- 172.21.15.114:0/3941003987 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f86840cdc30 con 0x7f86840cde50 2024-05-27T09:27:28.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.945+0000 7f8681600640 1 -- 172.21.15.114:0/3941003987 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8670081290 con 0x7f86840cde50 2024-05-27T09:27:28.946 INFO:teuthology.orchestra.run.smithi114.stdout:64424509445 2024-05-27T09:27:28.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.946+0000 7fd35d600640 1 -- 172.21.15.114:0/1446290160 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd330005580 con 0x7fd358164db0 2024-05-27T09:27:28.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.947+0000 7fd34ea00640 1 -- 172.21.15.114:0/1446290160 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd350060990 con 0x7fd358164db0 2024-05-27T09:27:28.948 INFO:teuthology.orchestra.run.smithi114.stdout:60129542149 2024-05-27T09:27:28.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.949+0000 7f868aa00640 1 -- 172.21.15.114:0/3941003987 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f86640763a0 msgr2=0x7f86640788b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.949+0000 7f868aa00640 1 --2- 172.21.15.114:0/3941003987 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f86640763a0 0x7f86640788b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f867c0074d0 tx=0x7f867c00b040 comp rx=0 tx=0).stop 2024-05-27T09:27:28.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.949+0000 7f868aa00640 1 -- 172.21.15.114:0/3941003987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f86840cde50 msgr2=0x7f868414ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.949+0000 7f868aa00640 1 --2- 172.21.15.114:0/3941003987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f86840cde50 0x7f868414ea00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f86840cf550 tx=0x7f86700046c0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.950+0000 7f868aa00640 1 -- 172.21.15.114:0/3941003987 shutdown_connections 2024-05-27T09:27:28.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.950+0000 7f868aa00640 1 --2- 172.21.15.114:0/3941003987 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f86640763a0 0x7f86640788b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.950+0000 7f868aa00640 1 --2- 172.21.15.114:0/3941003987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8684156550 0x7f86841f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.950+0000 7f868aa00640 1 --2- 172.21.15.114:0/3941003987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f86840cde50 0x7f868414ea00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.950+0000 7fd343e00640 1 -- 172.21.15.114:0/1446290160 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd3380763d0 msgr2=0x7fd3380788e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.950+0000 7fd343e00640 1 --2- 172.21.15.114:0/1446290160 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd3380763d0 0x7fd3380788e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd34402a330 tx=0x7fd34402e000 comp rx=0 tx=0).stop 2024-05-27T09:27:28.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.950+0000 7f868aa00640 1 --2- 172.21.15.114:0/3941003987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86840cd630 0x7f868415e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.951+0000 7fd343e00640 1 -- 172.21.15.114:0/1446290160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd358164db0 msgr2=0x7fd3581ec410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.951+0000 7fd343e00640 1 --2- 172.21.15.114:0/1446290160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd358164db0 0x7fd3581ec410 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd35815b1d0 tx=0x7fd35000cb90 comp rx=0 tx=0).stop 2024-05-27T09:27:28.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.950+0000 7f868aa00640 1 -- 172.21.15.114:0/3941003987 >> 172.21.15.114:0/3941003987 conn(0x7f868408d960 msgr2=0x7f86840d1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.951+0000 7fd343e00640 1 -- 172.21.15.114:0/1446290160 shutdown_connections 2024-05-27T09:27:28.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.951+0000 7fd343e00640 1 --2- 172.21.15.114:0/1446290160 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd3380763d0 0x7fd3380788e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.951+0000 7fd343e00640 1 --2- 172.21.15.114:0/1446290160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd358164db0 0x7fd3581ec410 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.951+0000 7fd343e00640 1 --2- 172.21.15.114:0/1446290160 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd358160b40 0x7fd358149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.951+0000 7fd343e00640 1 --2- 172.21.15.114:0/1446290160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd358074860 0x7fd3580ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.951+0000 7fd343e00640 1 -- 172.21.15.114:0/1446290160 >> 172.21.15.114:0/1446290160 conn(0x7fd35806f6e0 msgr2=0x7fd358158f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.951+0000 7f868aa00640 1 -- 172.21.15.114:0/3941003987 shutdown_connections 2024-05-27T09:27:28.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.951+0000 7fd343e00640 1 -- 172.21.15.114:0/1446290160 shutdown_connections 2024-05-27T09:27:28.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.952+0000 7fd343e00640 1 -- 172.21.15.114:0/1446290160 wait complete. 2024-05-27T09:27:28.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.952+0000 7f868aa00640 1 -- 172.21.15.114:0/3941003987 wait complete. 2024-05-27T09:27:28.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.954+0000 7f3183e00640 1 -- 172.21.15.114:0/2657996259 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f3184050860 con 0x7f31840c2840 2024-05-27T09:27:28.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.958+0000 7f317aa00640 1 -- 172.21.15.114:0/2657996259 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f316c081940 con 0x7f31840c2840 2024-05-27T09:27:28.959 INFO:teuthology.orchestra.run.smithi114.stdout:60129542149 2024-05-27T09:27:28.962 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509445 for osd.0 2024-05-27T09:27:28.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.962+0000 7f3183e00640 1 -- 172.21.15.114:0/2657996259 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f31600762d0 msgr2=0x7f31600787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.962+0000 7f3183e00640 1 --2- 172.21.15.114:0/2657996259 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f31600762d0 0x7f31600787e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f31840cc430 tx=0x7f3174009060 comp rx=0 tx=0).stop 2024-05-27T09:27:28.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.962+0000 7f3183e00640 1 -- 172.21.15.114:0/2657996259 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31840c2840 msgr2=0x7f31841579d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.962+0000 7f3183e00640 1 --2- 172.21.15.114:0/2657996259 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31840c2840 0x7f31841579d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f316c000c00 tx=0x7f316c0046c0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.962+0000 7f7ce4c00640 1 -- 172.21.15.114:0/1297270551 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f7cc4005580 con 0x7f7cf0073760 2024-05-27T09:27:28.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.962+0000 7f3183e00640 1 -- 172.21.15.114:0/2657996259 shutdown_connections 2024-05-27T09:27:28.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.962+0000 7f3183e00640 1 --2- 172.21.15.114:0/2657996259 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f31600762d0 0x7f31600787e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.962+0000 7f3183e00640 1 --2- 172.21.15.114:0/2657996259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31841f6b10 0x7f31841f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.962+0000 7f3183e00640 1 --2- 172.21.15.114:0/2657996259 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31840cae40 0x7f318414e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.962+0000 7f3183e00640 1 --2- 172.21.15.114:0/2657996259 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31840c2840 0x7f31841579d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.962+0000 7f3183e00640 1 -- 172.21.15.114:0/2657996259 >> 172.21.15.114:0/2657996259 conn(0x7f31840ca320 msgr2=0x7f3184158d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.962+0000 7f3183e00640 1 -- 172.21.15.114:0/2657996259 shutdown_connections 2024-05-27T09:27:28.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.962+0000 7f3183e00640 1 -- 172.21.15.114:0/2657996259 wait complete. 2024-05-27T09:27:28.966 INFO:teuthology.orchestra.run.smithi114.stdout:60129542149 2024-05-27T09:27:28.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.962+0000 7f7ce7400640 1 -- 172.21.15.114:0/1297270551 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7cdc063ca0 con 0x7f7cf0073760 2024-05-27T09:27:28.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.966+0000 7f7d9b800640 1 -- 172.21.15.114:0/1363775970 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f7d70005850 con 0x7f7d940ce3b0 2024-05-27T09:27:28.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.966+0000 7f7ce4c00640 1 -- 172.21.15.114:0/1297270551 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7cd00762d0 msgr2=0x7f7cd00787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.966+0000 7f7ce4c00640 1 --2- 172.21.15.114:0/1297270551 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7cd00762d0 0x7f7cd00787e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0034000 tx=0x7f7ce002e000 comp rx=0 tx=0).stop 2024-05-27T09:27:28.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.966+0000 7f7ce4c00640 1 -- 172.21.15.114:0/1297270551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cf0073760 msgr2=0x7f7cf01480f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.966+0000 7f7ce4c00640 1 --2- 172.21.15.114:0/1297270551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cf0073760 0x7f7cf01480f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00c390 tx=0x7f7cdc00c850 comp rx=0 tx=0).stop 2024-05-27T09:27:28.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.966+0000 7f7ce4c00640 1 -- 172.21.15.114:0/1297270551 shutdown_connections 2024-05-27T09:27:28.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.966+0000 7f7ce4c00640 1 --2- 172.21.15.114:0/1297270551 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7cd00762d0 0x7f7cd00787e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.966+0000 7f7ce4c00640 1 --2- 172.21.15.114:0/1297270551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7cf00ca9d0 0x7f7cf0148630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.966+0000 7f7ce4c00640 1 --2- 172.21.15.114:0/1297270551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cf0073760 0x7f7cf01480f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.966+0000 7f7ce4c00640 1 --2- 172.21.15.114:0/1297270551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cf0073260 0x7f7cf00c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.966+0000 7f7ce4c00640 1 -- 172.21.15.114:0/1297270551 >> 172.21.15.114:0/1297270551 conn(0x7f7cf006f6e0 msgr2=0x7f7cf0151670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.967+0000 7f7ce4c00640 1 -- 172.21.15.114:0/1297270551 shutdown_connections 2024-05-27T09:27:28.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.967+0000 7f7ce4c00640 1 -- 172.21.15.114:0/1297270551 wait complete. 2024-05-27T09:27:28.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.967+0000 7f7d90c00640 1 -- 172.21.15.114:0/1363775970 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7d88060bc0 con 0x7f7d940ce3b0 2024-05-27T09:27:28.970 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509445 for osd.3 2024-05-27T09:27:28.970 INFO:teuthology.orchestra.run.smithi114.stdout:64424509445 2024-05-27T09:27:28.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.971+0000 7f7d9b800640 1 -- 172.21.15.114:0/1363775970 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7d74076300 msgr2=0x7f7d74078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.971+0000 7f7d9b800640 1 --2- 172.21.15.114:0/1363775970 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7d74076300 0x7f7d74078810 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7d80034000 tx=0x7f7d80009000 comp rx=0 tx=0).stop 2024-05-27T09:27:28.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.971+0000 7f7d9b800640 1 -- 172.21.15.114:0/1363775970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d940ce3b0 msgr2=0x7f7d941f0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.971+0000 7f7d9b800640 1 --2- 172.21.15.114:0/1363775970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d940ce3b0 0x7f7d941f0ca0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800c820 tx=0x7f7d8800cce0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.972+0000 7f7d9b800640 1 -- 172.21.15.114:0/1363775970 shutdown_connections 2024-05-27T09:27:28.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.972+0000 7f7d9b800640 1 --2- 172.21.15.114:0/1363775970 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7d74076300 0x7f7d74078810 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.972+0000 7f7d9b800640 1 --2- 172.21.15.114:0/1363775970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d94152890 0x7f7d94140a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.972+0000 7f7d9b800640 1 --2- 172.21.15.114:0/1363775970 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d940ce3b0 0x7f7d941f0ca0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.972+0000 7f7d9b800640 1 --2- 172.21.15.114:0/1363775970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d940c2df0 0x7f7d941f4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.972+0000 7f7d9b800640 1 -- 172.21.15.114:0/1363775970 >> 172.21.15.114:0/1363775970 conn(0x7f7d940cd890 msgr2=0x7f7d94072f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.972+0000 7f7d9b800640 1 -- 172.21.15.114:0/1363775970 shutdown_connections 2024-05-27T09:27:28.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.972+0000 7f7d9b800640 1 -- 172.21.15.114:0/1363775970 wait complete. 2024-05-27T09:27:28.981 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509445 for osd.1 2024-05-27T09:27:28.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.993+0000 7f058b600640 1 -- 172.21.15.114:0/3009367558 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f0584152160 con 0x7f05840ca2d0 2024-05-27T09:27:28.995 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.4 2024-05-27T09:27:28.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.994+0000 7f0580c00640 1 -- 172.21.15.114:0/3009367558 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0574068160 con 0x7f05840ca2d0 2024-05-27T09:27:28.996 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.7 2024-05-27T09:27:28.996 INFO:teuthology.orchestra.run.smithi114.stdout:60129542149 2024-05-27T09:27:28.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.996+0000 7f058b600640 1 -- 172.21.15.114:0/3009367558 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f05640762d0 msgr2=0x7f05640787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.996+0000 7f058b600640 1 --2- 172.21.15.114:0/3009367558 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f05640762d0 0x7f05640787e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f057c003b80 tx=0x7f057c0098d0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.996+0000 7f058b600640 1 -- 172.21.15.114:0/3009367558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05840ca2d0 msgr2=0x7f0584146260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:28.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.996+0000 7f058b600640 1 --2- 172.21.15.114:0/3009367558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05840ca2d0 0x7f0584146260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0574001dc0 tx=0x7f057400c7c0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.996+0000 7f058b600640 1 -- 172.21.15.114:0/3009367558 shutdown_connections 2024-05-27T09:27:28.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.996+0000 7f058b600640 1 --2- 172.21.15.114:0/3009367558 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f05640762d0 0x7f05640787e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.996+0000 7f058b600640 1 --2- 172.21.15.114:0/3009367558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05840ca2d0 0x7f0584146260 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.996+0000 7f058b600640 1 --2- 172.21.15.114:0/3009367558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0584074a60 0x7f058415cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.996+0000 7f058b600640 1 --2- 172.21.15.114:0/3009367558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0584073260 0x7f0584144a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:28.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.996+0000 7f058b600640 1 -- 172.21.15.114:0/3009367558 >> 172.21.15.114:0/3009367558 conn(0x7f058406f6e0 msgr2=0x7f0584158e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:28.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.996+0000 7f058b600640 1 -- 172.21.15.114:0/3009367558 shutdown_connections 2024-05-27T09:27:28.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:28.996+0000 7f058b600640 1 -- 172.21.15.114:0/3009367558 wait complete. 2024-05-27T09:27:29.001 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.5 2024-05-27T09:27:29.009 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509445 for osd.2 2024-05-27T09:27:29.021 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.6 2024-05-27T09:27:29.963 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-27T09:27:29.971 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-27T09:27:29.982 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-27T09:27:29.996 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-27T09:27:29.997 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-27T09:27:30.002 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-27T09:27:30.010 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-27T09:27:30.021 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-27T09:27:30.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.360+0000 7f54ec800640 1 Processor -- start 2024-05-27T09:27:30.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.361+0000 7f54ec800640 1 -- start start 2024-05-27T09:27:30.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.365+0000 7f54ec800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f54e40c9330 0x7f54e40b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.366+0000 7f54ec800640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54e40c8660 con 0x7f54e40b31c0 2024-05-27T09:27:30.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.366+0000 7f54ec800640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54e40c8940 con 0x7f54e4152890 2024-05-27T09:27:30.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.366+0000 7f54ec800640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f54e40744d0 con 0x7f54e40c9330 2024-05-27T09:27:30.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.366+0000 7f54ea800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f54e40c9330 0x7f54e40b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.366+0000 7f54ea800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f54e40c9330 0x7f54e40b35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35656/0 (socket says 172.21.15.114:35656) 2024-05-27T09:27:30.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.366+0000 7f54ea800640 1 -- 172.21.15.114:0/2595996347 learned_addr learned my addr 172.21.15.114:0/2595996347 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:30.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.368+0000 7f54e8a00640 1 -- 172.21.15.114:0/2595996347 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1386790862 0 0) 0x7f54e40c8660 con 0x7f54e40b31c0 2024-05-27T09:27:30.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.368+0000 7f54e8a00640 1 -- 172.21.15.114:0/2595996347 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54cc003520 con 0x7f54e40b31c0 2024-05-27T09:27:30.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.368+0000 7f54ea800640 1 -- 172.21.15.114:0/2595996347 >> v1:172.21.15.163:6789/0 conn(0x7f54e4152890 legacy=0x7f54e40c28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:30.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.369+0000 7f54ea800640 1 -- 172.21.15.114:0/2595996347 >> v1:172.21.15.114:6789/0 conn(0x7f54e40b31c0 legacy=0x7f54e40c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.369+0000 7f54ea800640 1 -- 172.21.15.114:0/2595996347 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e40747b0 con 0x7f54e40c9330 2024-05-27T09:27:30.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.371+0000 7f54ea800640 1 --2- 172.21.15.114:0/2595996347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54e40c9330 0x7f54e40b35d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009e50 tx=0x7f54e002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf760ad5bfad7571 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.372+0000 7f54e8a00640 1 -- 172.21.15.114:0/2595996347 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0036070 con 0x7f54e40c9330 2024-05-27T09:27:30.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.372+0000 7f54e8a00640 1 -- 172.21.15.114:0/2595996347 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e0031600 con 0x7f54e40c9330 2024-05-27T09:27:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.373+0000 7f54ec800640 1 -- 172.21.15.114:0/2595996347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54e40c9330 msgr2=0x7f54e40b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.373+0000 7f54ec800640 1 --2- 172.21.15.114:0/2595996347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54e40c9330 0x7f54e40b35d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009e50 tx=0x7f54e002a710 comp rx=0 tx=0).stop 2024-05-27T09:27:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.374+0000 7f54ec800640 1 -- 172.21.15.114:0/2595996347 shutdown_connections 2024-05-27T09:27:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.374+0000 7f54ec800640 1 --2- 172.21.15.114:0/2595996347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54e40c9330 0x7f54e40b35d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.375+0000 7f54ec800640 1 -- 172.21.15.114:0/2595996347 >> 172.21.15.114:0/2595996347 conn(0x7f54e40cd890 msgr2=0x7f54e40cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:30.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.375+0000 7f54ec800640 1 -- 172.21.15.114:0/2595996347 shutdown_connections 2024-05-27T09:27:30.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.375+0000 7f54ec800640 1 -- 172.21.15.114:0/2595996347 wait complete. 2024-05-27T09:27:30.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.379+0000 7f54ec800640 1 Processor -- start 2024-05-27T09:27:30.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.379+0000 7f54ec800640 1 -- start start 2024-05-27T09:27:30.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.382+0000 7f54ec800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54e40b31c0 0x7f54e414eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.383+0000 7f54ec800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54e40c9330 0x7f54e414f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.384+0000 7f54ec800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54e4152890 0x7f54e41f1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.384+0000 7f54e9400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54e40c9330 0x7f54e414f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.384+0000 7f54ec800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f54e40c8290 con 0x7f54e40b31c0 2024-05-27T09:27:30.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.384+0000 7f54ec800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f54e40c2fc0 con 0x7f54e4152890 2024-05-27T09:27:30.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.384+0000 7f54ec800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f54e40c8410 con 0x7f54e40c9330 2024-05-27T09:27:30.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.384+0000 7f54ea800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54e4152890 0x7f54e41f1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.384+0000 7f54ea800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54e4152890 0x7f54e41f1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37820/0 (socket says 172.21.15.114:37820) 2024-05-27T09:27:30.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.384+0000 7f54ea800640 1 -- 172.21.15.114:0/2415490503 learned_addr learned my addr 172.21.15.114:0/2415490503 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:30.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.385+0000 7f54e9e00640 1 --2- 172.21.15.114:0/2415490503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54e40b31c0 0x7f54e414eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.385+0000 7f54ea800640 1 -- 172.21.15.114:0/2415490503 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54e40c9330 msgr2=0x7f54e414f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.385+0000 7f54ea800640 1 --2- 172.21.15.114:0/2415490503 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54e40c9330 0x7f54e414f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.385+0000 7f54ea800640 1 -- 172.21.15.114:0/2415490503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54e40b31c0 msgr2=0x7f54e414eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.386+0000 7f54ea800640 1 --2- 172.21.15.114:0/2415490503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54e40b31c0 0x7f54e414eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.386+0000 7f54ea800640 1 -- 172.21.15.114:0/2415490503 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e414c710 con 0x7f54e4152890 2024-05-27T09:27:30.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.386+0000 7f54e9400640 1 --2- 172.21.15.114:0/2415490503 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54e40c9330 0x7f54e414f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:27:30.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.386+0000 7f54e9e00640 1 --2- 172.21.15.114:0/2415490503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54e40b31c0 0x7f54e414eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:27:30.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.387+0000 7f54ea800640 1 --2- 172.21.15.114:0/2415490503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54e4152890 0x7f54e41f1610 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009f40 tx=0x7f54e0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.387+0000 7f54de000640 1 -- 172.21.15.114:0/2415490503 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0041020 con 0x7f54e4152890 2024-05-27T09:27:30.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.387+0000 7f54de000640 1 -- 172.21.15.114:0/2415490503 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e003c470 con 0x7f54e4152890 2024-05-27T09:27:30.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.387+0000 7f54de000640 1 -- 172.21.15.114:0/2415490503 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0036040 con 0x7f54e4152890 2024-05-27T09:27:30.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.389+0000 7f54ec800640 1 -- 172.21.15.114:0/2415490503 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e414c190 con 0x7f54e4152890 2024-05-27T09:27:30.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.389+0000 7f54ec800640 1 -- 172.21.15.114:0/2415490503 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e415b540 con 0x7f54e4152890 2024-05-27T09:27:30.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.391+0000 7f54ec800640 1 -- 172.21.15.114:0/2415490503 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e4050860 con 0x7f54e4152890 2024-05-27T09:27:30.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.397+0000 7f54de000640 1 -- 172.21.15.114:0/2415490503 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f54e0044020 con 0x7f54e4152890 2024-05-27T09:27:30.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.399+0000 7f54de000640 1 --2- 172.21.15.114:0/2415490503 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f54c80762d0 0x7f54c80787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.399+0000 7f54de000640 1 -- 172.21.15.114:0/2415490503 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f54e00b8f90 con 0x7f54e4152890 2024-05-27T09:27:30.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.400+0000 7f54de000640 1 -- 172.21.15.114:0/2415490503 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f54e004b030 con 0x7f54e4152890 2024-05-27T09:27:30.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.404+0000 7f54e9e00640 1 --2- 172.21.15.114:0/2415490503 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f54c80762d0 0x7f54c80787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.408+0000 7f54e9e00640 1 --2- 172.21.15.114:0/2415490503 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f54c80762d0 0x7f54c80787e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f54d8007fd0 tx=0x7f54d8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.411+0000 7f46e7000640 1 Processor -- start 2024-05-27T09:27:30.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.411+0000 7f46e7000640 1 -- start start 2024-05-27T09:27:30.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.414+0000 7f46e7000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f46e00c8c40 0x7f46e015af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.414+0000 7f46e7000640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f46e006c720 0x7f46e006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.414+0000 7f46e7000640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46e0161a40 con 0x7f46e006db40 2024-05-27T09:27:30.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.414+0000 7f46e7000640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f46e00c4df0 con 0x7f46e006c720 2024-05-27T09:27:30.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.414+0000 7f46e7000640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f46e00c50d0 con 0x7f46e00c8c40 2024-05-27T09:27:30.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.415+0000 7f46dfe00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f46e00c8c40 0x7f46e015af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.415+0000 7f46dfe00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f46e00c8c40 0x7f46e015af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37828/0 (socket says 172.21.15.114:37828) 2024-05-27T09:27:30.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.415+0000 7f46dfe00640 1 -- 172.21.15.114:0/1330304226 learned_addr learned my addr 172.21.15.114:0/1330304226 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:30.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.415+0000 7f46dfe00640 1 -- 172.21.15.114:0/1330304226 >> v2:172.21.15.114:3300/0 conn(0x7f46e006c720 msgr2=0x7f46e006cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:30.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.415+0000 7f46dfe00640 1 --2- 172.21.15.114:0/1330304226 >> v2:172.21.15.114:3300/0 conn(0x7f46e006c720 0x7f46e006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.415+0000 7f46dfe00640 1 -- 172.21.15.114:0/1330304226 >> v1:172.21.15.114:6789/0 conn(0x7f46e006db40 legacy=0x7f46e00713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:30.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.415+0000 7f46dfe00640 1 -- 172.21.15.114:0/1330304226 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e00c53b0 con 0x7f46e00c8c40 2024-05-27T09:27:30.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.417+0000 7fc2c3400640 1 Processor -- start 2024-05-27T09:27:30.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.417+0000 7fc2c3400640 1 -- start start 2024-05-27T09:27:30.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.417+0000 7f0a12800640 1 Processor -- start 2024-05-27T09:27:30.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.417+0000 7f46dfe00640 1 --2- 172.21.15.114:0/1330304226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e00c8c40 0x7f46e015af40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f46e00cf060 tx=0x7f46d802a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=551d5f5ea24c4ab8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.418+0000 7f0a12800640 1 -- start start 2024-05-27T09:27:30.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.418+0000 7f46dea00640 1 -- 172.21.15.114:0/1330304226 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8036070 con 0x7f46e00c8c40 2024-05-27T09:27:30.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.418+0000 7f46dea00640 1 -- 172.21.15.114:0/1330304226 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d8031600 con 0x7f46e00c8c40 2024-05-27T09:27:30.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.418+0000 7f46dea00640 1 -- 172.21.15.114:0/1330304226 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d80319d0 con 0x7f46e00c8c40 2024-05-27T09:27:30.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.420+0000 7f46e7000640 1 -- 172.21.15.114:0/1330304226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e00c8c40 msgr2=0x7f46e015af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.420+0000 7f46e7000640 1 --2- 172.21.15.114:0/1330304226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e00c8c40 0x7f46e015af40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f46e00cf060 tx=0x7f46d802a610 comp rx=0 tx=0).stop 2024-05-27T09:27:30.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.420+0000 7f46e7000640 1 -- 172.21.15.114:0/1330304226 shutdown_connections 2024-05-27T09:27:30.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.420+0000 7f46e7000640 1 --2- 172.21.15.114:0/1330304226 >> v2:172.21.15.114:3300/0 conn(0x7f46e006c720 0x7f46e006cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.420+0000 7f46e7000640 1 --2- 172.21.15.114:0/1330304226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e00c8c40 0x7f46e015af40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.420+0000 7f46e7000640 1 -- 172.21.15.114:0/1330304226 >> 172.21.15.114:0/1330304226 conn(0x7f46e00ce240 msgr2=0x7f46e00ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:30.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.420+0000 7fc2c3400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc2bc15c7c0 0x7fc2bc15cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.420+0000 7fc2c3400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc2bc0ce480 0x7fc2bc0ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.420+0000 7f0a12800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a0c0b2c60 0x7f0a0c15d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.420+0000 7f0a12800640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0a0c0ca140 0x7f0a0c0ca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.420+0000 7f0a12800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0a0c073310 0x7f0a0c073720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.420+0000 7f0a12800640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f0a0c15d7c0 con 0x7f0a0c073310 2024-05-27T09:27:30.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.420+0000 7f0a12800640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f0a0c15fd50 con 0x7f0a0c0ca140 2024-05-27T09:27:30.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.420+0000 7f0a12800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f0a0c156f80 con 0x7f0a0c0b2c60 2024-05-27T09:27:30.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.420+0000 7fc2c3400640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2bc0c72d0 con 0x7fc2bc156a70 2024-05-27T09:27:30.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.420+0000 7fc2c3400640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc2bc15d110 con 0x7fc2bc0ce480 2024-05-27T09:27:30.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.420+0000 7fc2c3400640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fc2bc15d3c0 con 0x7fc2bc15c7c0 2024-05-27T09:27:30.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.420+0000 7f46e7000640 1 -- 172.21.15.114:0/1330304226 shutdown_connections 2024-05-27T09:27:30.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.420+0000 7f46e7000640 1 -- 172.21.15.114:0/1330304226 wait complete. 2024-05-27T09:27:30.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.421+0000 7fc2bbe00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc2bc15c7c0 0x7fc2bc15cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.421+0000 7fc2bbe00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc2bc15c7c0 0x7fc2bc15cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37840/0 (socket says 172.21.15.114:37840) 2024-05-27T09:27:30.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.421+0000 7fc2bbe00640 1 -- 172.21.15.114:0/1619852918 learned_addr learned my addr 172.21.15.114:0/1619852918 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:30.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.422+0000 7fc2c1400640 1 --2- 172.21.15.114:0/1619852918 >> v2:172.21.15.114:3300/0 conn(0x7fc2bc0ce480 0x7fc2bc0ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.422+0000 7fc2bbe00640 1 -- 172.21.15.114:0/1619852918 >> v2:172.21.15.114:3300/0 conn(0x7fc2bc0ce480 msgr2=0x7fc2bc0ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:30.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.421+0000 7f0a0aa00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0a0c0ca140 0x7f0a0c0ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.421+0000 7f0a0aa00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0a0c0ca140 0x7f0a0c0ca550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37856/0 (socket says 172.21.15.114:37856) 2024-05-27T09:27:30.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.421+0000 7f0a0aa00640 1 -- 172.21.15.114:0/1240746189 learned_addr learned my addr 172.21.15.114:0/1240746189 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:30.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.422+0000 7f0a0be00640 1 --2- 172.21.15.114:0/1240746189 >> v2:172.21.15.114:3300/0 conn(0x7f0a0c073310 0x7f0a0c073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.422+0000 7f0a0b400640 1 --2- 172.21.15.114:0/1240746189 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a0c0b2c60 0x7f0a0c15d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.422+0000 7f0a0b400640 1 -- 172.21.15.114:0/1240746189 >> v2:172.21.15.163:3300/0 conn(0x7f0a0c0ca140 msgr2=0x7f0a0c0ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:30.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.422+0000 7f0a0b400640 1 --2- 172.21.15.114:0/1240746189 >> v2:172.21.15.163:3300/0 conn(0x7f0a0c0ca140 0x7f0a0c0ca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.422+0000 7f0a0b400640 1 -- 172.21.15.114:0/1240746189 >> v2:172.21.15.114:3300/0 conn(0x7f0a0c073310 msgr2=0x7f0a0c073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:30.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.423+0000 7fc2bb400640 1 -- 172.21.15.114:0/1619852918 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1841244951 0 0) 0x7fc2bc0c72d0 con 0x7fc2bc156a70 2024-05-27T09:27:30.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.423+0000 7f0a0b400640 1 --2- 172.21.15.114:0/1240746189 >> v2:172.21.15.114:3300/0 conn(0x7f0a0c073310 0x7f0a0c073720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.423+0000 7f0a0b400640 1 -- 172.21.15.114:0/1240746189 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0c0cac50 con 0x7f0a0c0b2c60 2024-05-27T09:27:30.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.423+0000 7fc2bbe00640 1 --2- 172.21.15.114:0/1619852918 >> v2:172.21.15.114:3300/0 conn(0x7fc2bc0ce480 0x7fc2bc0ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.423+0000 7fc2bbe00640 1 -- 172.21.15.114:0/1619852918 >> v1:172.21.15.163:6789/0 conn(0x7fc2bc156a70 legacy=0x7fc2bc0b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.423+0000 7fc2bbe00640 1 -- 172.21.15.114:0/1619852918 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2bc0c7570 con 0x7fc2bc15c7c0 2024-05-27T09:27:30.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.423+0000 7fc2c1400640 1 --2- 172.21.15.114:0/1619852918 >> v2:172.21.15.114:3300/0 conn(0x7fc2bc0ce480 0x7fc2bc0ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:30.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.423+0000 7f0a0be00640 1 --2- 172.21.15.114:0/1240746189 >> v2:172.21.15.114:3300/0 conn(0x7f0a0c073310 0x7f0a0c073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:27:30.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.424+0000 7f46e7000640 1 Processor -- start 2024-05-27T09:27:30.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.424+0000 7f46e7000640 1 -- start start 2024-05-27T09:27:30.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.425+0000 7fc2bbe00640 1 --2- 172.21.15.114:0/1619852918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc15c7c0 0x7fc2bc15cbd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc08e990 tx=0x7fc2a802a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3435e4052b43314c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.425+0000 7f0a0b400640 1 --2- 172.21.15.114:0/1240746189 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a0c0b2c60 0x7f0a0c15d280 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c06d7b0 tx=0x7f0a04033260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3829997b85cf9db server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.426+0000 7fc2bb400640 1 -- 172.21.15.114:0/1619852918 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8037070 con 0x7fc2bc15c7c0 2024-05-27T09:27:30.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.426+0000 7fc2bb400640 1 -- 172.21.15.114:0/1619852918 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a802ae50 con 0x7fc2bc15c7c0 2024-05-27T09:27:30.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.426+0000 7fc2bb400640 1 -- 172.21.15.114:0/1619852918 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8032860 con 0x7fc2bc15c7c0 2024-05-27T09:27:30.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.426+0000 7f0a0a000640 1 -- 172.21.15.114:0/1240746189 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0400fb40 con 0x7f0a0c0b2c60 2024-05-27T09:27:30.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.426+0000 7f0a0a000640 1 -- 172.21.15.114:0/1240746189 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a0400fce0 con 0x7f0a0c0b2c60 2024-05-27T09:27:30.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.426+0000 7f0a0a000640 1 -- 172.21.15.114:0/1240746189 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a040112f0 con 0x7f0a0c0b2c60 2024-05-27T09:27:30.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.426+0000 7fed6f000640 1 Processor -- start 2024-05-27T09:27:30.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.427+0000 7fed6f000640 1 -- start start 2024-05-27T09:27:30.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.427+0000 7f46e7000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46e006c720 0x7f46e0155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.427+0000 7f46e7000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46e006db40 0x7f46e014e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.427+0000 7f0a12800640 1 -- 172.21.15.114:0/1240746189 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a0c0b2c60 msgr2=0x7f0a0c15d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.427+0000 7f0a12800640 1 --2- 172.21.15.114:0/1240746189 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a0c0b2c60 0x7f0a0c15d280 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c06d7b0 tx=0x7f0a04033260 comp rx=0 tx=0).stop 2024-05-27T09:27:30.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.427+0000 7fc2c3400640 1 -- 172.21.15.114:0/1619852918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc15c7c0 msgr2=0x7fc2bc15cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.427+0000 7fc2c3400640 1 --2- 172.21.15.114:0/1619852918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc15c7c0 0x7fc2bc15cbd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc08e990 tx=0x7fc2a802a300 comp rx=0 tx=0).stop 2024-05-27T09:27:30.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.428+0000 7f0a12800640 1 -- 172.21.15.114:0/1240746189 shutdown_connections 2024-05-27T09:27:30.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.428+0000 7fc2c3400640 1 -- 172.21.15.114:0/1619852918 shutdown_connections 2024-05-27T09:27:30.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.428+0000 7fc2c3400640 1 --2- 172.21.15.114:0/1619852918 >> v2:172.21.15.114:3300/0 conn(0x7fc2bc0ce480 0x7fc2bc0ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.428+0000 7f0a12800640 1 --2- 172.21.15.114:0/1240746189 >> v2:172.21.15.114:3300/0 conn(0x7f0a0c073310 0x7f0a0c073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.428+0000 7fc2c3400640 1 --2- 172.21.15.114:0/1619852918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc15c7c0 0x7fc2bc15cbd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.428+0000 7fc2c3400640 1 -- 172.21.15.114:0/1619852918 >> 172.21.15.114:0/1619852918 conn(0x7fc2bc06f2a0 msgr2=0x7fc2bc068600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:30.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.428+0000 7f0a12800640 1 --2- 172.21.15.114:0/1240746189 >> v2:172.21.15.163:3300/0 conn(0x7f0a0c0ca140 0x7f0a0c0ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.428+0000 7f0a12800640 1 --2- 172.21.15.114:0/1240746189 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a0c0b2c60 0x7f0a0c15d280 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.428+0000 7f0a12800640 1 -- 172.21.15.114:0/1240746189 >> 172.21.15.114:0/1240746189 conn(0x7f0a0c06f6e0 msgr2=0x7f0a0c070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:30.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.428+0000 7f46dfe00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46e006c720 0x7f46e0155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.428+0000 7f46dfe00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46e006c720 0x7f46e0155840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:60646/0 (socket says 172.21.15.114:60646) 2024-05-27T09:27:30.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.428+0000 7f46dfe00640 1 -- 172.21.15.114:0/4230681227 learned_addr learned my addr 172.21.15.114:0/4230681227 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:30.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.429+0000 7fc2c3400640 1 -- 172.21.15.114:0/1619852918 shutdown_connections 2024-05-27T09:27:30.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.429+0000 7fc2c3400640 1 -- 172.21.15.114:0/1619852918 wait complete. 2024-05-27T09:27:30.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.428+0000 7f46e7000640 1 --2- 172.21.15.114:0/4230681227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e00c8c40 0x7f46e014e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.428+0000 7f46e7000640 1 -- 172.21.15.114:0/4230681227 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f46e006d070 con 0x7f46e006db40 2024-05-27T09:27:30.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.428+0000 7f46e7000640 1 -- 172.21.15.114:0/4230681227 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f46e00c5ac0 con 0x7f46e00c8c40 2024-05-27T09:27:30.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.428+0000 7f46e7000640 1 -- 172.21.15.114:0/4230681227 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f46e0161670 con 0x7f46e006c720 2024-05-27T09:27:30.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.429+0000 7f46e5000640 1 --2- 172.21.15.114:0/4230681227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e00c8c40 0x7f46e014e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.429+0000 7f46e5000640 1 -- 172.21.15.114:0/4230681227 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46e006c720 msgr2=0x7f46e0155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.429+0000 7f46e5000640 1 --2- 172.21.15.114:0/4230681227 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46e006c720 0x7f46e0155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.429+0000 7f46e5000640 1 -- 172.21.15.114:0/4230681227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46e006db40 msgr2=0x7f46e014e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.429+0000 7fed6f000640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fed68164db0 0x7fed680c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.429+0000 7fed6f000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed68074860 0x7fed6815aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.430+0000 7fed6f000640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed6815b000 con 0x7fed68160b40 2024-05-27T09:27:30.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.430+0000 7fed6f000640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fed680cade0 con 0x7fed68164db0 2024-05-27T09:27:30.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.430+0000 7fed6f000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fed680c3d60 con 0x7fed68074860 2024-05-27T09:27:30.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.430+0000 7fed67e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fed68164db0 0x7fed680c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.430+0000 7fed67e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fed68164db0 0x7fed680c3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35720/0 (socket says 172.21.15.114:35720) 2024-05-27T09:27:30.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.429+0000 7f46df400640 1 --2- 172.21.15.114:0/4230681227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46e006db40 0x7f46e014e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.430+0000 7f0a12800640 1 -- 172.21.15.114:0/1240746189 shutdown_connections 2024-05-27T09:27:30.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.430+0000 7fed67e00640 1 -- 172.21.15.114:0/522054585 learned_addr learned my addr 172.21.15.114:0/522054585 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:30.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.430+0000 7fed67400640 1 --2- 172.21.15.114:0/522054585 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed68074860 0x7fed6815aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.431+0000 7fed67e00640 1 -- 172.21.15.114:0/522054585 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed68074860 msgr2=0x7fed6815aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:30.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.431+0000 7fed67e00640 1 --2- 172.21.15.114:0/522054585 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed68074860 0x7fed6815aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.431+0000 7fed67e00640 1 -- 172.21.15.114:0/522054585 >> v1:172.21.15.114:6789/0 conn(0x7fed68160b40 legacy=0x7fed68073310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:30.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.431+0000 7fed67e00640 1 -- 172.21.15.114:0/522054585 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed681615d0 con 0x7fed68164db0 2024-05-27T09:27:30.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.431+0000 7f46e5000640 1 --2- 172.21.15.114:0/4230681227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46e006db40 0x7f46e014e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.431+0000 7f46e5000640 1 -- 172.21.15.114:0/4230681227 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e0157000 con 0x7f46e00c8c40 2024-05-27T09:27:30.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.431+0000 7f46e5000640 1 --2- 172.21.15.114:0/4230681227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e00c8c40 0x7f46e014e9e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f46c800b360 tx=0x7f46c800b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.432+0000 7f46d7e00640 1 -- 172.21.15.114:0/4230681227 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8015020 con 0x7f46e00c8c40 2024-05-27T09:27:30.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.432+0000 7f46d7e00640 1 -- 172.21.15.114:0/4230681227 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c80118f0 con 0x7f46e00c8c40 2024-05-27T09:27:30.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.432+0000 7f46d7e00640 1 -- 172.21.15.114:0/4230681227 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8011ce0 con 0x7f46e00c8c40 2024-05-27T09:27:30.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.432+0000 7f46e7000640 1 -- 172.21.15.114:0/4230681227 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e01f6940 con 0x7f46e00c8c40 2024-05-27T09:27:30.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.433+0000 7f46e7000640 1 -- 172.21.15.114:0/4230681227 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e01f6f00 con 0x7f46e00c8c40 2024-05-27T09:27:30.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.433+0000 7fed67e00640 1 --2- 172.21.15.114:0/522054585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed68164db0 0x7fed680c3820 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fed6806d780 tx=0x7fed6002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab0ae81005fc780 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.434+0000 7fed66a00640 1 -- 172.21.15.114:0/522054585 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60036070 con 0x7fed68164db0 2024-05-27T09:27:30.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.434+0000 7fed66a00640 1 -- 172.21.15.114:0/522054585 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed60031600 con 0x7fed68164db0 2024-05-27T09:27:30.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.434+0000 7fed66a00640 1 -- 172.21.15.114:0/522054585 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed600319d0 con 0x7fed68164db0 2024-05-27T09:27:30.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.433+0000 7fa94a000640 1 Processor -- start 2024-05-27T09:27:30.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.433+0000 7fc2c3400640 1 Processor -- start 2024-05-27T09:27:30.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.434+0000 7fc2c3400640 1 -- start start 2024-05-27T09:27:30.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.430+0000 7f0a12800640 1 -- 172.21.15.114:0/1240746189 wait complete. 2024-05-27T09:27:30.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.434+0000 7f46df400640 1 --2- 172.21.15.114:0/4230681227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46e006db40 0x7f46e014e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:27:30.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.435+0000 7f46e7000640 1 -- 172.21.15.114:0/4230681227 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46e006a380 con 0x7f46e00c8c40 2024-05-27T09:27:30.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.434+0000 7fa94a000640 1 -- start start 2024-05-27T09:27:30.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.435+0000 7f0a12800640 1 Processor -- start 2024-05-27T09:27:30.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.436+0000 7f0a12800640 1 -- start start 2024-05-27T09:27:30.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.436+0000 7fed6f000640 1 -- 172.21.15.114:0/522054585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed68164db0 msgr2=0x7fed680c3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.436+0000 7fed6f000640 1 --2- 172.21.15.114:0/522054585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed68164db0 0x7fed680c3820 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fed6806d780 tx=0x7fed6002a6f0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.436+0000 7fc2c3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc0ce480 0x7fc2bc0ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.437+0000 7fc2c3400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2bc156a70 0x7fc2bc0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.437+0000 7fa94a000640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa944164db0 0x7fa9440c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.437+0000 7fa94a000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa944074860 0x7fa94415aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.437+0000 7fa94a000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944160b40 0x7fa944160f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.437+0000 7fa94a000640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa9440cade0 con 0x7fa944164db0 2024-05-27T09:27:30.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.437+0000 7fa94a000640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fa9440c3d60 con 0x7fa944074860 2024-05-27T09:27:30.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.437+0000 7fa94a000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa94415b000 con 0x7fa944160b40 2024-05-27T09:27:30.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.437+0000 7fed6f000640 1 -- 172.21.15.114:0/522054585 shutdown_connections 2024-05-27T09:27:30.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.437+0000 7fed6f000640 1 --2- 172.21.15.114:0/522054585 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed68074860 0x7fed6815aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.437+0000 7fed6f000640 1 --2- 172.21.15.114:0/522054585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed68164db0 0x7fed680c3820 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.437+0000 7fed6f000640 1 -- 172.21.15.114:0/522054585 >> 172.21.15.114:0/522054585 conn(0x7fed6806f6e0 msgr2=0x7fed68070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:30.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.437+0000 7fa942a00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa944074860 0x7fa94415aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.437+0000 7f46d7e00640 1 -- 172.21.15.114:0/4230681227 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f46c8018020 con 0x7f46e00c8c40 2024-05-27T09:27:30.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.437+0000 7fa942a00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa944074860 0x7fa94415aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37884/0 (socket says 172.21.15.114:37884) 2024-05-27T09:27:30.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.437+0000 7fa942a00640 1 -- 172.21.15.114:0/2965358272 learned_addr learned my addr 172.21.15.114:0/2965358272 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:30.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.437+0000 7fa943e00640 1 --2- 172.21.15.114:0/2965358272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944160b40 0x7fa944160f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.440+0000 7f0a12800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a0c073310 0x7f0a0c150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.440+0000 7f0a12800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a0c0b2c60 0x7f0a0c153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.440+0000 7f0a12800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a0c0ca140 0x7f0a0c14f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.440+0000 7f0a12800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c160560 con 0x7f0a0c0ca140 2024-05-27T09:27:30.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.440+0000 7f0a12800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c160280 con 0x7f0a0c073310 2024-05-27T09:27:30.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.440+0000 7f0a12800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f0a0c160840 con 0x7f0a0c0b2c60 2024-05-27T09:27:30.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.439+0000 7f46d7e00640 1 --2- 172.21.15.114:0/4230681227 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f46bc0762d0 0x7f46bc0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.439+0000 7f46d7e00640 1 -- 172.21.15.114:0/4230681227 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f46c8097610 con 0x7f46e00c8c40 2024-05-27T09:27:30.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.439+0000 7f46dfe00640 1 --2- 172.21.15.114:0/4230681227 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f46bc0762d0 0x7f46bc0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.440+0000 7f46dfe00640 1 --2- 172.21.15.114:0/4230681227 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f46bc0762d0 0x7f46bc0787e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f46d8000f80 tx=0x7f46d8009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.437+0000 7fc2c0a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc0ce480 0x7fc2bc0ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.437+0000 7fc2c0a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc0ce480 0x7fc2bc0ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37876/0 (socket says 172.21.15.114:37876) 2024-05-27T09:27:30.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.438+0000 7fc2c3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2bc15c7c0 0x7fc2bc14cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.438+0000 7fc2c3400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc15d6a0 con 0x7fc2bc15c7c0 2024-05-27T09:27:30.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.438+0000 7fc2c3400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc0c7b30 con 0x7fc2bc0ce480 2024-05-27T09:27:30.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.438+0000 7fc2c3400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fc2bc0c8050 con 0x7fc2bc156a70 2024-05-27T09:27:30.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.438+0000 7fc2c0a00640 1 -- 172.21.15.114:0/410353100 learned_addr learned my addr 172.21.15.114:0/410353100 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:30.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.438+0000 7fc2bbe00640 1 --2- 172.21.15.114:0/410353100 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2bc156a70 0x7fc2bc0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.438+0000 7fc2c0a00640 1 -- 172.21.15.114:0/410353100 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2bc156a70 msgr2=0x7fc2bc0cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.438+0000 7fc2c0a00640 1 --2- 172.21.15.114:0/410353100 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2bc156a70 0x7fc2bc0cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.438+0000 7fc2c0a00640 1 -- 172.21.15.114:0/410353100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2bc15c7c0 msgr2=0x7fc2bc14cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.438+0000 7fc2c1400640 1 --2- 172.21.15.114:0/410353100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2bc15c7c0 0x7fc2bc14cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.439+0000 7fc2c0a00640 1 --2- 172.21.15.114:0/410353100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2bc15c7c0 0x7fc2bc14cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.439+0000 7fc2c0a00640 1 -- 172.21.15.114:0/410353100 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2bc0cfd20 con 0x7fc2bc0ce480 2024-05-27T09:27:30.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.439+0000 7fc2c1400640 1 --2- 172.21.15.114:0/410353100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2bc15c7c0 0x7fc2bc14cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:27:30.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.440+0000 7fc2c0a00640 1 --2- 172.21.15.114:0/410353100 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc0ce480 0x7fc2bc0ca830 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000c3f0 tx=0x7fc2b000c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.438+0000 7fed6f000640 1 -- 172.21.15.114:0/522054585 shutdown_connections 2024-05-27T09:27:30.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.438+0000 7fed6f000640 1 -- 172.21.15.114:0/522054585 wait complete. 2024-05-27T09:27:30.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.438+0000 7fa943400640 1 --2- 172.21.15.114:0/2965358272 >> v2:172.21.15.114:3300/0 conn(0x7fa944164db0 0x7fa9440c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.438+0000 7fa942a00640 1 -- 172.21.15.114:0/2965358272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944160b40 msgr2=0x7fa944160f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:30.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.438+0000 7fa942a00640 1 --2- 172.21.15.114:0/2965358272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944160b40 0x7fa944160f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.438+0000 7fa942a00640 1 -- 172.21.15.114:0/2965358272 >> v2:172.21.15.114:3300/0 conn(0x7fa944164db0 msgr2=0x7fa9440c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:30.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.438+0000 7fa942a00640 1 --2- 172.21.15.114:0/2965358272 >> v2:172.21.15.114:3300/0 conn(0x7fa944164db0 0x7fa9440c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.438+0000 7fa942a00640 1 -- 172.21.15.114:0/2965358272 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9441615d0 con 0x7fa944074860 2024-05-27T09:27:30.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.438+0000 7fa943400640 1 --2- 172.21.15.114:0/2965358272 >> v2:172.21.15.114:3300/0 conn(0x7fa944164db0 0x7fa9440c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:27:30.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.439+0000 7f1624600640 1 Processor -- start 2024-05-27T09:27:30.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.439+0000 7f1624600640 1 -- start start 2024-05-27T09:27:30.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.440+0000 7f0a0b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a0c073310 0x7f0a0c150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.440+0000 7f0a0b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a0c073310 0x7f0a0c150c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37894/0 (socket says 172.21.15.114:37894) 2024-05-27T09:27:30.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.441+0000 7f0a0b400640 1 -- 172.21.15.114:0/811045973 learned_addr learned my addr 172.21.15.114:0/811045973 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:30.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.441+0000 7f0a0aa00640 1 --2- 172.21.15.114:0/811045973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a0c0b2c60 0x7f0a0c153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.441+0000 7f0a0b400640 1 -- 172.21.15.114:0/811045973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a0c0b2c60 msgr2=0x7f0a0c153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.441+0000 7f0a0b400640 1 --2- 172.21.15.114:0/811045973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a0c0b2c60 0x7f0a0c153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.441+0000 7f0a0b400640 1 -- 172.21.15.114:0/811045973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a0c0ca140 msgr2=0x7f0a0c14f410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T09:27:30.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.441+0000 7f1624600640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f161c073780 0x7f161c152a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.441+0000 7f1624600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f161c0c4720 0x7f161c15bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.441+0000 7f1624600640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f161c15c1d0 0x7f161c0ca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.441+0000 7f1624600640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f161c0c8590 con 0x7f161c073780 2024-05-27T09:27:30.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.441+0000 7f1624600640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f161c1586a0 con 0x7f161c15c1d0 2024-05-27T09:27:30.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.441+0000 7f1624600640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f161c073c50 con 0x7f161c0c4720 2024-05-27T09:27:30.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.441+0000 7f0a0b400640 1 --2- 172.21.15.114:0/811045973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a0c0ca140 0x7f0a0c14f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.441+0000 7f0a0b400640 1 -- 172.21.15.114:0/811045973 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0c14f950 con 0x7f0a0c073310 2024-05-27T09:27:30.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.441+0000 7fa942a00640 1 --2- 172.21.15.114:0/2965358272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa944074860 0x7fa94415aac0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa944071e10 tx=0x7fa93002a260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=565fe97d0c35d454 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.442+0000 7f1621200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f161c0c4720 0x7f161c15bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.442+0000 7f0a0b400640 1 --2- 172.21.15.114:0/811045973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a0c073310 0x7f0a0c150c60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0a04033850 tx=0x7f0a04004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.442+0000 7f1621200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f161c0c4720 0x7f161c15bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:60698/0 (socket says 172.21.15.114:60698) 2024-05-27T09:27:30.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.442+0000 7f1621200640 1 -- 172.21.15.114:0/3272672812 learned_addr learned my addr 172.21.15.114:0/3272672812 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:30.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.442+0000 7f1621c00640 1 --2- 172.21.15.114:0/3272672812 >> v2:172.21.15.114:3300/0 conn(0x7f161c073780 0x7f161c152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.442+0000 7f1622600640 1 --2- 172.21.15.114:0/3272672812 >> v2:172.21.15.163:3300/0 conn(0x7f161c15c1d0 0x7f161c0ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.442+0000 7fed6f000640 1 Processor -- start 2024-05-27T09:27:30.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.442+0000 7f1621200640 1 -- 172.21.15.114:0/3272672812 >> v2:172.21.15.163:3300/0 conn(0x7f161c15c1d0 msgr2=0x7f161c0ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:30.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.442+0000 7fed6f000640 1 -- start start 2024-05-27T09:27:30.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.442+0000 7f1621200640 1 --2- 172.21.15.114:0/3272672812 >> v2:172.21.15.163:3300/0 conn(0x7f161c15c1d0 0x7f161c0ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.442+0000 7f1621200640 1 -- 172.21.15.114:0/3272672812 >> v2:172.21.15.114:3300/0 conn(0x7f161c073780 msgr2=0x7f161c152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:30.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.442+0000 7f1621200640 1 --2- 172.21.15.114:0/3272672812 >> v2:172.21.15.114:3300/0 conn(0x7f161c073780 0x7f161c152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.443+0000 7f1621200640 1 -- 172.21.15.114:0/3272672812 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161c158960 con 0x7f161c0c4720 2024-05-27T09:27:30.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.442+0000 7f0a03400640 1 -- 172.21.15.114:0/811045973 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0403bbb0 con 0x7f0a0c073310 2024-05-27T09:27:30.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.443+0000 7fa942000640 1 -- 172.21.15.114:0/2965358272 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930037070 con 0x7fa944074860 2024-05-27T09:27:30.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.443+0000 7fa942000640 1 -- 172.21.15.114:0/2965358272 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa930032440 con 0x7fa944074860 2024-05-27T09:27:30.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.443+0000 7fa942000640 1 -- 172.21.15.114:0/2965358272 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930032810 con 0x7fa944074860 2024-05-27T09:27:30.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.443+0000 7f1621c00640 1 --2- 172.21.15.114:0/3272672812 >> v2:172.21.15.114:3300/0 conn(0x7f161c073780 0x7f161c152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:27:30.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.443+0000 7f0a03400640 1 -- 172.21.15.114:0/811045973 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a040119b0 con 0x7f0a0c073310 2024-05-27T09:27:30.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.443+0000 7f0a03400640 1 -- 172.21.15.114:0/811045973 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0403bbb0 con 0x7f0a0c073310 2024-05-27T09:27:30.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.443+0000 7fc2b8c00640 1 -- 172.21.15.114:0/410353100 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0017070 con 0x7fc2bc0ce480 2024-05-27T09:27:30.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.443+0000 7fc2b8c00640 1 -- 172.21.15.114:0/410353100 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2b00052f0 con 0x7fc2bc0ce480 2024-05-27T09:27:30.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.443+0000 7fc2b8c00640 1 -- 172.21.15.114:0/410353100 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0003850 con 0x7fc2bc0ce480 2024-05-27T09:27:30.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.443+0000 7fc2c3400640 1 -- 172.21.15.114:0/410353100 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2bc214190 con 0x7fc2bc0ce480 2024-05-27T09:27:30.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.443+0000 7fc2c3400640 1 -- 172.21.15.114:0/410353100 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2bc0cbcb0 con 0x7fc2bc0ce480 2024-05-27T09:27:30.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.444+0000 7f0a12800640 1 -- 172.21.15.114:0/811045973 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a0c151770 con 0x7f0a0c073310 2024-05-27T09:27:30.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.444+0000 7f0a12800640 1 -- 172.21.15.114:0/811045973 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a0c151cf0 con 0x7f0a0c073310 2024-05-27T09:27:30.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.444+0000 7f46d7e00640 1 -- 172.21.15.114:0/4230681227 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f46c801f020 con 0x7f46e00c8c40 2024-05-27T09:27:30.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.444+0000 7fa94a000640 1 -- 172.21.15.114:0/2965358272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa944074860 msgr2=0x7fa94415aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.444+0000 7fa94a000640 1 --2- 172.21.15.114:0/2965358272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa944074860 0x7fa94415aac0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa944071e10 tx=0x7fa93002a260 comp rx=0 tx=0).stop 2024-05-27T09:27:30.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.444+0000 7fed6f000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed68074860 0x7fed68154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.445+0000 7fed67e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed68074860 0x7fed68154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.445+0000 7fed67e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed68074860 0x7fed68154730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35788/0 (socket says 172.21.15.114:35788) 2024-05-27T09:27:30.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.445+0000 7fed67e00640 1 -- 172.21.15.114:0/839899574 learned_addr learned my addr 172.21.15.114:0/839899574 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:30.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.445+0000 7fed6f000640 1 --2- 172.21.15.114:0/839899574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed68160b40 0x7fed68163560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.445+0000 7fed6f000640 1 --2- 172.21.15.114:0/839899574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed68164db0 0x7fed68158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.445+0000 7fed6f000640 1 -- 172.21.15.114:0/839899574 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fed681645b0 con 0x7fed68074860 2024-05-27T09:27:30.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.445+0000 7fed6f000640 1 -- 172.21.15.114:0/839899574 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fed681642d0 con 0x7fed68160b40 2024-05-27T09:27:30.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.445+0000 7fed6f000640 1 -- 172.21.15.114:0/839899574 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fed68164890 con 0x7fed68164db0 2024-05-27T09:27:30.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.445+0000 7fed6d000640 1 --2- 172.21.15.114:0/839899574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed68164db0 0x7fed68158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.446+0000 7f1621200640 1 --2- 172.21.15.114:0/3272672812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f161c0c4720 0x7f161c15bc90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f161c0b30d0 tx=0x7f160802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84e092ccd40b6874 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.446+0000 7fed6d000640 1 -- 172.21.15.114:0/839899574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed68160b40 msgr2=0x7fed68163560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.446+0000 7f0a12800640 1 -- 172.21.15.114:0/811045973 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a0c06a380 con 0x7f0a0c073310 2024-05-27T09:27:30.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.446+0000 7fed67400640 1 --2- 172.21.15.114:0/839899574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed68160b40 0x7fed68163560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.447+0000 7fed6d000640 1 --2- 172.21.15.114:0/839899574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed68160b40 0x7fed68163560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.447+0000 7fed6d000640 1 -- 172.21.15.114:0/839899574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed68074860 msgr2=0x7fed68154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.447+0000 7f161be00640 1 -- 172.21.15.114:0/3272672812 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608037070 con 0x7f161c0c4720 2024-05-27T09:27:30.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.447+0000 7f161be00640 1 -- 172.21.15.114:0/3272672812 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1608032440 con 0x7f161c0c4720 2024-05-27T09:27:30.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.447+0000 7f161be00640 1 -- 172.21.15.114:0/3272672812 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608032810 con 0x7f161c0c4720 2024-05-27T09:27:30.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.447+0000 7fed6d000640 1 --2- 172.21.15.114:0/839899574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed68074860 0x7fed68154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.447+0000 7fed6d000640 1 -- 172.21.15.114:0/839899574 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed681538f0 con 0x7fed68164db0 2024-05-27T09:27:30.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.447+0000 7fed67400640 1 --2- 172.21.15.114:0/839899574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed68160b40 0x7fed68163560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:27:30.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.447+0000 7fed67e00640 1 --2- 172.21.15.114:0/839899574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed68074860 0x7fed68154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:27:30.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.447+0000 7fed6d000640 1 --2- 172.21.15.114:0/839899574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed68164db0 0x7fed68158f10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fed5400b440 tx=0x7fed5400b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.447+0000 7fed5fe00640 1 -- 172.21.15.114:0/839899574 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed54015020 con 0x7fed68164db0 2024-05-27T09:27:30.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.448+0000 7fed5fe00640 1 -- 172.21.15.114:0/839899574 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed54011850 con 0x7fed68164db0 2024-05-27T09:27:30.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.448+0000 7fed5fe00640 1 -- 172.21.15.114:0/839899574 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed54011c40 con 0x7fed68164db0 2024-05-27T09:27:30.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.448+0000 7fa94a000640 1 -- 172.21.15.114:0/2965358272 shutdown_connections 2024-05-27T09:27:30.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.448+0000 7fa94a000640 1 --2- 172.21.15.114:0/2965358272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944160b40 0x7fa944160f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.448+0000 7fa94a000640 1 --2- 172.21.15.114:0/2965358272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa944074860 0x7fa94415aac0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.448+0000 7fa94a000640 1 --2- 172.21.15.114:0/2965358272 >> v2:172.21.15.114:3300/0 conn(0x7fa944164db0 0x7fa9440c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.448+0000 7fa94a000640 1 -- 172.21.15.114:0/2965358272 >> 172.21.15.114:0/2965358272 conn(0x7fa94406f6e0 msgr2=0x7fa944070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:30.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.448+0000 7f1624600640 1 -- 172.21.15.114:0/3272672812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f161c0c4720 msgr2=0x7f161c15bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.448+0000 7f1624600640 1 --2- 172.21.15.114:0/3272672812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f161c0c4720 0x7f161c15bc90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f161c0b30d0 tx=0x7f160802a270 comp rx=0 tx=0).stop 2024-05-27T09:27:30.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.448+0000 7fed6f000640 1 -- 172.21.15.114:0/839899574 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed6815f8e0 con 0x7fed68164db0 2024-05-27T09:27:30.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.448+0000 7fed6f000640 1 -- 172.21.15.114:0/839899574 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fed68157e20 con 0x7fed68164db0 2024-05-27T09:27:30.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.448+0000 7fc2b8c00640 1 -- 172.21.15.114:0/410353100 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc2b001a070 con 0x7fc2bc0ce480 2024-05-27T09:27:30.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.448+0000 7fc2c3400640 1 -- 172.21.15.114:0/410353100 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc298005290 con 0x7fc2bc0ce480 2024-05-27T09:27:30.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.449+0000 7fa94a000640 1 -- 172.21.15.114:0/2965358272 shutdown_connections 2024-05-27T09:27:30.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.449+0000 7fa94a000640 1 -- 172.21.15.114:0/2965358272 wait complete. 2024-05-27T09:27:30.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.449+0000 7fc2b8c00640 1 --2- 172.21.15.114:0/410353100 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc29c076350 0x7fc29c078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.450+0000 7f1624600640 1 -- 172.21.15.114:0/3272672812 shutdown_connections 2024-05-27T09:27:30.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.450+0000 7f1624600640 1 --2- 172.21.15.114:0/3272672812 >> v2:172.21.15.163:3300/0 conn(0x7f161c15c1d0 0x7f161c0ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.450+0000 7f1624600640 1 --2- 172.21.15.114:0/3272672812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f161c0c4720 0x7f161c15bc90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.450+0000 7f1624600640 1 --2- 172.21.15.114:0/3272672812 >> v2:172.21.15.114:3300/0 conn(0x7f161c073780 0x7f161c152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.450+0000 7f1624600640 1 -- 172.21.15.114:0/3272672812 >> 172.21.15.114:0/3272672812 conn(0x7f161c06f6e0 msgr2=0x7f161c070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:30.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.449+0000 7fc2b8c00640 1 -- 172.21.15.114:0/410353100 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc2b00967c0 con 0x7fc2bc0ce480 2024-05-27T09:27:30.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.449+0000 7f0a03400640 1 -- 172.21.15.114:0/811045973 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0a04042020 con 0x7f0a0c073310 2024-05-27T09:27:30.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.450+0000 7fed6f000640 1 -- 172.21.15.114:0/839899574 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed6806a380 con 0x7fed68164db0 2024-05-27T09:27:30.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.451+0000 7f0a03400640 1 --2- 172.21.15.114:0/811045973 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f09ec0764d0 0x7f09ec0789e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.451+0000 7f0a03400640 1 -- 172.21.15.114:0/811045973 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0a040b8bd0 con 0x7f0a0c073310 2024-05-27T09:27:30.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.450+0000 7fc2bbe00640 1 --2- 172.21.15.114:0/410353100 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc29c076350 0x7fc29c078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.451+0000 7f1624600640 1 -- 172.21.15.114:0/3272672812 shutdown_connections 2024-05-27T09:27:30.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.451+0000 7f0a0aa00640 1 --2- 172.21.15.114:0/811045973 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f09ec0764d0 0x7f09ec0789e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.451+0000 7f1624600640 1 -- 172.21.15.114:0/3272672812 wait complete. 2024-05-27T09:27:30.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.452+0000 7fa94a000640 1 Processor -- start 2024-05-27T09:27:30.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.452+0000 7fa94a000640 1 -- start start 2024-05-27T09:27:30.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.452+0000 7f0a0aa00640 1 --2- 172.21.15.114:0/811045973 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f09ec0764d0 0x7f09ec0789e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c15d4c0 tx=0x7f09f8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.452+0000 7fed5fe00640 1 -- 172.21.15.114:0/839899574 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fed54018020 con 0x7fed68164db0 2024-05-27T09:27:30.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.453+0000 7fc2bbe00640 1 --2- 172.21.15.114:0/410353100 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc29c076350 0x7fc29c078860 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8034000 tx=0x7fc2a802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.453+0000 7fed5fe00640 1 --2- 172.21.15.114:0/839899574 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fed48076320 0x7fed48078830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.454+0000 7fed5fe00640 1 -- 172.21.15.114:0/839899574 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fed54097320 con 0x7fed68164db0 2024-05-27T09:27:30.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.454+0000 7fed67e00640 1 --2- 172.21.15.114:0/839899574 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fed48076320 0x7fed48078830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.454+0000 7fed67e00640 1 --2- 172.21.15.114:0/839899574 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fed48076320 0x7fed48078830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fed60009730 tx=0x7fed60009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.455+0000 7fc2b8c00640 1 -- 172.21.15.114:0/410353100 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc2b0060190 con 0x7fc2bc0ce480 2024-05-27T09:27:30.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.456+0000 7f1624600640 1 Processor -- start 2024-05-27T09:27:30.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.456+0000 7f1624600640 1 -- start start 2024-05-27T09:27:30.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.456+0000 7fa94a000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa944074860 0x7fa9441636e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.456+0000 7fa94a000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944160b40 0x7fa9440cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.456+0000 7fa94a000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa944164db0 0x7fa9440cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.456+0000 7fa94a000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa9441645b0 con 0x7fa944074860 2024-05-27T09:27:30.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.456+0000 7fa94a000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa9441642d0 con 0x7fa944164db0 2024-05-27T09:27:30.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.456+0000 7fa94a000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa944164890 con 0x7fa944160b40 2024-05-27T09:27:30.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.456+0000 7fa942a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944160b40 0x7fa9440cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.457+0000 7fa943400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa944074860 0x7fa9441636e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.457+0000 7fa942a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944160b40 0x7fa9440cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:60718/0 (socket says 172.21.15.114:60718) 2024-05-27T09:27:30.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.457+0000 7fa942a00640 1 -- 172.21.15.114:0/438488568 learned_addr learned my addr 172.21.15.114:0/438488568 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:30.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.457+0000 7fa943e00640 1 --2- 172.21.15.114:0/438488568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa944164db0 0x7fa9440cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.457+0000 7fa942a00640 1 -- 172.21.15.114:0/438488568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa944164db0 msgr2=0x7fa9440cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.457+0000 7fa942a00640 1 --2- 172.21.15.114:0/438488568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa944164db0 0x7fa9440cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.457+0000 7fa942a00640 1 -- 172.21.15.114:0/438488568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa944074860 msgr2=0x7fa9441636e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.457+0000 7fa942a00640 1 --2- 172.21.15.114:0/438488568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa944074860 0x7fa9441636e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.457+0000 7fa942a00640 1 -- 172.21.15.114:0/438488568 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9441fa9a0 con 0x7fa944160b40 2024-05-27T09:27:30.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.457+0000 7fa943400640 1 --2- 172.21.15.114:0/438488568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa944074860 0x7fa9441636e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:30.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.459+0000 7fed5fe00640 1 -- 172.21.15.114:0/839899574 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fed5401f020 con 0x7fed68164db0 2024-05-27T09:27:30.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.459+0000 7f0a03400640 1 -- 172.21.15.114:0/811045973 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0a04082520 con 0x7f0a0c073310 2024-05-27T09:27:30.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.459+0000 7fa942a00640 1 --2- 172.21.15.114:0/438488568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944160b40 0x7fa9440cd670 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa930000c00 tx=0x7fa9300046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.460+0000 7fa93b400640 1 -- 172.21.15.114:0/438488568 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930041020 con 0x7fa944160b40 2024-05-27T09:27:30.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.460+0000 7fa93b400640 1 -- 172.21.15.114:0/438488568 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa930030780 con 0x7fa944160b40 2024-05-27T09:27:30.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.460+0000 7fa93b400640 1 -- 172.21.15.114:0/438488568 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930041020 con 0x7fa944160b40 2024-05-27T09:27:30.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.460+0000 7f1624600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f161c073780 0x7f161c1f64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.460+0000 7f1624600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f161c0c4720 0x7f161c14c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.460+0000 7f1624600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f161c15c1d0 0x7f161c15aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.460+0000 7f1624600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f161c159200 con 0x7f161c0c4720 2024-05-27T09:27:30.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.460+0000 7f1624600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f161c158f20 con 0x7f161c073780 2024-05-27T09:27:30.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.460+0000 7f1624600640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f161c0c4b30 con 0x7f161c15c1d0 2024-05-27T09:27:30.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.460+0000 7f1622600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f161c15c1d0 0x7f161c15aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.460+0000 7f1622600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f161c15c1d0 0x7f161c15aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:60720/0 (socket says 172.21.15.114:60720) 2024-05-27T09:27:30.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.460+0000 7f1622600640 1 -- 172.21.15.114:0/35548313 learned_addr learned my addr 172.21.15.114:0/35548313 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:30.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.462+0000 7f1621c00640 1 --2- 172.21.15.114:0/35548313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f161c073780 0x7f161c1f64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.463+0000 7f1621200640 1 --2- 172.21.15.114:0/35548313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f161c0c4720 0x7f161c14c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.462+0000 7fa94a000640 1 -- 172.21.15.114:0/438488568 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9441f7770 con 0x7fa944160b40 2024-05-27T09:27:30.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.463+0000 7f1621c00640 1 -- 172.21.15.114:0/35548313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f161c15c1d0 msgr2=0x7f161c15aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.463+0000 7f1621c00640 1 --2- 172.21.15.114:0/35548313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f161c15c1d0 0x7f161c15aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.463+0000 7f1621c00640 1 -- 172.21.15.114:0/35548313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f161c0c4720 msgr2=0x7f161c14c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.463+0000 7f1621c00640 1 --2- 172.21.15.114:0/35548313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f161c0c4720 0x7f161c14c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.463+0000 7fa94a000640 1 -- 172.21.15.114:0/438488568 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9441f69b0 con 0x7fa944160b40 2024-05-27T09:27:30.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.463+0000 7f1621c00640 1 -- 172.21.15.114:0/35548313 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161c1f6a30 con 0x7f161c073780 2024-05-27T09:27:30.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.464+0000 7f1621200640 1 --2- 172.21.15.114:0/35548313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f161c0c4720 0x7f161c14c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:27:30.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.465+0000 7fa94a000640 1 -- 172.21.15.114:0/438488568 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa94406a380 con 0x7fa944160b40 2024-05-27T09:27:30.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.467+0000 7f1621c00640 1 --2- 172.21.15.114:0/35548313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f161c073780 0x7f161c1f64f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f161000d5c0 tx=0x7f161000da80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.467+0000 7f1619600640 1 -- 172.21.15.114:0/35548313 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610004030 con 0x7f161c073780 2024-05-27T09:27:30.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.467+0000 7f1619600640 1 -- 172.21.15.114:0/35548313 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16100041d0 con 0x7f161c073780 2024-05-27T09:27:30.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.467+0000 7f1619600640 1 -- 172.21.15.114:0/35548313 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610014620 con 0x7f161c073780 2024-05-27T09:27:30.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.468+0000 7f1624600640 1 -- 172.21.15.114:0/35548313 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f161c076780 con 0x7f161c073780 2024-05-27T09:27:30.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.468+0000 7f1624600640 1 -- 172.21.15.114:0/35548313 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f161c0b0ed0 con 0x7f161c073780 2024-05-27T09:27:30.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.467+0000 7fa93b400640 1 -- 172.21.15.114:0/438488568 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa930044020 con 0x7fa944160b40 2024-05-27T09:27:30.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.471+0000 7f1624600640 1 -- 172.21.15.114:0/35548313 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f161c0c3cf0 con 0x7f161c073780 2024-05-27T09:27:30.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.471+0000 7fa93b400640 1 --2- 172.21.15.114:0/438488568 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa9240764d0 0x7fa9240789e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.472+0000 7fa93b400640 1 -- 172.21.15.114:0/438488568 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa9300b79a0 con 0x7fa944160b40 2024-05-27T09:27:30.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.472+0000 7fa943400640 1 --2- 172.21.15.114:0/438488568 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa9240764d0 0x7fa9240789e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.473+0000 7fa93b400640 1 -- 172.21.15.114:0/438488568 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa93004b030 con 0x7fa944160b40 2024-05-27T09:27:30.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.473+0000 7f1619600640 1 -- 172.21.15.114:0/35548313 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f16100147c0 con 0x7f161c073780 2024-05-27T09:27:30.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.473+0000 7fa943400640 1 --2- 172.21.15.114:0/438488568 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa9240764d0 0x7fa9240789e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa934006fd0 tx=0x7fa934008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.474+0000 7f1619600640 1 --2- 172.21.15.114:0/35548313 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f15fc076350 0x7f15fc078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.475+0000 7f1619600640 1 -- 172.21.15.114:0/35548313 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1610097c40 con 0x7f161c073780 2024-05-27T09:27:30.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.478+0000 7f1621200640 1 --2- 172.21.15.114:0/35548313 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f15fc076350 0x7f15fc078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.479+0000 7f1621200640 1 --2- 172.21.15.114:0/35548313 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f15fc076350 0x7f15fc078860 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1608034000 tx=0x7f160802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.479+0000 7f1619600640 1 -- 172.21.15.114:0/35548313 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1610061580 con 0x7f161c073780 2024-05-27T09:27:30.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.482+0000 7ff959600640 1 Processor -- start 2024-05-27T09:27:30.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.482+0000 7ff959600640 1 -- start start 2024-05-27T09:27:30.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.485+0000 7ff959600640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff9540cabd0 0x7ff95415c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.485+0000 7ff959600640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff95406ea70 0x7ff9540cb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.485+0000 7ff959600640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9541570a0 con 0x7ff9540ca530 2024-05-27T09:27:30.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.485+0000 7ff959600640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7ff95406f710 con 0x7ff95406ea70 2024-05-27T09:27:30.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.485+0000 7ff959600640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7ff9540cc5d0 con 0x7ff9540cabd0 2024-05-27T09:27:30.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.486+0000 7ff953400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff95406ea70 0x7ff9540cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.486+0000 7ff953400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff95406ea70 0x7ff9540cb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35810/0 (socket says 172.21.15.114:35810) 2024-05-27T09:27:30.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.486+0000 7ff953400640 1 -- 172.21.15.114:0/3890445198 learned_addr learned my addr 172.21.15.114:0/3890445198 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:30.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.486+0000 7ff953400640 1 -- 172.21.15.114:0/3890445198 >> v2:172.21.15.163:3300/0 conn(0x7ff9540cabd0 msgr2=0x7ff95415c960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T09:27:30.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.486+0000 7ff953400640 1 --2- 172.21.15.114:0/3890445198 >> v2:172.21.15.163:3300/0 conn(0x7ff9540cabd0 0x7ff95415c960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.486+0000 7ff953400640 1 -- 172.21.15.114:0/3890445198 >> v1:172.21.15.163:6789/0 conn(0x7ff9540ca530 legacy=0x7ff9541608f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T09:27:30.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.486+0000 7ff953400640 1 -- 172.21.15.114:0/3890445198 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9540cc8b0 con 0x7ff95406ea70 2024-05-27T09:27:30.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.488+0000 7ff953400640 1 --2- 172.21.15.114:0/3890445198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff95406ea70 0x7ff9540cb870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff9540695c0 tx=0x7ff94002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59740c253dc415b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.488+0000 7ff951600640 1 -- 172.21.15.114:0/3890445198 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940037070 con 0x7ff95406ea70 2024-05-27T09:27:30.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.489+0000 7ff951600640 1 -- 172.21.15.114:0/3890445198 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff940032440 con 0x7ff95406ea70 2024-05-27T09:27:30.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.489+0000 7ff951600640 1 -- 172.21.15.114:0/3890445198 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940032810 con 0x7ff95406ea70 2024-05-27T09:27:30.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.490+0000 7ff959600640 1 -- 172.21.15.114:0/3890445198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff95406ea70 msgr2=0x7ff9540cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.490+0000 7ff959600640 1 --2- 172.21.15.114:0/3890445198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff95406ea70 0x7ff9540cb870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff9540695c0 tx=0x7ff94002a200 comp rx=0 tx=0).stop 2024-05-27T09:27:30.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.491+0000 7ff959600640 1 -- 172.21.15.114:0/3890445198 shutdown_connections 2024-05-27T09:27:30.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.491+0000 7ff959600640 1 --2- 172.21.15.114:0/3890445198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff95406ea70 0x7ff9540cb870 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.491+0000 7ff959600640 1 --2- 172.21.15.114:0/3890445198 >> v2:172.21.15.163:3300/0 conn(0x7ff9540cabd0 0x7ff95415c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.491+0000 7ff959600640 1 -- 172.21.15.114:0/3890445198 >> 172.21.15.114:0/3890445198 conn(0x7ff9540715f0 msgr2=0x7ff954068600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:30.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.491+0000 7ff959600640 1 -- 172.21.15.114:0/3890445198 shutdown_connections 2024-05-27T09:27:30.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.491+0000 7ff959600640 1 -- 172.21.15.114:0/3890445198 wait complete. 2024-05-27T09:27:30.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.495+0000 7ff959600640 1 Processor -- start 2024-05-27T09:27:30.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.495+0000 7ff959600640 1 -- start start 2024-05-27T09:27:30.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.497+0000 7ff959600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff95406ea70 0x7ff9541f2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.497+0000 7ff959600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9540ca530 0x7ff9540b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.497+0000 7ff959600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9540cabd0 0x7ff9541e82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.497+0000 7ff959600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541603d0 con 0x7ff9540cabd0 2024-05-27T09:27:30.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.497+0000 7ff959600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff9541600f0 con 0x7ff95406ea70 2024-05-27T09:27:30.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.497+0000 7ff959600640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7ff9540cb3b0 con 0x7ff9540ca530 2024-05-27T09:27:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.498+0000 7ff952000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9540ca530 0x7ff9540b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.498+0000 7ff952000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9540ca530 0x7ff9540b4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:60728/0 (socket says 172.21.15.114:60728) 2024-05-27T09:27:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.498+0000 7ff952000640 1 -- 172.21.15.114:0/11277081 learned_addr learned my addr 172.21.15.114:0/11277081 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.498+0000 7ff952a00640 1 --2- 172.21.15.114:0/11277081 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff95406ea70 0x7ff9541f2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.498+0000 7ff953400640 1 --2- 172.21.15.114:0/11277081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9540cabd0 0x7ff9541e82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.498+0000 7ff952a00640 1 -- 172.21.15.114:0/11277081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9540ca530 msgr2=0x7ff9540b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.498+0000 7ff952a00640 1 --2- 172.21.15.114:0/11277081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9540ca530 0x7ff9540b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.498+0000 7ff952a00640 1 -- 172.21.15.114:0/11277081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9540cabd0 msgr2=0x7ff9541e82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.498+0000 7ff952a00640 1 --2- 172.21.15.114:0/11277081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9540cabd0 0x7ff9541e82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.498+0000 7ff952a00640 1 -- 172.21.15.114:0/11277081 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95415fcc0 con 0x7ff95406ea70 2024-05-27T09:27:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.498+0000 7ff952000640 1 --2- 172.21.15.114:0/11277081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9540ca530 0x7ff9540b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.499+0000 7ff952a00640 1 --2- 172.21.15.114:0/11277081 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff95406ea70 0x7ff9541f2ae0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00be20 tx=0x7ff94c009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.499+0000 7ff94aa00640 1 -- 172.21.15.114:0/11277081 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c019070 con 0x7ff95406ea70 2024-05-27T09:27:30.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.499+0000 7ff94aa00640 1 -- 172.21.15.114:0/11277081 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff94c014440 con 0x7ff95406ea70 2024-05-27T09:27:30.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.499+0000 7ff94aa00640 1 -- 172.21.15.114:0/11277081 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c014830 con 0x7ff95406ea70 2024-05-27T09:27:30.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.499+0000 7ff959600640 1 -- 172.21.15.114:0/11277081 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9540b4a90 con 0x7ff95406ea70 2024-05-27T09:27:30.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.499+0000 7ff959600640 1 -- 172.21.15.114:0/11277081 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9541f62d0 con 0x7ff95406ea70 2024-05-27T09:27:30.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.502+0000 7ff959600640 1 -- 172.21.15.114:0/11277081 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff95406a380 con 0x7ff95406ea70 2024-05-27T09:27:30.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.503+0000 7ff94aa00640 1 -- 172.21.15.114:0/11277081 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff94c01c360 con 0x7ff95406ea70 2024-05-27T09:27:30.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.504+0000 7ff94aa00640 1 --2- 172.21.15.114:0/11277081 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff934076300 0x7ff934078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:30.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.505+0000 7ff94aa00640 1 -- 172.21.15.114:0/11277081 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff94c0974f0 con 0x7ff95406ea70 2024-05-27T09:27:30.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.505+0000 7ff952000640 1 --2- 172.21.15.114:0/11277081 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff934076300 0x7ff934078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:30.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.506+0000 7ff952000640 1 --2- 172.21.15.114:0/11277081 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff934076300 0x7ff934078810 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff9540c5b40 tx=0x7ff9440090e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:30.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.510+0000 7ff94aa00640 1 -- 172.21.15.114:0/11277081 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff94c0606d0 con 0x7ff95406ea70 2024-05-27T09:27:30.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.836+0000 7f54ec800640 1 -- 172.21.15.114:0/2415490503 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f54e4150290 con 0x7f54e4152890 2024-05-27T09:27:30.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.837+0000 7f54de000640 1 -- 172.21.15.114:0/2415490503 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f54e00828d0 con 0x7f54e4152890 2024-05-27T09:27:30.839 INFO:teuthology.orchestra.run.smithi114.stdout:64424509447 2024-05-27T09:27:30.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.841+0000 7f54ec800640 1 -- 172.21.15.114:0/2415490503 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f54c80762d0 msgr2=0x7f54c80787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.841+0000 7f54ec800640 1 --2- 172.21.15.114:0/2415490503 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f54c80762d0 0x7f54c80787e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f54d8007fd0 tx=0x7f54d8009040 comp rx=0 tx=0).stop 2024-05-27T09:27:30.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.841+0000 7f54ec800640 1 -- 172.21.15.114:0/2415490503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54e4152890 msgr2=0x7f54e41f1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.841+0000 7f54ec800640 1 --2- 172.21.15.114:0/2415490503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54e4152890 0x7f54e41f1610 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009f40 tx=0x7f54e0004a40 comp rx=0 tx=0).stop 2024-05-27T09:27:30.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.842+0000 7f54ec800640 1 -- 172.21.15.114:0/2415490503 shutdown_connections 2024-05-27T09:27:30.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.842+0000 7f54ec800640 1 --2- 172.21.15.114:0/2415490503 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f54c80762d0 0x7f54c80787e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.842+0000 7f54ec800640 1 --2- 172.21.15.114:0/2415490503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54e4152890 0x7f54e41f1610 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.842+0000 7f54ec800640 1 --2- 172.21.15.114:0/2415490503 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54e40c9330 0x7f54e414f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.842+0000 7f54ec800640 1 --2- 172.21.15.114:0/2415490503 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f54e40b31c0 0x7f54e414eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.842+0000 7f54ec800640 1 -- 172.21.15.114:0/2415490503 >> 172.21.15.114:0/2415490503 conn(0x7f54e40cd890 msgr2=0x7f54e4150b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:30.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.844+0000 7f54ec800640 1 -- 172.21.15.114:0/2415490503 shutdown_connections 2024-05-27T09:27:30.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.844+0000 7f54ec800640 1 -- 172.21.15.114:0/2415490503 wait complete. 2024-05-27T09:27:30.885 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.0 2024-05-27T09:27:30.885 DEBUG:teuthology.parallel:result is None 2024-05-27T09:27:30.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.890+0000 7fc2c3400640 1 -- 172.21.15.114:0/410353100 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fc298005580 con 0x7fc2bc0ce480 2024-05-27T09:27:30.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.891+0000 7fc2b8c00640 1 -- 172.21.15.114:0/410353100 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc2b005fa90 con 0x7fc2bc0ce480 2024-05-27T09:27:30.892 INFO:teuthology.orchestra.run.smithi114.stdout:60129542150 2024-05-27T09:27:30.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.894+0000 7fed6f000640 1 -- 172.21.15.114:0/839899574 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fed68073e30 con 0x7fed68164db0 2024-05-27T09:27:30.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.894+0000 7fc2c3400640 1 -- 172.21.15.114:0/410353100 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc29c076350 msgr2=0x7fc29c078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.894+0000 7fc2c3400640 1 --2- 172.21.15.114:0/410353100 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc29c076350 0x7fc29c078860 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8034000 tx=0x7fc2a802e000 comp rx=0 tx=0).stop 2024-05-27T09:27:30.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.894+0000 7fc2c3400640 1 -- 172.21.15.114:0/410353100 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc0ce480 msgr2=0x7fc2bc0ca830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.894+0000 7fc2c3400640 1 --2- 172.21.15.114:0/410353100 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc0ce480 0x7fc2bc0ca830 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000c3f0 tx=0x7fc2b000c8b0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.894+0000 7fc2c3400640 1 -- 172.21.15.114:0/410353100 shutdown_connections 2024-05-27T09:27:30.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.894+0000 7fc2c3400640 1 --2- 172.21.15.114:0/410353100 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc29c076350 0x7fc29c078860 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.894+0000 7fc2c3400640 1 --2- 172.21.15.114:0/410353100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc2bc15c7c0 0x7fc2bc14cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.894+0000 7fc2c3400640 1 --2- 172.21.15.114:0/410353100 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2bc156a70 0x7fc2bc0cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.894+0000 7fc2c3400640 1 --2- 172.21.15.114:0/410353100 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2bc0ce480 0x7fc2bc0ca830 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.894+0000 7fc2c3400640 1 -- 172.21.15.114:0/410353100 >> 172.21.15.114:0/410353100 conn(0x7fc2bc06f2a0 msgr2=0x7fc2bc0cd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:30.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.896+0000 7fc2c3400640 1 -- 172.21.15.114:0/410353100 shutdown_connections 2024-05-27T09:27:30.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.896+0000 7fc2c3400640 1 -- 172.21.15.114:0/410353100 wait complete. 2024-05-27T09:27:30.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.896+0000 7fed5fe00640 1 -- 172.21.15.114:0/839899574 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fed54060d60 con 0x7fed68164db0 2024-05-27T09:27:30.898 INFO:teuthology.orchestra.run.smithi114.stdout:60129542150 2024-05-27T09:27:30.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.897+0000 7f46e7000640 1 -- 172.21.15.114:0/4230681227 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f46e015a5a0 con 0x7f46e00c8c40 2024-05-27T09:27:30.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.898+0000 7f46d7e00640 1 -- 172.21.15.114:0/4230681227 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f46c8060e40 con 0x7f46e00c8c40 2024-05-27T09:27:30.899 INFO:teuthology.orchestra.run.smithi114.stdout:64424509447 2024-05-27T09:27:30.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.899+0000 7fed6f000640 1 -- 172.21.15.114:0/839899574 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fed48076320 msgr2=0x7fed48078830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.899+0000 7fed6f000640 1 --2- 172.21.15.114:0/839899574 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fed48076320 0x7fed48078830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fed60009730 tx=0x7fed60009000 comp rx=0 tx=0).stop 2024-05-27T09:27:30.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.899+0000 7fed6f000640 1 -- 172.21.15.114:0/839899574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed68164db0 msgr2=0x7fed68158f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.899+0000 7fed6f000640 1 --2- 172.21.15.114:0/839899574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed68164db0 0x7fed68158f10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fed5400b440 tx=0x7fed5400b900 comp rx=0 tx=0).stop 2024-05-27T09:27:30.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.899+0000 7fed6f000640 1 -- 172.21.15.114:0/839899574 shutdown_connections 2024-05-27T09:27:30.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.899+0000 7fed6f000640 1 --2- 172.21.15.114:0/839899574 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fed48076320 0x7fed48078830 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.899+0000 7fed6f000640 1 --2- 172.21.15.114:0/839899574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed68164db0 0x7fed68158f10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.899+0000 7fed6f000640 1 --2- 172.21.15.114:0/839899574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed68160b40 0x7fed68163560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.899+0000 7fed6f000640 1 --2- 172.21.15.114:0/839899574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fed68074860 0x7fed68154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.899+0000 7fed6f000640 1 -- 172.21.15.114:0/839899574 >> 172.21.15.114:0/839899574 conn(0x7fed6806f6e0 msgr2=0x7fed680c8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:30.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.900+0000 7fed6f000640 1 -- 172.21.15.114:0/839899574 shutdown_connections 2024-05-27T09:27:30.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.900+0000 7fed6f000640 1 -- 172.21.15.114:0/839899574 wait complete. 2024-05-27T09:27:30.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.901+0000 7f46e7000640 1 -- 172.21.15.114:0/4230681227 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f46bc0762d0 msgr2=0x7f46bc0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.901+0000 7f46e7000640 1 --2- 172.21.15.114:0/4230681227 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f46bc0762d0 0x7f46bc0787e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f46d8000f80 tx=0x7f46d8009000 comp rx=0 tx=0).stop 2024-05-27T09:27:30.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.901+0000 7f46e7000640 1 -- 172.21.15.114:0/4230681227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e00c8c40 msgr2=0x7f46e014e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.901+0000 7f46e7000640 1 --2- 172.21.15.114:0/4230681227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e00c8c40 0x7f46e014e9e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f46c800b360 tx=0x7f46c800b820 comp rx=0 tx=0).stop 2024-05-27T09:27:30.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.901+0000 7f46e7000640 1 -- 172.21.15.114:0/4230681227 shutdown_connections 2024-05-27T09:27:30.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.901+0000 7f46e7000640 1 --2- 172.21.15.114:0/4230681227 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f46bc0762d0 0x7f46bc0787e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.901+0000 7f46e7000640 1 --2- 172.21.15.114:0/4230681227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e00c8c40 0x7f46e014e9e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.901+0000 7f46e7000640 1 --2- 172.21.15.114:0/4230681227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46e006db40 0x7f46e014e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.901+0000 7f46e7000640 1 --2- 172.21.15.114:0/4230681227 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46e006c720 0x7f46e0155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.901+0000 7f46e7000640 1 -- 172.21.15.114:0/4230681227 >> 172.21.15.114:0/4230681227 conn(0x7f46e00ce240 msgr2=0x7f46e00ca5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:30.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.901+0000 7f46e7000640 1 -- 172.21.15.114:0/4230681227 shutdown_connections 2024-05-27T09:27:30.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.901+0000 7f46e7000640 1 -- 172.21.15.114:0/4230681227 wait complete. 2024-05-27T09:27:30.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.911+0000 7f0a12800640 1 -- 172.21.15.114:0/811045973 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f0a0c15ad90 con 0x7f0a0c073310 2024-05-27T09:27:30.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.915+0000 7f0a03400640 1 -- 172.21.15.114:0/811045973 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0a04081e20 con 0x7f0a0c073310 2024-05-27T09:27:30.920 INFO:teuthology.orchestra.run.smithi114.stdout:64424509447 2024-05-27T09:27:30.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.919+0000 7f0a12800640 1 -- 172.21.15.114:0/811045973 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f09ec0764d0 msgr2=0x7f09ec0789e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.919+0000 7f0a12800640 1 --2- 172.21.15.114:0/811045973 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f09ec0764d0 0x7f09ec0789e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c15d4c0 tx=0x7f09f8008040 comp rx=0 tx=0).stop 2024-05-27T09:27:30.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.919+0000 7f0a12800640 1 -- 172.21.15.114:0/811045973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a0c073310 msgr2=0x7f0a0c150c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.919+0000 7f0a12800640 1 --2- 172.21.15.114:0/811045973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a0c073310 0x7f0a0c150c60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0a04033850 tx=0x7f0a04004a40 comp rx=0 tx=0).stop 2024-05-27T09:27:30.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.919+0000 7f0a12800640 1 -- 172.21.15.114:0/811045973 shutdown_connections 2024-05-27T09:27:30.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.919+0000 7f0a12800640 1 --2- 172.21.15.114:0/811045973 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f09ec0764d0 0x7f09ec0789e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.919+0000 7f0a12800640 1 --2- 172.21.15.114:0/811045973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a0c0ca140 0x7f0a0c14f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.919+0000 7f0a12800640 1 --2- 172.21.15.114:0/811045973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a0c0b2c60 0x7f0a0c153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.919+0000 7f0a12800640 1 --2- 172.21.15.114:0/811045973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a0c073310 0x7f0a0c150c60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.919+0000 7f0a12800640 1 -- 172.21.15.114:0/811045973 >> 172.21.15.114:0/811045973 conn(0x7f0a0c06f6e0 msgr2=0x7f0a0c15f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:30.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.920+0000 7f0a12800640 1 -- 172.21.15.114:0/811045973 shutdown_connections 2024-05-27T09:27:30.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.920+0000 7f0a12800640 1 -- 172.21.15.114:0/811045973 wait complete. 2024-05-27T09:27:30.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.927+0000 7fa94a000640 1 -- 172.21.15.114:0/438488568 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa944073e30 con 0x7fa944160b40 2024-05-27T09:27:30.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.928+0000 7fa93b400640 1 -- 172.21.15.114:0/438488568 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa930080c10 con 0x7fa944160b40 2024-05-27T09:27:30.929 INFO:teuthology.orchestra.run.smithi114.stdout:60129542150 2024-05-27T09:27:30.930 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.7 2024-05-27T09:27:30.930 DEBUG:teuthology.parallel:result is None 2024-05-27T09:27:30.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.931+0000 7fa94a000640 1 -- 172.21.15.114:0/438488568 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa9240764d0 msgr2=0x7fa9240789e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.931+0000 7fa94a000640 1 --2- 172.21.15.114:0/438488568 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa9240764d0 0x7fa9240789e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa934006fd0 tx=0x7fa934008040 comp rx=0 tx=0).stop 2024-05-27T09:27:30.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.931+0000 7fa94a000640 1 -- 172.21.15.114:0/438488568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944160b40 msgr2=0x7fa9440cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.931+0000 7fa94a000640 1 --2- 172.21.15.114:0/438488568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944160b40 0x7fa9440cd670 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa930000c00 tx=0x7fa9300046c0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.931+0000 7fa94a000640 1 -- 172.21.15.114:0/438488568 shutdown_connections 2024-05-27T09:27:30.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.931+0000 7fa94a000640 1 --2- 172.21.15.114:0/438488568 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa9240764d0 0x7fa9240789e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.931+0000 7fa94a000640 1 --2- 172.21.15.114:0/438488568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa944164db0 0x7fa9440cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.931+0000 7fa94a000640 1 --2- 172.21.15.114:0/438488568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944160b40 0x7fa9440cd670 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.931+0000 7fa94a000640 1 --2- 172.21.15.114:0/438488568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa944074860 0x7fa9441636e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.931+0000 7fa94a000640 1 -- 172.21.15.114:0/438488568 >> 172.21.15.114:0/438488568 conn(0x7fa94406f6e0 msgr2=0x7fa94415e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:30.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.931+0000 7fa94a000640 1 -- 172.21.15.114:0/438488568 shutdown_connections 2024-05-27T09:27:30.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.932+0000 7fa94a000640 1 -- 172.21.15.114:0/438488568 wait complete. 2024-05-27T09:27:30.936 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.4 2024-05-27T09:27:30.936 DEBUG:teuthology.parallel:result is None 2024-05-27T09:27:30.939 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.1 2024-05-27T09:27:30.939 DEBUG:teuthology.parallel:result is None 2024-05-27T09:27:30.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.944+0000 7f1624600640 1 -- 172.21.15.114:0/35548313 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f161c06a380 con 0x7f161c073780 2024-05-27T09:27:30.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.945+0000 7f1619600640 1 -- 172.21.15.114:0/35548313 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f16100045c0 con 0x7f161c073780 2024-05-27T09:27:30.946 INFO:teuthology.orchestra.run.smithi114.stdout:64424509447 2024-05-27T09:27:30.949 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.3 2024-05-27T09:27:30.949 DEBUG:teuthology.parallel:result is None 2024-05-27T09:27:30.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.948+0000 7f1624600640 1 -- 172.21.15.114:0/35548313 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f15fc076350 msgr2=0x7f15fc078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.948+0000 7f1624600640 1 --2- 172.21.15.114:0/35548313 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f15fc076350 0x7f15fc078860 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1608034000 tx=0x7f160802e000 comp rx=0 tx=0).stop 2024-05-27T09:27:30.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.948+0000 7f1624600640 1 -- 172.21.15.114:0/35548313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f161c073780 msgr2=0x7f161c1f64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.948+0000 7f1624600640 1 --2- 172.21.15.114:0/35548313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f161c073780 0x7f161c1f64f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f161000d5c0 tx=0x7f161000da80 comp rx=0 tx=0).stop 2024-05-27T09:27:30.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.948+0000 7f1624600640 1 -- 172.21.15.114:0/35548313 shutdown_connections 2024-05-27T09:27:30.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.948+0000 7f1624600640 1 --2- 172.21.15.114:0/35548313 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f15fc076350 0x7f15fc078860 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.948+0000 7f1624600640 1 --2- 172.21.15.114:0/35548313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f161c15c1d0 0x7f161c15aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.948+0000 7f1624600640 1 --2- 172.21.15.114:0/35548313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f161c0c4720 0x7f161c14c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.948+0000 7f1624600640 1 --2- 172.21.15.114:0/35548313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f161c073780 0x7f161c1f64f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.948+0000 7f1624600640 1 -- 172.21.15.114:0/35548313 >> 172.21.15.114:0/35548313 conn(0x7f161c06f6e0 msgr2=0x7f161c0711a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:30.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.948+0000 7f1624600640 1 -- 172.21.15.114:0/35548313 shutdown_connections 2024-05-27T09:27:30.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.948+0000 7f1624600640 1 -- 172.21.15.114:0/35548313 wait complete. 2024-05-27T09:27:30.956 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.5 2024-05-27T09:27:30.956 DEBUG:teuthology.parallel:result is None 2024-05-27T09:27:30.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.962+0000 7ff959600640 1 -- 172.21.15.114:0/11277081 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff9540ca9b0 con 0x7ff95406ea70 2024-05-27T09:27:30.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.963+0000 7ff94aa00640 1 -- 172.21.15.114:0/11277081 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff94c0649d0 con 0x7ff95406ea70 2024-05-27T09:27:30.964 INFO:teuthology.orchestra.run.smithi114.stdout:60129542150 2024-05-27T09:27:30.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.966+0000 7ff959600640 1 -- 172.21.15.114:0/11277081 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff934076300 msgr2=0x7ff934078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.966+0000 7ff959600640 1 --2- 172.21.15.114:0/11277081 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff934076300 0x7ff934078810 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff9540c5b40 tx=0x7ff9440090e0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.966+0000 7ff959600640 1 -- 172.21.15.114:0/11277081 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff95406ea70 msgr2=0x7ff9541f2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:30.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.966+0000 7ff959600640 1 --2- 172.21.15.114:0/11277081 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff95406ea70 0x7ff9541f2ae0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00be20 tx=0x7ff94c009f70 comp rx=0 tx=0).stop 2024-05-27T09:27:30.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.966+0000 7ff959600640 1 -- 172.21.15.114:0/11277081 shutdown_connections 2024-05-27T09:27:30.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.966+0000 7ff959600640 1 --2- 172.21.15.114:0/11277081 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff934076300 0x7ff934078810 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.966+0000 7ff959600640 1 --2- 172.21.15.114:0/11277081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9540cabd0 0x7ff9541e82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.966+0000 7ff959600640 1 --2- 172.21.15.114:0/11277081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9540ca530 0x7ff9540b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.966+0000 7ff959600640 1 --2- 172.21.15.114:0/11277081 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff95406ea70 0x7ff9541f2ae0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:30.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.966+0000 7ff959600640 1 -- 172.21.15.114:0/11277081 >> 172.21.15.114:0/11277081 conn(0x7ff9540715f0 msgr2=0x7ff954068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:30.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.966+0000 7ff959600640 1 -- 172.21.15.114:0/11277081 shutdown_connections 2024-05-27T09:27:30.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:30.966+0000 7ff959600640 1 -- 172.21.15.114:0/11277081 wait complete. 2024-05-27T09:27:30.980 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.2 2024-05-27T09:27:30.980 DEBUG:teuthology.parallel:result is None 2024-05-27T09:27:30.993 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.6 2024-05-27T09:27:30.993 DEBUG:teuthology.parallel:result is None 2024-05-27T09:27:30.993 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-27T09:27:30.993 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T09:27:31.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.293+0000 7fdf7ba00640 1 Processor -- start 2024-05-27T09:27:31.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.293+0000 7fdf7ba00640 1 -- start start 2024-05-27T09:27:31.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.295+0000 7fdf7ba00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdf74154820 0x7fdf740ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:31.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.295+0000 7fdf7ba00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf740ca810 0x7fdf7415dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:31.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.296+0000 7fdf7ba00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf740cac20 con 0x7fdf74074a60 2024-05-27T09:27:31.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.296+0000 7fdf7ba00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fdf74154c30 con 0x7fdf74154820 2024-05-27T09:27:31.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.296+0000 7fdf7ba00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fdf7415db60 con 0x7fdf740ca810 2024-05-27T09:27:31.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.296+0000 7fdf79000640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fdf74074a60 0x7fdf74073260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:46676/0 (socket says 172.21.15.114:46676) 2024-05-27T09:27:31.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.296+0000 7fdf79000640 1 -- 172.21.15.114:0/184703511 learned_addr learned my addr 172.21.15.114:0/184703511 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:31.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.296+0000 7fdf79a00640 1 --2- 172.21.15.114:0/184703511 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf740ca810 0x7fdf7415dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:31.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.296+0000 7fdf73e00640 1 --2- 172.21.15.114:0/184703511 >> v2:172.21.15.114:3300/0 conn(0x7fdf74154820 0x7fdf740ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:31.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.296+0000 7fdf73e00640 1 -- 172.21.15.114:0/184703511 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf740ca810 msgr2=0x7fdf7415dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:31.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.296+0000 7fdf73e00640 1 --2- 172.21.15.114:0/184703511 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf740ca810 0x7fdf7415dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:31.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.296+0000 7fdf73e00640 1 -- 172.21.15.114:0/184703511 >> v1:172.21.15.114:6789/0 conn(0x7fdf74074a60 legacy=0x7fdf74073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:31.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.297+0000 7fdf73e00640 1 -- 172.21.15.114:0/184703511 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7415a810 con 0x7fdf74154820 2024-05-27T09:27:31.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.297+0000 7fdf79a00640 1 --2- 172.21.15.114:0/184703511 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf740ca810 0x7fdf7415dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:31.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.297+0000 7fdf73e00640 1 --2- 172.21.15.114:0/184703511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf74154820 0x7fdf740ca2d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdf600097a0 tx=0x7fdf6002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2381f5a644459bb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:31.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.298+0000 7fdf73400640 1 -- 172.21.15.114:0/184703511 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf60037070 con 0x7fdf74154820 2024-05-27T09:27:31.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.298+0000 7fdf73400640 1 -- 172.21.15.114:0/184703511 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf60032440 con 0x7fdf74154820 2024-05-27T09:27:31.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.298+0000 7fdf73400640 1 -- 172.21.15.114:0/184703511 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf60032810 con 0x7fdf74154820 2024-05-27T09:27:31.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.298+0000 7fdf7ba00640 1 -- 172.21.15.114:0/184703511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf74154820 msgr2=0x7fdf740ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:31.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.298+0000 7fdf7ba00640 1 --2- 172.21.15.114:0/184703511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf74154820 0x7fdf740ca2d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdf600097a0 tx=0x7fdf6002a230 comp rx=0 tx=0).stop 2024-05-27T09:27:31.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.299+0000 7fdf7ba00640 1 -- 172.21.15.114:0/184703511 shutdown_connections 2024-05-27T09:27:31.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.299+0000 7fdf7ba00640 1 --2- 172.21.15.114:0/184703511 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf740ca810 0x7fdf7415dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:31.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.299+0000 7fdf7ba00640 1 --2- 172.21.15.114:0/184703511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf74154820 0x7fdf740ca2d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:31.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.299+0000 7fdf7ba00640 1 -- 172.21.15.114:0/184703511 >> 172.21.15.114:0/184703511 conn(0x7fdf7406f6e0 msgr2=0x7fdf74070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:31.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.299+0000 7fdf7ba00640 1 -- 172.21.15.114:0/184703511 shutdown_connections 2024-05-27T09:27:31.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.299+0000 7fdf7ba00640 1 -- 172.21.15.114:0/184703511 wait complete. 2024-05-27T09:27:31.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.301+0000 7fdf7ba00640 1 Processor -- start 2024-05-27T09:27:31.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.301+0000 7fdf7ba00640 1 -- start start 2024-05-27T09:27:31.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.302+0000 7fdf7ba00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf74074a60 0x7fdf741476e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:31.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.302+0000 7fdf7ba00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf740ca810 0x7fdf74149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:31.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.302+0000 7fdf7ba00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf74154820 0x7fdf7415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:31.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.302+0000 7fdf7ba00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7415b0b0 con 0x7fdf74154820 2024-05-27T09:27:31.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.302+0000 7fdf7ba00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7415add0 con 0x7fdf74074a60 2024-05-27T09:27:31.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.302+0000 7fdf7ba00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fdf7415b390 con 0x7fdf740ca810 2024-05-27T09:27:31.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.303+0000 7fdf79a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf74154820 0x7fdf7415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:31.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.303+0000 7fdf79a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf74154820 0x7fdf7415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35852/0 (socket says 172.21.15.114:35852) 2024-05-27T09:27:31.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.303+0000 7fdf79a00640 1 -- 172.21.15.114:0/347209996 learned_addr learned my addr 172.21.15.114:0/347209996 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:31.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.303+0000 7fdf73e00640 1 --2- 172.21.15.114:0/347209996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf740ca810 0x7fdf74149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:31.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.303+0000 7fdf79000640 1 --2- 172.21.15.114:0/347209996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf74074a60 0x7fdf741476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:31.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.303+0000 7fdf79a00640 1 -- 172.21.15.114:0/347209996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf740ca810 msgr2=0x7fdf74149d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:31.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.303+0000 7fdf79a00640 1 --2- 172.21.15.114:0/347209996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf740ca810 0x7fdf74149d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:31.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.303+0000 7fdf79a00640 1 -- 172.21.15.114:0/347209996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf74074a60 msgr2=0x7fdf741476e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:31.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.303+0000 7fdf79a00640 1 --2- 172.21.15.114:0/347209996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf74074a60 0x7fdf741476e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:31.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.303+0000 7fdf79a00640 1 -- 172.21.15.114:0/347209996 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf74147c20 con 0x7fdf74154820 2024-05-27T09:27:31.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.303+0000 7fdf79a00640 1 --2- 172.21.15.114:0/347209996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf74154820 0x7fdf7415b6a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00efb0 tx=0x7fdf6c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:31.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.304+0000 7fdf70c00640 1 -- 172.21.15.114:0/347209996 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c018020 con 0x7fdf74154820 2024-05-27T09:27:31.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.304+0000 7fdf70c00640 1 -- 172.21.15.114:0/347209996 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf6c014480 con 0x7fdf74154820 2024-05-27T09:27:31.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.304+0000 7fdf70c00640 1 -- 172.21.15.114:0/347209996 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c014850 con 0x7fdf74154820 2024-05-27T09:27:31.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.304+0000 7fdf7ba00640 1 -- 172.21.15.114:0/347209996 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf7414a250 con 0x7fdf74154820 2024-05-27T09:27:31.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.304+0000 7fdf7ba00640 1 -- 172.21.15.114:0/347209996 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf741e51d0 con 0x7fdf74154820 2024-05-27T09:27:31.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.305+0000 7fdf7ba00640 1 -- 172.21.15.114:0/347209996 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf7406a380 con 0x7fdf74154820 2024-05-27T09:27:31.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.306+0000 7fdf70c00640 1 -- 172.21.15.114:0/347209996 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdf6c005d10 con 0x7fdf74154820 2024-05-27T09:27:31.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.308+0000 7fdf70c00640 1 --2- 172.21.15.114:0/347209996 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdf54076320 0x7fdf54078830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:31.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.308+0000 7fdf70c00640 1 -- 172.21.15.114:0/347209996 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdf6c098260 con 0x7fdf74154820 2024-05-27T09:27:31.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.308+0000 7fdf79000640 1 --2- 172.21.15.114:0/347209996 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdf54076320 0x7fdf54078830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:31.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.309+0000 7fdf79000640 1 --2- 172.21.15.114:0/347209996 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdf54076320 0x7fdf54078830 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdf68000f60 tx=0x7fdf68009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:31.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.313+0000 7fdf70c00640 1 -- 172.21.15.114:0/347209996 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdf6c022030 con 0x7fdf74154820 2024-05-27T09:27:31.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.498+0000 7fdf7ba00640 1 -- 172.21.15.114:0/347209996 --> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdf74149a70 con 0x7fdf54076320 2024-05-27T09:27:31.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.506+0000 7fdf70c00640 1 -- 172.21.15.114:0/347209996 <== mgr.4108 v2:172.21.15.163:6832/1122018204 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251898 (secure 0 0 0) 0x7fdf74149a70 con 0x7fdf54076320 2024-05-27T09:27:31.511 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:31.512 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-05-27T09:27:31.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.512+0000 7fdf7ba00640 1 -- 172.21.15.114:0/347209996 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdf54076320 msgr2=0x7fdf54078830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:31.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.512+0000 7fdf7ba00640 1 --2- 172.21.15.114:0/347209996 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdf54076320 0x7fdf54078830 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdf68000f60 tx=0x7fdf68009040 comp rx=0 tx=0).stop 2024-05-27T09:27:31.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.512+0000 7fdf7ba00640 1 -- 172.21.15.114:0/347209996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf74154820 msgr2=0x7fdf7415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:31.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.512+0000 7fdf7ba00640 1 --2- 172.21.15.114:0/347209996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf74154820 0x7fdf7415b6a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00efb0 tx=0x7fdf6c00c490 comp rx=0 tx=0).stop 2024-05-27T09:27:31.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.512+0000 7fdf7ba00640 1 -- 172.21.15.114:0/347209996 shutdown_connections 2024-05-27T09:27:31.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.512+0000 7fdf7ba00640 1 --2- 172.21.15.114:0/347209996 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdf54076320 0x7fdf54078830 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:31.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.512+0000 7fdf7ba00640 1 --2- 172.21.15.114:0/347209996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdf74154820 0x7fdf7415b6a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:31.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.512+0000 7fdf7ba00640 1 --2- 172.21.15.114:0/347209996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf740ca810 0x7fdf74149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:31.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.512+0000 7fdf7ba00640 1 --2- 172.21.15.114:0/347209996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf74074a60 0x7fdf741476e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:31.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.512+0000 7fdf7ba00640 1 -- 172.21.15.114:0/347209996 >> 172.21.15.114:0/347209996 conn(0x7fdf7406f6e0 msgr2=0x7fdf7414ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:31.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.512+0000 7fdf7ba00640 1 -- 172.21.15.114:0/347209996 shutdown_connections 2024-05-27T09:27:31.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.512+0000 7fdf7ba00640 1 -- 172.21.15.114:0/347209996 wait complete. 2024-05-27T09:27:31.539 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-05-27T09:27:30.765502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217960,"kb_used_data":1824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756760,"statfs":{"total":773094113280,"available":772870922240,"internally_reserved":0,"allocated":1867776,"data_stored":1766956,"data_compressed":65078,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.230779"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544950+0000","last_change":"2024-05-27T09:27:18.585655+0000","last_active":"2024-05-27T09:27:19.544950+0000","last_peered":"2024-05-27T09:27:19.544950+0000","last_clean":"2024-05-27T09:27:19.544950+0000","last_became_active":"2024-05-27T09:27:18.585169+0000","last_became_peered":"2024-05-27T09:27:18.585169+0000","last_unstale":"2024-05-27T09:27:19.544950+0000","last_undegraded":"2024-05-27T09:27:19.544950+0000","last_fullsized":"2024-05-27T09:27:19.544950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:39:59.034873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835983+0000","last_change":"2024-05-27T09:27:18.566621+0000","last_active":"2024-05-27T09:27:20.835983+0000","last_peered":"2024-05-27T09:27:20.835983+0000","last_clean":"2024-05-27T09:27:20.835983+0000","last_became_active":"2024-05-27T09:27:18.566269+0000","last_became_peered":"2024-05-27T09:27:18.566269+0000","last_unstale":"2024-05-27T09:27:20.835983+0000","last_undegraded":"2024-05-27T09:27:20.835983+0000","last_fullsized":"2024-05-27T09:27:20.835983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:24:24.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835531+0000","last_change":"2024-05-27T09:27:18.560958+0000","last_active":"2024-05-27T09:27:20.835531+0000","last_peered":"2024-05-27T09:27:20.835531+0000","last_clean":"2024-05-27T09:27:20.835531+0000","last_became_active":"2024-05-27T09:27:18.560620+0000","last_became_peered":"2024-05-27T09:27:18.560620+0000","last_unstale":"2024-05-27T09:27:20.835531+0000","last_undegraded":"2024-05-27T09:27:20.835531+0000","last_fullsized":"2024-05-27T09:27:20.835531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:42:38.351682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542664+0000","last_change":"2024-05-27T09:27:18.569804+0000","last_active":"2024-05-27T09:27:19.542664+0000","last_peered":"2024-05-27T09:27:19.542664+0000","last_clean":"2024-05-27T09:27:19.542664+0000","last_became_active":"2024-05-27T09:27:18.569481+0000","last_became_peered":"2024-05-27T09:27:18.569481+0000","last_unstale":"2024-05-27T09:27:19.542664+0000","last_undegraded":"2024-05-27T09:27:19.542664+0000","last_fullsized":"2024-05-27T09:27:19.542664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:55:03.583659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543808+0000","last_change":"2024-05-27T09:27:18.568138+0000","last_active":"2024-05-27T09:27:19.543808+0000","last_peered":"2024-05-27T09:27:19.543808+0000","last_clean":"2024-05-27T09:27:19.543808+0000","last_became_active":"2024-05-27T09:27:18.567795+0000","last_became_peered":"2024-05-27T09:27:18.567795+0000","last_unstale":"2024-05-27T09:27:19.543808+0000","last_undegraded":"2024-05-27T09:27:19.543808+0000","last_fullsized":"2024-05-27T09:27:19.543808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:17:42.005593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.540719+0000","last_change":"2024-05-27T09:27:18.573984+0000","last_active":"2024-05-27T09:27:19.540719+0000","last_peered":"2024-05-27T09:27:19.540719+0000","last_clean":"2024-05-27T09:27:19.540719+0000","last_became_active":"2024-05-27T09:27:18.573576+0000","last_became_peered":"2024-05-27T09:27:18.573576+0000","last_unstale":"2024-05-27T09:27:19.540719+0000","last_undegraded":"2024-05-27T09:27:19.540719+0000","last_fullsized":"2024-05-27T09:27:19.540719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:06.224637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543010+0000","last_change":"2024-05-27T09:27:18.543122+0000","last_active":"2024-05-27T09:27:19.543010+0000","last_peered":"2024-05-27T09:27:19.543010+0000","last_clean":"2024-05-27T09:27:19.543010+0000","last_became_active":"2024-05-27T09:27:18.542593+0000","last_became_peered":"2024-05-27T09:27:18.542593+0000","last_unstale":"2024-05-27T09:27:19.543010+0000","last_undegraded":"2024-05-27T09:27:19.543010+0000","last_fullsized":"2024-05-27T09:27:19.543010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:28:19.579432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835946+0000","last_change":"2024-05-27T09:27:18.562292+0000","last_active":"2024-05-27T09:27:20.835946+0000","last_peered":"2024-05-27T09:27:20.835946+0000","last_clean":"2024-05-27T09:27:20.835946+0000","last_became_active":"2024-05-27T09:27:18.561788+0000","last_became_peered":"2024-05-27T09:27:18.561788+0000","last_unstale":"2024-05-27T09:27:20.835946+0000","last_undegraded":"2024-05-27T09:27:20.835946+0000","last_fullsized":"2024-05-27T09:27:20.835946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:32:39.043268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542996+0000","last_change":"2024-05-27T09:27:18.583770+0000","last_active":"2024-05-27T09:27:19.542996+0000","last_peered":"2024-05-27T09:27:19.542996+0000","last_clean":"2024-05-27T09:27:19.542996+0000","last_became_active":"2024-05-27T09:27:18.582041+0000","last_became_peered":"2024-05-27T09:27:18.582041+0000","last_unstale":"2024-05-27T09:27:19.542996+0000","last_undegraded":"2024-05-27T09:27:19.542996+0000","last_fullsized":"2024-05-27T09:27:19.542996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:19.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.545125+0000","last_change":"2024-05-27T09:27:18.574658+0000","last_active":"2024-05-27T09:27:19.545125+0000","last_peered":"2024-05-27T09:27:19.545125+0000","last_clean":"2024-05-27T09:27:19.545125+0000","last_became_active":"2024-05-27T09:27:18.574409+0000","last_became_peered":"2024-05-27T09:27:18.574409+0000","last_unstale":"2024-05-27T09:27:19.545125+0000","last_undegraded":"2024-05-27T09:27:19.545125+0000","last_fullsized":"2024-05-27T09:27:19.545125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:19.131343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544558+0000","last_change":"2024-05-27T09:27:18.579141+0000","last_active":"2024-05-27T09:27:19.544558+0000","last_peered":"2024-05-27T09:27:19.544558+0000","last_clean":"2024-05-27T09:27:19.544558+0000","last_became_active":"2024-05-27T09:27:18.578679+0000","last_became_peered":"2024-05-27T09:27:18.578679+0000","last_unstale":"2024-05-27T09:27:19.544558+0000","last_undegraded":"2024-05-27T09:27:19.544558+0000","last_fullsized":"2024-05-27T09:27:19.544558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:44.336453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544110+0000","last_change":"2024-05-27T09:27:18.547375+0000","last_active":"2024-05-27T09:27:19.544110+0000","last_peered":"2024-05-27T09:27:19.544110+0000","last_clean":"2024-05-27T09:27:19.544110+0000","last_became_active":"2024-05-27T09:27:18.544328+0000","last_became_peered":"2024-05-27T09:27:18.544328+0000","last_unstale":"2024-05-27T09:27:19.544110+0000","last_undegraded":"2024-05-27T09:27:19.544110+0000","last_fullsized":"2024-05-27T09:27:19.544110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:59:50.499894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836590+0000","last_change":"2024-05-27T09:27:18.561903+0000","last_active":"2024-05-27T09:27:20.836590+0000","last_peered":"2024-05-27T09:27:20.836590+0000","last_clean":"2024-05-27T09:27:20.836590+0000","last_became_active":"2024-05-27T09:27:18.561493+0000","last_became_peered":"2024-05-27T09:27:18.561493+0000","last_unstale":"2024-05-27T09:27:20.836590+0000","last_undegraded":"2024-05-27T09:27:20.836590+0000","last_fullsized":"2024-05-27T09:27:20.836590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:28:31.470269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836378+0000","last_change":"2024-05-27T09:27:18.563168+0000","last_active":"2024-05-27T09:27:20.836378+0000","last_peered":"2024-05-27T09:27:20.836378+0000","last_clean":"2024-05-27T09:27:20.836378+0000","last_became_active":"2024-05-27T09:27:18.562843+0000","last_became_peered":"2024-05-27T09:27:18.562843+0000","last_unstale":"2024-05-27T09:27:20.836378+0000","last_undegraded":"2024-05-27T09:27:20.836378+0000","last_fullsized":"2024-05-27T09:27:20.836378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:33:36.671764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835457+0000","last_change":"2024-05-27T09:27:18.566502+0000","last_active":"2024-05-27T09:27:20.835457+0000","last_peered":"2024-05-27T09:27:20.835457+0000","last_clean":"2024-05-27T09:27:20.835457+0000","last_became_active":"2024-05-27T09:27:18.566108+0000","last_became_peered":"2024-05-27T09:27:18.566108+0000","last_unstale":"2024-05-27T09:27:20.835457+0000","last_undegraded":"2024-05-27T09:27:20.835457+0000","last_fullsized":"2024-05-27T09:27:20.835457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:44:47.447245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836471+0000","last_change":"2024-05-27T09:27:18.582595+0000","last_active":"2024-05-27T09:27:20.836471+0000","last_peered":"2024-05-27T09:27:20.836471+0000","last_clean":"2024-05-27T09:27:20.836471+0000","last_became_active":"2024-05-27T09:27:18.581719+0000","last_became_peered":"2024-05-27T09:27:18.581719+0000","last_unstale":"2024-05-27T09:27:20.836471+0000","last_undegraded":"2024-05-27T09:27:20.836471+0000","last_fullsized":"2024-05-27T09:27:20.836471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:48:19.573654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837234+0000","last_change":"2024-05-27T09:27:18.577565+0000","last_active":"2024-05-27T09:27:20.837234+0000","last_peered":"2024-05-27T09:27:20.837234+0000","last_clean":"2024-05-27T09:27:20.837234+0000","last_became_active":"2024-05-27T09:27:18.577193+0000","last_became_peered":"2024-05-27T09:27:18.577193+0000","last_unstale":"2024-05-27T09:27:20.837234+0000","last_undegraded":"2024-05-27T09:27:20.837234+0000","last_fullsized":"2024-05-27T09:27:20.837234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:43:20.065873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835530+0000","last_change":"2024-05-27T09:27:18.571089+0000","last_active":"2024-05-27T09:27:20.835530+0000","last_peered":"2024-05-27T09:27:20.835530+0000","last_clean":"2024-05-27T09:27:20.835530+0000","last_became_active":"2024-05-27T09:27:18.570712+0000","last_became_peered":"2024-05-27T09:27:18.570712+0000","last_unstale":"2024-05-27T09:27:20.835530+0000","last_undegraded":"2024-05-27T09:27:20.835530+0000","last_fullsized":"2024-05-27T09:27:20.835530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:40:19.310581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837956+0000","last_change":"2024-05-27T09:27:18.572675+0000","last_active":"2024-05-27T09:27:20.837956+0000","last_peered":"2024-05-27T09:27:20.837956+0000","last_clean":"2024-05-27T09:27:20.837956+0000","last_became_active":"2024-05-27T09:27:18.564643+0000","last_became_peered":"2024-05-27T09:27:18.564643+0000","last_unstale":"2024-05-27T09:27:20.837956+0000","last_undegraded":"2024-05-27T09:27:20.837956+0000","last_fullsized":"2024-05-27T09:27:20.837956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:36:15.602241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544179+0000","last_change":"2024-05-27T09:27:18.544554+0000","last_active":"2024-05-27T09:27:19.544179+0000","last_peered":"2024-05-27T09:27:19.544179+0000","last_clean":"2024-05-27T09:27:19.544179+0000","last_became_active":"2024-05-27T09:27:18.543777+0000","last_became_peered":"2024-05-27T09:27:18.543777+0000","last_unstale":"2024-05-27T09:27:19.544179+0000","last_undegraded":"2024-05-27T09:27:19.544179+0000","last_fullsized":"2024-05-27T09:27:19.544179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:35:43.354270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835033+0000","last_change":"2024-05-27T09:27:18.571321+0000","last_active":"2024-05-27T09:27:20.835033+0000","last_peered":"2024-05-27T09:27:20.835033+0000","last_clean":"2024-05-27T09:27:20.835033+0000","last_became_active":"2024-05-27T09:27:18.570948+0000","last_became_peered":"2024-05-27T09:27:18.570948+0000","last_unstale":"2024-05-27T09:27:20.835033+0000","last_undegraded":"2024-05-27T09:27:20.835033+0000","last_fullsized":"2024-05-27T09:27:20.835033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:28:31.470269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539490+0000","last_change":"2024-05-27T09:27:18.544282+0000","last_active":"2024-05-27T09:27:19.539490+0000","last_peered":"2024-05-27T09:27:19.539490+0000","last_clean":"2024-05-27T09:27:19.539490+0000","last_became_active":"2024-05-27T09:27:18.543921+0000","last_became_peered":"2024-05-27T09:27:18.543921+0000","last_unstale":"2024-05-27T09:27:19.539490+0000","last_undegraded":"2024-05-27T09:27:19.539490+0000","last_fullsized":"2024-05-27T09:27:19.539490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:27.225150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835732+0000","last_change":"2024-05-27T09:27:18.575415+0000","last_active":"2024-05-27T09:27:20.835732+0000","last_peered":"2024-05-27T09:27:20.835732+0000","last_clean":"2024-05-27T09:27:20.835732+0000","last_became_active":"2024-05-27T09:27:18.575115+0000","last_became_peered":"2024-05-27T09:27:18.575115+0000","last_unstale":"2024-05-27T09:27:20.835732+0000","last_undegraded":"2024-05-27T09:27:20.835732+0000","last_fullsized":"2024-05-27T09:27:20.835732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:50:48.091125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836496+0000","last_change":"2024-05-27T09:27:18.564422+0000","last_active":"2024-05-27T09:27:20.836496+0000","last_peered":"2024-05-27T09:27:20.836496+0000","last_clean":"2024-05-27T09:27:20.836496+0000","last_became_active":"2024-05-27T09:27:18.563991+0000","last_became_peered":"2024-05-27T09:27:18.563991+0000","last_unstale":"2024-05-27T09:27:20.836496+0000","last_undegraded":"2024-05-27T09:27:20.836496+0000","last_fullsized":"2024-05-27T09:27:20.836496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:19.131343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836032+0000","last_change":"2024-05-27T09:27:18.563557+0000","last_active":"2024-05-27T09:27:20.836032+0000","last_peered":"2024-05-27T09:27:20.836032+0000","last_clean":"2024-05-27T09:27:20.836032+0000","last_became_active":"2024-05-27T09:27:18.562803+0000","last_became_peered":"2024-05-27T09:27:18.562803+0000","last_unstale":"2024-05-27T09:27:20.836032+0000","last_undegraded":"2024-05-27T09:27:20.836032+0000","last_fullsized":"2024-05-27T09:27:20.836032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:47:54.271208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542223+0000","last_change":"2024-05-27T09:27:18.590430+0000","last_active":"2024-05-27T09:27:19.542223+0000","last_peered":"2024-05-27T09:27:19.542223+0000","last_clean":"2024-05-27T09:27:19.542223+0000","last_became_active":"2024-05-27T09:27:18.590083+0000","last_became_peered":"2024-05-27T09:27:18.590083+0000","last_unstale":"2024-05-27T09:27:19.542223+0000","last_undegraded":"2024-05-27T09:27:19.542223+0000","last_fullsized":"2024-05-27T09:27:19.542223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:03.762818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837167+0000","last_change":"2024-05-27T09:27:18.558248+0000","last_active":"2024-05-27T09:27:20.837167+0000","last_peered":"2024-05-27T09:27:20.837167+0000","last_clean":"2024-05-27T09:27:20.837167+0000","last_became_active":"2024-05-27T09:27:18.557953+0000","last_became_peered":"2024-05-27T09:27:18.557953+0000","last_unstale":"2024-05-27T09:27:20.837167+0000","last_undegraded":"2024-05-27T09:27:20.837167+0000","last_fullsized":"2024-05-27T09:27:20.837167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:44:47.447245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836514+0000","last_change":"2024-05-27T09:27:18.563213+0000","last_active":"2024-05-27T09:27:20.836514+0000","last_peered":"2024-05-27T09:27:20.836514+0000","last_clean":"2024-05-27T09:27:20.836514+0000","last_became_active":"2024-05-27T09:27:18.562839+0000","last_became_peered":"2024-05-27T09:27:18.562839+0000","last_unstale":"2024-05-27T09:27:20.836514+0000","last_undegraded":"2024-05-27T09:27:20.836514+0000","last_fullsized":"2024-05-27T09:27:20.836514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:43.851698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836053+0000","last_change":"2024-05-27T09:27:18.562237+0000","last_active":"2024-05-27T09:27:20.836053+0000","last_peered":"2024-05-27T09:27:20.836053+0000","last_clean":"2024-05-27T09:27:20.836053+0000","last_became_active":"2024-05-27T09:27:18.561475+0000","last_became_peered":"2024-05-27T09:27:18.561475+0000","last_unstale":"2024-05-27T09:27:20.836053+0000","last_undegraded":"2024-05-27T09:27:20.836053+0000","last_fullsized":"2024-05-27T09:27:20.836053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:39:58.110527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837611+0000","last_change":"2024-05-27T09:27:18.563469+0000","last_active":"2024-05-27T09:27:20.837611+0000","last_peered":"2024-05-27T09:27:20.837611+0000","last_clean":"2024-05-27T09:27:20.837611+0000","last_became_active":"2024-05-27T09:27:18.563103+0000","last_became_peered":"2024-05-27T09:27:18.563103+0000","last_unstale":"2024-05-27T09:27:20.837611+0000","last_undegraded":"2024-05-27T09:27:20.837611+0000","last_fullsized":"2024-05-27T09:27:20.837611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:53:57.281169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.834984+0000","last_change":"2024-05-27T09:27:18.560551+0000","last_active":"2024-05-27T09:27:20.834984+0000","last_peered":"2024-05-27T09:27:20.834984+0000","last_clean":"2024-05-27T09:27:20.834984+0000","last_became_active":"2024-05-27T09:27:18.560244+0000","last_became_peered":"2024-05-27T09:27:18.560244+0000","last_unstale":"2024-05-27T09:27:20.834984+0000","last_undegraded":"2024-05-27T09:27:20.834984+0000","last_fullsized":"2024-05-27T09:27:20.834984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:00.131180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.538536+0000","last_change":"2024-05-27T09:27:18.568200+0000","last_active":"2024-05-27T09:27:19.538536+0000","last_peered":"2024-05-27T09:27:19.538536+0000","last_clean":"2024-05-27T09:27:19.538536+0000","last_became_active":"2024-05-27T09:27:18.567845+0000","last_became_peered":"2024-05-27T09:27:18.567845+0000","last_unstale":"2024-05-27T09:27:19.538536+0000","last_undegraded":"2024-05-27T09:27:19.538536+0000","last_fullsized":"2024-05-27T09:27:19.538536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:54.726617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.540019+0000","last_change":"2024-05-27T09:27:18.560521+0000","last_active":"2024-05-27T09:27:19.540019+0000","last_peered":"2024-05-27T09:27:19.540019+0000","last_clean":"2024-05-27T09:27:19.540019+0000","last_became_active":"2024-05-27T09:27:18.556156+0000","last_became_peered":"2024-05-27T09:27:18.556156+0000","last_unstale":"2024-05-27T09:27:19.540019+0000","last_undegraded":"2024-05-27T09:27:19.540019+0000","last_fullsized":"2024-05-27T09:27:19.540019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:03.762818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:21.370264+0000","last_change":"2024-05-27T09:27:17.548038+0000","last_active":"2024-05-27T09:27:21.370264+0000","last_peered":"2024-05-27T09:27:21.370264+0000","last_clean":"2024-05-27T09:27:21.370264+0000","last_became_active":"2024-05-27T09:27:17.547653+0000","last_became_peered":"2024-05-27T09:27:17.547653+0000","last_unstale":"2024-05-27T09:27:21.370264+0000","last_undegraded":"2024-05-27T09:27:21.370264+0000","last_fullsized":"2024-05-27T09:27:21.370264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835265+0000","last_change":"2024-05-27T09:27:18.543048+0000","last_active":"2024-05-27T09:27:20.835265+0000","last_peered":"2024-05-27T09:27:20.835265+0000","last_clean":"2024-05-27T09:27:20.835265+0000","last_became_active":"2024-05-27T09:27:18.542715+0000","last_became_peered":"2024-05-27T09:27:18.542715+0000","last_unstale":"2024-05-27T09:27:20.835265+0000","last_undegraded":"2024-05-27T09:27:20.835265+0000","last_fullsized":"2024-05-27T09:27:20.835265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:46:49.444709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837344+0000","last_change":"2024-05-27T09:27:17.570014+0000","last_active":"2024-05-27T09:27:20.837344+0000","last_peered":"2024-05-27T09:27:20.837344+0000","last_clean":"2024-05-27T09:27:20.837344+0000","last_became_active":"2024-05-27T09:27:17.569673+0000","last_became_peered":"2024-05-27T09:27:17.569673+0000","last_unstale":"2024-05-27T09:27:20.837344+0000","last_undegraded":"2024-05-27T09:27:20.837344+0000","last_fullsized":"2024-05-27T09:27:20.837344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:26.210041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836496+0000","last_change":"2024-05-27T09:27:17.565489+0000","last_active":"2024-05-27T09:27:20.836496+0000","last_peered":"2024-05-27T09:27:20.836496+0000","last_clean":"2024-05-27T09:27:20.836496+0000","last_became_active":"2024-05-27T09:27:17.565137+0000","last_became_peered":"2024-05-27T09:27:17.565137+0000","last_unstale":"2024-05-27T09:27:20.836496+0000","last_undegraded":"2024-05-27T09:27:20.836496+0000","last_fullsized":"2024-05-27T09:27:20.836496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836835+0000","last_change":"2024-05-27T09:27:18.565294+0000","last_active":"2024-05-27T09:27:20.836835+0000","last_peered":"2024-05-27T09:27:20.836835+0000","last_clean":"2024-05-27T09:27:20.836835+0000","last_became_active":"2024-05-27T09:27:18.564115+0000","last_became_peered":"2024-05-27T09:27:18.564115+0000","last_unstale":"2024-05-27T09:27:20.836835+0000","last_undegraded":"2024-05-27T09:27:20.836835+0000","last_fullsized":"2024-05-27T09:27:20.836835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:57:20.488849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836662+0000","last_change":"2024-05-27T09:27:17.561204+0000","last_active":"2024-05-27T09:27:20.836662+0000","last_peered":"2024-05-27T09:27:20.836662+0000","last_clean":"2024-05-27T09:27:20.836662+0000","last_became_active":"2024-05-27T09:27:17.560149+0000","last_became_peered":"2024-05-27T09:27:17.560149+0000","last_unstale":"2024-05-27T09:27:20.836662+0000","last_undegraded":"2024-05-27T09:27:20.836662+0000","last_fullsized":"2024-05-27T09:27:20.836662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.284270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836343+0000","last_change":"2024-05-27T09:27:18.542204+0000","last_active":"2024-05-27T09:27:20.836343+0000","last_peered":"2024-05-27T09:27:20.836343+0000","last_clean":"2024-05-27T09:27:20.836343+0000","last_became_active":"2024-05-27T09:27:18.541946+0000","last_became_peered":"2024-05-27T09:27:18.541946+0000","last_unstale":"2024-05-27T09:27:20.836343+0000","last_undegraded":"2024-05-27T09:27:20.836343+0000","last_fullsized":"2024-05-27T09:27:20.836343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:50:48.091125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.541754+0000","last_change":"2024-05-27T09:27:17.561714+0000","last_active":"2024-05-27T09:27:19.541754+0000","last_peered":"2024-05-27T09:27:19.541754+0000","last_clean":"2024-05-27T09:27:19.541754+0000","last_became_active":"2024-05-27T09:27:17.561355+0000","last_became_peered":"2024-05-27T09:27:17.561355+0000","last_unstale":"2024-05-27T09:27:19.541754+0000","last_undegraded":"2024-05-27T09:27:19.541754+0000","last_fullsized":"2024-05-27T09:27:19.541754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:07:02.481451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542128+0000","last_change":"2024-05-27T09:27:18.588199+0000","last_active":"2024-05-27T09:27:19.542128+0000","last_peered":"2024-05-27T09:27:19.542128+0000","last_clean":"2024-05-27T09:27:19.542128+0000","last_became_active":"2024-05-27T09:27:18.587921+0000","last_became_peered":"2024-05-27T09:27:18.587921+0000","last_unstale":"2024-05-27T09:27:19.542128+0000","last_undegraded":"2024-05-27T09:27:19.542128+0000","last_fullsized":"2024-05-27T09:27:19.542128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:17.895211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544807+0000","last_change":"2024-05-27T09:27:17.565854+0000","last_active":"2024-05-27T09:27:19.544807+0000","last_peered":"2024-05-27T09:27:19.544807+0000","last_clean":"2024-05-27T09:27:19.544807+0000","last_became_active":"2024-05-27T09:27:17.565488+0000","last_became_peered":"2024-05-27T09:27:17.565488+0000","last_unstale":"2024-05-27T09:27:19.544807+0000","last_undegraded":"2024-05-27T09:27:19.544807+0000","last_fullsized":"2024-05-27T09:27:19.544807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:53.711508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836966+0000","last_change":"2024-05-27T09:27:18.585592+0000","last_active":"2024-05-27T09:27:20.836966+0000","last_peered":"2024-05-27T09:27:20.836966+0000","last_clean":"2024-05-27T09:27:20.836966+0000","last_became_active":"2024-05-27T09:27:18.585243+0000","last_became_peered":"2024-05-27T09:27:18.585243+0000","last_unstale":"2024-05-27T09:27:20.836966+0000","last_undegraded":"2024-05-27T09:27:20.836966+0000","last_fullsized":"2024-05-27T09:27:20.836966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:28.872653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.547985+0000","last_change":"2024-05-27T09:27:17.548651+0000","last_active":"2024-05-27T09:27:23.547985+0000","last_peered":"2024-05-27T09:27:23.547985+0000","last_clean":"2024-05-27T09:27:23.547985+0000","last_became_active":"2024-05-27T09:27:17.548307+0000","last_became_peered":"2024-05-27T09:27:17.548307+0000","last_unstale":"2024-05-27T09:27:23.547985+0000","last_undegraded":"2024-05-27T09:27:23.547985+0000","last_fullsized":"2024-05-27T09:27:23.547985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:48.905422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837273+0000","last_change":"2024-05-27T09:27:18.582895+0000","last_active":"2024-05-27T09:27:20.837273+0000","last_peered":"2024-05-27T09:27:20.837273+0000","last_clean":"2024-05-27T09:27:20.837273+0000","last_became_active":"2024-05-27T09:27:18.581742+0000","last_became_peered":"2024-05-27T09:27:18.581742+0000","last_unstale":"2024-05-27T09:27:20.837273+0000","last_undegraded":"2024-05-27T09:27:20.837273+0000","last_fullsized":"2024-05-27T09:27:20.837273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:23:26.888905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.560732+0000","last_change":"2024-05-27T09:27:17.556419+0000","last_active":"2024-05-27T09:27:23.560732+0000","last_peered":"2024-05-27T09:27:23.560732+0000","last_clean":"2024-05-27T09:27:23.560732+0000","last_became_active":"2024-05-27T09:27:17.556036+0000","last_became_peered":"2024-05-27T09:27:17.556036+0000","last_unstale":"2024-05-27T09:27:23.560732+0000","last_undegraded":"2024-05-27T09:27:23.560732+0000","last_fullsized":"2024-05-27T09:27:23.560732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:48:20.718085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544281+0000","last_change":"2024-05-27T09:27:18.568187+0000","last_active":"2024-05-27T09:27:19.544281+0000","last_peered":"2024-05-27T09:27:19.544281+0000","last_clean":"2024-05-27T09:27:19.544281+0000","last_became_active":"2024-05-27T09:27:18.565500+0000","last_became_peered":"2024-05-27T09:27:18.565500+0000","last_unstale":"2024-05-27T09:27:19.544281+0000","last_undegraded":"2024-05-27T09:27:19.544281+0000","last_fullsized":"2024-05-27T09:27:19.544281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:16:08.688048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.550202+0000","last_change":"2024-05-27T09:27:17.572168+0000","last_active":"2024-05-27T09:27:23.550202+0000","last_peered":"2024-05-27T09:27:23.550202+0000","last_clean":"2024-05-27T09:27:23.550202+0000","last_became_active":"2024-05-27T09:27:17.571709+0000","last_became_peered":"2024-05-27T09:27:17.571709+0000","last_unstale":"2024-05-27T09:27:23.550202+0000","last_undegraded":"2024-05-27T09:27:23.550202+0000","last_fullsized":"2024-05-27T09:27:23.550202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836574+0000","last_change":"2024-05-27T09:27:18.564930+0000","last_active":"2024-05-27T09:27:20.836574+0000","last_peered":"2024-05-27T09:27:20.836574+0000","last_clean":"2024-05-27T09:27:20.836574+0000","last_became_active":"2024-05-27T09:27:18.564589+0000","last_became_peered":"2024-05-27T09:27:18.564589+0000","last_unstale":"2024-05-27T09:27:20.836574+0000","last_undegraded":"2024-05-27T09:27:20.836574+0000","last_fullsized":"2024-05-27T09:27:20.836574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:21:00.644551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837541+0000","last_change":"2024-05-27T09:27:17.561344+0000","last_active":"2024-05-27T09:27:20.837541+0000","last_peered":"2024-05-27T09:27:20.837541+0000","last_clean":"2024-05-27T09:27:20.837541+0000","last_became_active":"2024-05-27T09:27:17.554860+0000","last_became_peered":"2024-05-27T09:27:17.554860+0000","last_unstale":"2024-05-27T09:27:20.837541+0000","last_undegraded":"2024-05-27T09:27:20.837541+0000","last_fullsized":"2024-05-27T09:27:20.837541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:16.880101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837236+0000","last_change":"2024-05-27T09:27:18.575317+0000","last_active":"2024-05-27T09:27:20.837236+0000","last_peered":"2024-05-27T09:27:20.837236+0000","last_clean":"2024-05-27T09:27:20.837236+0000","last_became_active":"2024-05-27T09:27:18.572024+0000","last_became_peered":"2024-05-27T09:27:18.572024+0000","last_unstale":"2024-05-27T09:27:20.837236+0000","last_undegraded":"2024-05-27T09:27:20.837236+0000","last_fullsized":"2024-05-27T09:27:20.837236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:23:35.418697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543181+0000","last_change":"2024-05-27T09:27:17.575859+0000","last_active":"2024-05-27T09:27:19.543181+0000","last_peered":"2024-05-27T09:27:19.543181+0000","last_clean":"2024-05-27T09:27:19.543181+0000","last_became_active":"2024-05-27T09:27:17.574695+0000","last_became_peered":"2024-05-27T09:27:17.574695+0000","last_unstale":"2024-05-27T09:27:19.543181+0000","last_undegraded":"2024-05-27T09:27:19.543181+0000","last_fullsized":"2024-05-27T09:27:19.543181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:24:19.379819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539060+0000","last_change":"2024-05-27T09:27:18.555137+0000","last_active":"2024-05-27T09:27:19.539060+0000","last_peered":"2024-05-27T09:27:19.539060+0000","last_clean":"2024-05-27T09:27:19.539060+0000","last_became_active":"2024-05-27T09:27:18.554932+0000","last_became_peered":"2024-05-27T09:27:18.554932+0000","last_unstale":"2024-05-27T09:27:19.539060+0000","last_undegraded":"2024-05-27T09:27:19.539060+0000","last_fullsized":"2024-05-27T09:27:19.539060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:06.224637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.556322+0000","last_change":"2024-05-27T09:27:17.547895+0000","last_active":"2024-05-27T09:27:23.556322+0000","last_peered":"2024-05-27T09:27:23.556322+0000","last_clean":"2024-05-27T09:27:23.556322+0000","last_became_active":"2024-05-27T09:27:17.547523+0000","last_became_peered":"2024-05-27T09:27:17.547523+0000","last_unstale":"2024-05-27T09:27:23.556322+0000","last_undegraded":"2024-05-27T09:27:23.556322+0000","last_fullsized":"2024-05-27T09:27:23.556322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836706+0000","last_change":"2024-05-27T09:27:18.584284+0000","last_active":"2024-05-27T09:27:20.836706+0000","last_peered":"2024-05-27T09:27:20.836706+0000","last_clean":"2024-05-27T09:27:20.836706+0000","last_became_active":"2024-05-27T09:27:18.582360+0000","last_became_peered":"2024-05-27T09:27:18.582360+0000","last_unstale":"2024-05-27T09:27:20.836706+0000","last_undegraded":"2024-05-27T09:27:20.836706+0000","last_fullsized":"2024-05-27T09:27:20.836706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:32.535070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.545397+0000","last_change":"2024-05-27T09:27:17.571481+0000","last_active":"2024-05-27T09:27:19.545397+0000","last_peered":"2024-05-27T09:27:19.545397+0000","last_clean":"2024-05-27T09:27:19.545397+0000","last_became_active":"2024-05-27T09:27:17.571146+0000","last_became_peered":"2024-05-27T09:27:17.571146+0000","last_unstale":"2024-05-27T09:27:19.545397+0000","last_undegraded":"2024-05-27T09:27:19.545397+0000","last_fullsized":"2024-05-27T09:27:19.545397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:18.038221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542481+0000","last_change":"2024-05-27T09:27:18.580425+0000","last_active":"2024-05-27T09:27:19.542481+0000","last_peered":"2024-05-27T09:27:19.542481+0000","last_clean":"2024-05-27T09:27:19.542481+0000","last_became_active":"2024-05-27T09:27:18.579440+0000","last_became_peered":"2024-05-27T09:27:18.579440+0000","last_unstale":"2024-05-27T09:27:19.542481+0000","last_undegraded":"2024-05-27T09:27:19.542481+0000","last_fullsized":"2024-05-27T09:27:19.542481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:19.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.554242+0000","last_change":"2024-05-27T09:27:17.551295+0000","last_active":"2024-05-27T09:27:23.554242+0000","last_peered":"2024-05-27T09:27:23.554242+0000","last_clean":"2024-05-27T09:27:23.554242+0000","last_became_active":"2024-05-27T09:27:17.550861+0000","last_became_peered":"2024-05-27T09:27:17.550861+0000","last_unstale":"2024-05-27T09:27:23.554242+0000","last_undegraded":"2024-05-27T09:27:23.554242+0000","last_fullsized":"2024-05-27T09:27:23.554242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:13.955191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.545103+0000","last_change":"2024-05-27T09:27:18.547199+0000","last_active":"2024-05-27T09:27:19.545103+0000","last_peered":"2024-05-27T09:27:19.545103+0000","last_clean":"2024-05-27T09:27:19.545103+0000","last_became_active":"2024-05-27T09:27:18.543683+0000","last_became_peered":"2024-05-27T09:27:18.543683+0000","last_unstale":"2024-05-27T09:27:19.545103+0000","last_undegraded":"2024-05-27T09:27:19.545103+0000","last_fullsized":"2024-05-27T09:27:19.545103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:48:19.573654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:21.370295+0000","last_change":"2024-05-27T09:27:17.559255+0000","last_active":"2024-05-27T09:27:21.370295+0000","last_peered":"2024-05-27T09:27:21.370295+0000","last_clean":"2024-05-27T09:27:21.370295+0000","last_became_active":"2024-05-27T09:27:17.558973+0000","last_became_peered":"2024-05-27T09:27:17.558973+0000","last_unstale":"2024-05-27T09:27:21.370295+0000","last_undegraded":"2024-05-27T09:27:21.370295+0000","last_fullsized":"2024-05-27T09:27:21.370295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544602+0000","last_change":"2024-05-27T09:27:18.557423+0000","last_active":"2024-05-27T09:27:19.544602+0000","last_peered":"2024-05-27T09:27:19.544602+0000","last_clean":"2024-05-27T09:27:19.544602+0000","last_became_active":"2024-05-27T09:27:18.557076+0000","last_became_peered":"2024-05-27T09:27:18.557076+0000","last_unstale":"2024-05-27T09:27:19.544602+0000","last_undegraded":"2024-05-27T09:27:19.544602+0000","last_fullsized":"2024-05-27T09:27:19.544602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:11:04.612363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836934+0000","last_change":"2024-05-27T09:27:17.573700+0000","last_active":"2024-05-27T09:27:20.836934+0000","last_peered":"2024-05-27T09:27:20.836934+0000","last_clean":"2024-05-27T09:27:20.836934+0000","last_became_active":"2024-05-27T09:27:17.573484+0000","last_became_peered":"2024-05-27T09:27:17.573484+0000","last_unstale":"2024-05-27T09:27:20.836934+0000","last_undegraded":"2024-05-27T09:27:20.836934+0000","last_fullsized":"2024-05-27T09:27:20.836934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:48.905422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837609+0000","last_change":"2024-05-27T09:27:18.582777+0000","last_active":"2024-05-27T09:27:20.837609+0000","last_peered":"2024-05-27T09:27:20.837609+0000","last_clean":"2024-05-27T09:27:20.837609+0000","last_became_active":"2024-05-27T09:27:18.574959+0000","last_became_peered":"2024-05-27T09:27:18.574959+0000","last_unstale":"2024-05-27T09:27:20.837609+0000","last_undegraded":"2024-05-27T09:27:20.837609+0000","last_fullsized":"2024-05-27T09:27:20.837609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:40:41.206755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":110,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539210+0000","last_change":"2024-05-27T09:27:12.693423+0000","last_active":"2024-05-27T09:27:19.539210+0000","last_peered":"2024-05-27T09:27:19.539210+0000","last_clean":"2024-05-27T09:27:19.539210+0000","last_became_active":"2024-05-27T09:27:12.690301+0000","last_became_peered":"2024-05-27T09:27:12.690301+0000","last_unstale":"2024-05-27T09:27:19.539210+0000","last_undegraded":"2024-05-27T09:27:19.539210+0000","last_fullsized":"2024-05-27T09:27:19.539210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:11.463226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:11.463226+0000","last_clean_scrub_stamp":"2024-05-27T09:27:11.463226+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:01.810071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837730+0000","last_change":"2024-05-27T09:27:17.567670+0000","last_active":"2024-05-27T09:27:20.837730+0000","last_peered":"2024-05-27T09:27:20.837730+0000","last_clean":"2024-05-27T09:27:20.837730+0000","last_became_active":"2024-05-27T09:27:17.566292+0000","last_became_peered":"2024-05-27T09:27:17.566292+0000","last_unstale":"2024-05-27T09:27:20.837730+0000","last_undegraded":"2024-05-27T09:27:20.837730+0000","last_fullsized":"2024-05-27T09:27:20.837730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:18.038221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836742+0000","last_change":"2024-05-27T09:27:18.571124+0000","last_active":"2024-05-27T09:27:20.836742+0000","last_peered":"2024-05-27T09:27:20.836742+0000","last_clean":"2024-05-27T09:27:20.836742+0000","last_became_active":"2024-05-27T09:27:18.570782+0000","last_became_peered":"2024-05-27T09:27:18.570782+0000","last_unstale":"2024-05-27T09:27:20.836742+0000","last_undegraded":"2024-05-27T09:27:20.836742+0000","last_fullsized":"2024-05-27T09:27:20.836742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:39:58.110527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836667+0000","last_change":"2024-05-27T09:27:17.563259+0000","last_active":"2024-05-27T09:27:20.836667+0000","last_peered":"2024-05-27T09:27:20.836667+0000","last_clean":"2024-05-27T09:27:20.836667+0000","last_became_active":"2024-05-27T09:27:17.563010+0000","last_became_peered":"2024-05-27T09:27:17.563010+0000","last_unstale":"2024-05-27T09:27:20.836667+0000","last_undegraded":"2024-05-27T09:27:20.836667+0000","last_fullsized":"2024-05-27T09:27:20.836667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:53.711508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836371+0000","last_change":"2024-05-27T09:27:18.574466+0000","last_active":"2024-05-27T09:27:20.836371+0000","last_peered":"2024-05-27T09:27:20.836371+0000","last_clean":"2024-05-27T09:27:20.836371+0000","last_became_active":"2024-05-27T09:27:18.574090+0000","last_became_peered":"2024-05-27T09:27:18.574090+0000","last_unstale":"2024-05-27T09:27:20.836371+0000","last_undegraded":"2024-05-27T09:27:20.836371+0000","last_fullsized":"2024-05-27T09:27:20.836371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:41:22.515484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835113+0000","last_change":"2024-05-27T09:27:17.551310+0000","last_active":"2024-05-27T09:27:20.835113+0000","last_peered":"2024-05-27T09:27:20.835113+0000","last_clean":"2024-05-27T09:27:20.835113+0000","last_became_active":"2024-05-27T09:27:17.550982+0000","last_became_peered":"2024-05-27T09:27:17.550982+0000","last_unstale":"2024-05-27T09:27:20.835113+0000","last_undegraded":"2024-05-27T09:27:20.835113+0000","last_fullsized":"2024-05-27T09:27:20.835113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836574+0000","last_change":"2024-05-27T09:27:18.559416+0000","last_active":"2024-05-27T09:27:20.836574+0000","last_peered":"2024-05-27T09:27:20.836574+0000","last_clean":"2024-05-27T09:27:20.836574+0000","last_became_active":"2024-05-27T09:27:18.559017+0000","last_became_peered":"2024-05-27T09:27:18.559017+0000","last_unstale":"2024-05-27T09:27:20.836574+0000","last_undegraded":"2024-05-27T09:27:20.836574+0000","last_fullsized":"2024-05-27T09:27:20.836574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:33:48.400306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837680+0000","last_change":"2024-05-27T09:27:17.563947+0000","last_active":"2024-05-27T09:27:20.837680+0000","last_peered":"2024-05-27T09:27:20.837680+0000","last_clean":"2024-05-27T09:27:20.837680+0000","last_became_active":"2024-05-27T09:27:17.563640+0000","last_became_peered":"2024-05-27T09:27:17.563640+0000","last_unstale":"2024-05-27T09:27:20.837680+0000","last_undegraded":"2024-05-27T09:27:20.837680+0000","last_fullsized":"2024-05-27T09:27:20.837680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:02.747708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543615+0000","last_change":"2024-05-27T09:27:18.575835+0000","last_active":"2024-05-27T09:27:19.543615+0000","last_peered":"2024-05-27T09:27:19.543615+0000","last_clean":"2024-05-27T09:27:19.543615+0000","last_became_active":"2024-05-27T09:27:18.574710+0000","last_became_peered":"2024-05-27T09:27:18.574710+0000","last_unstale":"2024-05-27T09:27:19.543615+0000","last_undegraded":"2024-05-27T09:27:19.543615+0000","last_fullsized":"2024-05-27T09:27:19.543615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:16:00.934068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835781+0000","last_change":"2024-05-27T09:27:17.550749+0000","last_active":"2024-05-27T09:27:20.835781+0000","last_peered":"2024-05-27T09:27:20.835781+0000","last_clean":"2024-05-27T09:27:20.835781+0000","last_became_active":"2024-05-27T09:27:17.550378+0000","last_became_peered":"2024-05-27T09:27:17.550378+0000","last_unstale":"2024-05-27T09:27:20.835781+0000","last_undegraded":"2024-05-27T09:27:20.835781+0000","last_fullsized":"2024-05-27T09:27:20.835781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:20.167629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.541045+0000","last_change":"2024-05-27T09:27:18.567233+0000","last_active":"2024-05-27T09:27:19.541045+0000","last_peered":"2024-05-27T09:27:19.541045+0000","last_clean":"2024-05-27T09:27:19.541045+0000","last_became_active":"2024-05-27T09:27:18.566326+0000","last_became_peered":"2024-05-27T09:27:18.566326+0000","last_unstale":"2024-05-27T09:27:19.541045+0000","last_undegraded":"2024-05-27T09:27:19.541045+0000","last_fullsized":"2024-05-27T09:27:19.541045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:27.225150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837758+0000","last_change":"2024-05-27T09:27:17.557912+0000","last_active":"2024-05-27T09:27:20.837758+0000","last_peered":"2024-05-27T09:27:20.837758+0000","last_clean":"2024-05-27T09:27:20.837758+0000","last_became_active":"2024-05-27T09:27:17.557216+0000","last_became_peered":"2024-05-27T09:27:17.557216+0000","last_unstale":"2024-05-27T09:27:20.837758+0000","last_undegraded":"2024-05-27T09:27:20.837758+0000","last_fullsized":"2024-05-27T09:27:20.837758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:21:22.020724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836847+0000","last_change":"2024-05-27T09:27:18.563649+0000","last_active":"2024-05-27T09:27:20.836847+0000","last_peered":"2024-05-27T09:27:20.836847+0000","last_clean":"2024-05-27T09:27:20.836847+0000","last_became_active":"2024-05-27T09:27:18.562565+0000","last_became_peered":"2024-05-27T09:27:18.562565+0000","last_unstale":"2024-05-27T09:27:20.836847+0000","last_undegraded":"2024-05-27T09:27:20.836847+0000","last_fullsized":"2024-05-27T09:27:20.836847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:46:49.444709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835661+0000","last_change":"2024-05-27T09:27:17.571911+0000","last_active":"2024-05-27T09:27:20.835661+0000","last_peered":"2024-05-27T09:27:20.835661+0000","last_clean":"2024-05-27T09:27:20.835661+0000","last_became_active":"2024-05-27T09:27:17.571551+0000","last_became_peered":"2024-05-27T09:27:17.571551+0000","last_unstale":"2024-05-27T09:27:20.835661+0000","last_undegraded":"2024-05-27T09:27:20.835661+0000","last_fullsized":"2024-05-27T09:27:20.835661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:20.167629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835899+0000","last_change":"2024-05-27T09:27:18.558780+0000","last_active":"2024-05-27T09:27:20.835899+0000","last_peered":"2024-05-27T09:27:20.835899+0000","last_clean":"2024-05-27T09:27:20.835899+0000","last_became_active":"2024-05-27T09:27:18.558463+0000","last_became_peered":"2024-05-27T09:27:18.558463+0000","last_unstale":"2024-05-27T09:27:20.835899+0000","last_undegraded":"2024-05-27T09:27:20.835899+0000","last_fullsized":"2024-05-27T09:27:20.835899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:42:38.351682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.559527+0000","last_change":"2024-05-27T09:27:17.567973+0000","last_active":"2024-05-27T09:27:23.559527+0000","last_peered":"2024-05-27T09:27:23.559527+0000","last_clean":"2024-05-27T09:27:23.559527+0000","last_became_active":"2024-05-27T09:27:17.567650+0000","last_became_peered":"2024-05-27T09:27:17.567650+0000","last_unstale":"2024-05-27T09:27:23.559527+0000","last_undegraded":"2024-05-27T09:27:23.559527+0000","last_fullsized":"2024-05-27T09:27:23.559527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:25:52.924185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835408+0000","last_change":"2024-05-27T09:27:18.554845+0000","last_active":"2024-05-27T09:27:20.835408+0000","last_peered":"2024-05-27T09:27:20.835408+0000","last_clean":"2024-05-27T09:27:20.835408+0000","last_became_active":"2024-05-27T09:27:18.554462+0000","last_became_peered":"2024-05-27T09:27:18.554462+0000","last_unstale":"2024-05-27T09:27:20.835408+0000","last_undegraded":"2024-05-27T09:27:20.835408+0000","last_fullsized":"2024-05-27T09:27:20.835408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:44:47.447245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.541867+0000","last_change":"2024-05-27T09:27:17.570300+0000","last_active":"2024-05-27T09:27:19.541867+0000","last_peered":"2024-05-27T09:27:19.541867+0000","last_clean":"2024-05-27T09:27:19.541867+0000","last_became_active":"2024-05-27T09:27:17.569685+0000","last_became_peered":"2024-05-27T09:27:17.569685+0000","last_unstale":"2024-05-27T09:27:19.541867+0000","last_undegraded":"2024-05-27T09:27:19.541867+0000","last_fullsized":"2024-05-27T09:27:19.541867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:26.210041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836430+0000","last_change":"2024-05-27T09:27:18.562774+0000","last_active":"2024-05-27T09:27:20.836430+0000","last_peered":"2024-05-27T09:27:20.836430+0000","last_clean":"2024-05-27T09:27:20.836430+0000","last_became_active":"2024-05-27T09:27:18.562455+0000","last_became_peered":"2024-05-27T09:27:18.562455+0000","last_unstale":"2024-05-27T09:27:20.836430+0000","last_undegraded":"2024-05-27T09:27:20.836430+0000","last_fullsized":"2024-05-27T09:27:20.836430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:39:59.034873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836403+0000","last_change":"2024-05-27T09:27:17.563172+0000","last_active":"2024-05-27T09:27:20.836403+0000","last_peered":"2024-05-27T09:27:20.836403+0000","last_clean":"2024-05-27T09:27:20.836403+0000","last_became_active":"2024-05-27T09:27:17.562941+0000","last_became_peered":"2024-05-27T09:27:17.562941+0000","last_unstale":"2024-05-27T09:27:20.836403+0000","last_undegraded":"2024-05-27T09:27:20.836403+0000","last_fullsized":"2024-05-27T09:27:20.836403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:51.742078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836477+0000","last_change":"2024-05-27T09:27:18.555617+0000","last_active":"2024-05-27T09:27:20.836477+0000","last_peered":"2024-05-27T09:27:20.836477+0000","last_clean":"2024-05-27T09:27:20.836477+0000","last_became_active":"2024-05-27T09:27:18.555280+0000","last_became_peered":"2024-05-27T09:27:18.555280+0000","last_unstale":"2024-05-27T09:27:20.836477+0000","last_undegraded":"2024-05-27T09:27:20.836477+0000","last_fullsized":"2024-05-27T09:27:20.836477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:32:39.043268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837070+0000","last_change":"2024-05-27T09:27:17.548266+0000","last_active":"2024-05-27T09:27:20.837070+0000","last_peered":"2024-05-27T09:27:20.837070+0000","last_clean":"2024-05-27T09:27:20.837070+0000","last_became_active":"2024-05-27T09:27:17.547938+0000","last_became_peered":"2024-05-27T09:27:17.547938+0000","last_unstale":"2024-05-27T09:27:20.837070+0000","last_undegraded":"2024-05-27T09:27:20.837070+0000","last_fullsized":"2024-05-27T09:27:20.837070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835677+0000","last_change":"2024-05-27T09:27:18.563586+0000","last_active":"2024-05-27T09:27:20.835677+0000","last_peered":"2024-05-27T09:27:20.835677+0000","last_clean":"2024-05-27T09:27:20.835677+0000","last_became_active":"2024-05-27T09:27:18.563227+0000","last_became_peered":"2024-05-27T09:27:18.563227+0000","last_unstale":"2024-05-27T09:27:20.835677+0000","last_undegraded":"2024-05-27T09:27:20.835677+0000","last_fullsized":"2024-05-27T09:27:20.835677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:53.222040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.561110+0000","last_change":"2024-05-27T09:27:17.562200+0000","last_active":"2024-05-27T09:27:23.561110+0000","last_peered":"2024-05-27T09:27:23.561110+0000","last_clean":"2024-05-27T09:27:23.561110+0000","last_became_active":"2024-05-27T09:27:17.561857+0000","last_became_peered":"2024-05-27T09:27:17.561857+0000","last_unstale":"2024-05-27T09:27:23.561110+0000","last_undegraded":"2024-05-27T09:27:23.561110+0000","last_fullsized":"2024-05-27T09:27:23.561110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:05.209528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836936+0000","last_change":"2024-05-27T09:27:18.572676+0000","last_active":"2024-05-27T09:27:20.836936+0000","last_peered":"2024-05-27T09:27:20.836936+0000","last_clean":"2024-05-27T09:27:20.836936+0000","last_became_active":"2024-05-27T09:27:18.572339+0000","last_became_peered":"2024-05-27T09:27:18.572339+0000","last_unstale":"2024-05-27T09:27:20.836936+0000","last_undegraded":"2024-05-27T09:27:20.836936+0000","last_fullsized":"2024-05-27T09:27:20.836936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:09:23.795528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837085+0000","last_change":"2024-05-27T09:27:18.575394+0000","last_active":"2024-05-27T09:27:20.837085+0000","last_peered":"2024-05-27T09:27:20.837085+0000","last_clean":"2024-05-27T09:27:20.837085+0000","last_became_active":"2024-05-27T09:27:18.572307+0000","last_became_peered":"2024-05-27T09:27:18.572307+0000","last_unstale":"2024-05-27T09:27:20.837085+0000","last_undegraded":"2024-05-27T09:27:20.837085+0000","last_fullsized":"2024-05-27T09:27:20.837085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:29:05.116582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.555603+0000","last_change":"2024-05-27T09:27:17.570634+0000","last_active":"2024-05-27T09:27:23.555603+0000","last_peered":"2024-05-27T09:27:23.555603+0000","last_clean":"2024-05-27T09:27:23.555603+0000","last_became_active":"2024-05-27T09:27:17.569866+0000","last_became_peered":"2024-05-27T09:27:17.569866+0000","last_unstale":"2024-05-27T09:27:23.555603+0000","last_undegraded":"2024-05-27T09:27:23.555603+0000","last_fullsized":"2024-05-27T09:27:23.555603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:21:38.268535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539980+0000","last_change":"2024-05-27T09:27:18.549339+0000","last_active":"2024-05-27T09:27:19.539980+0000","last_peered":"2024-05-27T09:27:19.539980+0000","last_clean":"2024-05-27T09:27:19.539980+0000","last_became_active":"2024-05-27T09:27:18.549035+0000","last_became_peered":"2024-05-27T09:27:18.549035+0000","last_unstale":"2024-05-27T09:27:19.539980+0000","last_undegraded":"2024-05-27T09:27:19.539980+0000","last_fullsized":"2024-05-27T09:27:19.539980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:17.895211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.551871+0000","last_change":"2024-05-27T09:27:17.568143+0000","last_active":"2024-05-27T09:27:23.551871+0000","last_peered":"2024-05-27T09:27:23.551871+0000","last_clean":"2024-05-27T09:27:23.551871+0000","last_became_active":"2024-05-27T09:27:17.567235+0000","last_became_peered":"2024-05-27T09:27:17.567235+0000","last_unstale":"2024-05-27T09:27:23.551871+0000","last_undegraded":"2024-05-27T09:27:23.551871+0000","last_fullsized":"2024-05-27T09:27:23.551871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.284270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836871+0000","last_change":"2024-05-27T09:27:18.563270+0000","last_active":"2024-05-27T09:27:20.836871+0000","last_peered":"2024-05-27T09:27:20.836871+0000","last_clean":"2024-05-27T09:27:20.836871+0000","last_became_active":"2024-05-27T09:27:18.562919+0000","last_became_peered":"2024-05-27T09:27:18.562919+0000","last_unstale":"2024-05-27T09:27:20.836871+0000","last_undegraded":"2024-05-27T09:27:20.836871+0000","last_fullsized":"2024-05-27T09:27:20.836871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:55:47.799529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:21.370330+0000","last_change":"2024-05-27T09:27:17.551193+0000","last_active":"2024-05-27T09:27:21.370330+0000","last_peered":"2024-05-27T09:27:21.370330+0000","last_clean":"2024-05-27T09:27:21.370330+0000","last_became_active":"2024-05-27T09:27:17.550849+0000","last_became_peered":"2024-05-27T09:27:17.550849+0000","last_unstale":"2024-05-27T09:27:21.370330+0000","last_undegraded":"2024-05-27T09:27:21.370330+0000","last_fullsized":"2024-05-27T09:27:21.370330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:51.742078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836183+0000","last_change":"2024-05-27T09:27:18.579803+0000","last_active":"2024-05-27T09:27:20.836183+0000","last_peered":"2024-05-27T09:27:20.836183+0000","last_clean":"2024-05-27T09:27:20.836183+0000","last_became_active":"2024-05-27T09:27:18.579574+0000","last_became_peered":"2024-05-27T09:27:18.579574+0000","last_unstale":"2024-05-27T09:27:20.836183+0000","last_undegraded":"2024-05-27T09:27:20.836183+0000","last_fullsized":"2024-05-27T09:27:20.836183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:11:22.320778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.560320+0000","last_change":"2024-05-27T09:27:17.549711+0000","last_active":"2024-05-27T09:27:23.560320+0000","last_peered":"2024-05-27T09:27:23.560320+0000","last_clean":"2024-05-27T09:27:23.560320+0000","last_became_active":"2024-05-27T09:27:17.549291+0000","last_became_peered":"2024-05-27T09:27:17.549291+0000","last_unstale":"2024-05-27T09:27:23.560320+0000","last_undegraded":"2024-05-27T09:27:23.560320+0000","last_fullsized":"2024-05-27T09:27:23.560320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:16:22.057803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65078,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":212992,"data_stored":73257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":74287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":663113,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":72731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":72799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":72675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":663135,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":74959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T09:27:31.543 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T09:27:31.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.844+0000 7ff13f200640 1 Processor -- start 2024-05-27T09:27:31.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.844+0000 7ff13f200640 1 -- start start 2024-05-27T09:27:31.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.846+0000 7ff13f200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff140156880 0x7ff14006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:31.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.846+0000 7ff13f200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff140070060 0x7ff14006f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:31.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.846+0000 7ff13f200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1400c8470 con 0x7ff140074a80 2024-05-27T09:27:31.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.846+0000 7ff13f200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7ff14015d480 con 0x7ff140156880 2024-05-27T09:27:31.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.847+0000 7ff13f200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7ff1400c8220 con 0x7ff140070060 2024-05-27T09:27:31.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.847+0000 7ff137e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff140156880 0x7ff14006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:31.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.847+0000 7ff137e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff140156880 0x7ff14006edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35862/0 (socket says 172.21.15.114:35862) 2024-05-27T09:27:31.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.847+0000 7ff137e00640 1 -- 172.21.15.114:0/1617088573 learned_addr learned my addr 172.21.15.114:0/1617088573 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:31.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.847+0000 7ff13e800640 1 --2- 172.21.15.114:0/1617088573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff140070060 0x7ff14006f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:31.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.847+0000 7ff137e00640 1 -- 172.21.15.114:0/1617088573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff140070060 msgr2=0x7ff14006f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:31.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.847+0000 7ff137e00640 1 --2- 172.21.15.114:0/1617088573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff140070060 0x7ff14006f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:31.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.847+0000 7ff137e00640 1 -- 172.21.15.114:0/1617088573 >> v1:172.21.15.114:6789/0 conn(0x7ff140074a80 legacy=0x7ff14015cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:31.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.847+0000 7ff137e00640 1 -- 172.21.15.114:0/1617088573 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1400c8dd0 con 0x7ff140156880 2024-05-27T09:27:31.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.848+0000 7ff137e00640 1 --2- 172.21.15.114:0/1617088573 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff140156880 0x7ff14006edf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff128009870 tx=0x7ff12802a240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990319a1d105b753 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:31.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.848+0000 7ff13d400640 1 -- 172.21.15.114:0/1617088573 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128037070 con 0x7ff140156880 2024-05-27T09:27:31.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.848+0000 7ff13d400640 1 -- 172.21.15.114:0/1617088573 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff128032440 con 0x7ff140156880 2024-05-27T09:27:31.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.848+0000 7ff13d400640 1 -- 172.21.15.114:0/1617088573 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128032810 con 0x7ff140156880 2024-05-27T09:27:31.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.849+0000 7ff13f200640 1 -- 172.21.15.114:0/1617088573 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff140156880 msgr2=0x7ff14006edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:31.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.849+0000 7ff13f200640 1 --2- 172.21.15.114:0/1617088573 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff140156880 0x7ff14006edf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff128009870 tx=0x7ff12802a240 comp rx=0 tx=0).stop 2024-05-27T09:27:31.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.849+0000 7ff13f200640 1 -- 172.21.15.114:0/1617088573 shutdown_connections 2024-05-27T09:27:31.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.849+0000 7ff13f200640 1 --2- 172.21.15.114:0/1617088573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff140070060 0x7ff14006f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:31.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.849+0000 7ff13f200640 1 --2- 172.21.15.114:0/1617088573 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff140156880 0x7ff14006edf0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:31.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.849+0000 7ff13f200640 1 -- 172.21.15.114:0/1617088573 >> 172.21.15.114:0/1617088573 conn(0x7ff1400ca320 msgr2=0x7ff1400ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:31.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.849+0000 7ff13f200640 1 -- 172.21.15.114:0/1617088573 shutdown_connections 2024-05-27T09:27:31.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.849+0000 7ff13f200640 1 -- 172.21.15.114:0/1617088573 wait complete. 2024-05-27T09:27:31.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.851+0000 7ff13f200640 1 Processor -- start 2024-05-27T09:27:31.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.852+0000 7ff13f200640 1 -- start start 2024-05-27T09:27:31.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.853+0000 7ff13f200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff140070060 0x7ff1401f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:31.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.853+0000 7ff13f200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff140074a80 0x7ff14014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:31.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.853+0000 7ff13f200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff140156880 0x7ff140150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:31.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.853+0000 7ff13f200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff14015c8e0 con 0x7ff140070060 2024-05-27T09:27:31.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.853+0000 7ff13f200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff1400c8750 con 0x7ff140074a80 2024-05-27T09:27:31.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.853+0000 7ff13f200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7ff1400ca0c0 con 0x7ff140156880 2024-05-27T09:27:31.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.853+0000 7ff13de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff140070060 0x7ff1401f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:31.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.853+0000 7ff13de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff140070060 0x7ff1401f34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35876/0 (socket says 172.21.15.114:35876) 2024-05-27T09:27:31.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.853+0000 7ff13de00640 1 -- 172.21.15.114:0/3524228973 learned_addr learned my addr 172.21.15.114:0/3524228973 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:31.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.853+0000 7ff13e800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff140156880 0x7ff140150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:31.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.853+0000 7ff137e00640 1 --2- 172.21.15.114:0/3524228973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff140074a80 0x7ff14014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:31.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.854+0000 7ff13de00640 1 -- 172.21.15.114:0/3524228973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff140156880 msgr2=0x7ff140150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:31.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.854+0000 7ff13de00640 1 --2- 172.21.15.114:0/3524228973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff140156880 0x7ff140150b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:31.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.854+0000 7ff13de00640 1 -- 172.21.15.114:0/3524228973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff140074a80 msgr2=0x7ff14014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:31.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.854+0000 7ff13de00640 1 --2- 172.21.15.114:0/3524228973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff140074a80 0x7ff14014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:31.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.854+0000 7ff13de00640 1 -- 172.21.15.114:0/3524228973 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1401f39e0 con 0x7ff140070060 2024-05-27T09:27:31.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.854+0000 7ff13e800640 1 --2- 172.21.15.114:0/3524228973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff140156880 0x7ff140150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:31.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.854+0000 7ff13de00640 1 --2- 172.21.15.114:0/3524228973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff140070060 0x7ff1401f34a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff13000c3d0 tx=0x7ff13000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:31.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.854+0000 7ff136000640 1 -- 172.21.15.114:0/3524228973 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff130017070 con 0x7ff140070060 2024-05-27T09:27:31.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.854+0000 7ff136000640 1 -- 172.21.15.114:0/3524228973 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1300052d0 con 0x7ff140070060 2024-05-27T09:27:31.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.854+0000 7ff13f200640 1 -- 172.21.15.114:0/3524228973 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14015e5c0 con 0x7ff140070060 2024-05-27T09:27:31.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.854+0000 7ff136000640 1 -- 172.21.15.114:0/3524228973 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff130003830 con 0x7ff140070060 2024-05-27T09:27:31.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.854+0000 7ff13f200640 1 -- 172.21.15.114:0/3524228973 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff14015eb80 con 0x7ff140070060 2024-05-27T09:27:31.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.855+0000 7ff13f200640 1 -- 172.21.15.114:0/3524228973 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff140050860 con 0x7ff140070060 2024-05-27T09:27:31.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.856+0000 7ff136000640 1 -- 172.21.15.114:0/3524228973 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff13001a070 con 0x7ff140070060 2024-05-27T09:27:31.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.857+0000 7ff136000640 1 --2- 172.21.15.114:0/3524228973 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff11c0762d0 0x7ff11c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:31.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.857+0000 7ff136000640 1 -- 172.21.15.114:0/3524228973 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff130096880 con 0x7ff140070060 2024-05-27T09:27:31.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.858+0000 7ff137e00640 1 --2- 172.21.15.114:0/3524228973 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff11c0762d0 0x7ff11c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:31.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.858+0000 7ff137e00640 1 --2- 172.21.15.114:0/3524228973 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff11c0762d0 0x7ff11c0787e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff128034000 tx=0x7ff12802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:31.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:31.862+0000 7ff136000640 1 -- 172.21.15.114:0/3524228973 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff13005fb80 con 0x7ff140070060 2024-05-27T09:27:32.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.051+0000 7ff13f200640 1 -- 172.21.15.114:0/3524228973 --> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff1401f2e40 con 0x7ff11c0762d0 2024-05-27T09:27:32.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.060+0000 7ff136000640 1 -- 172.21.15.114:0/3524228973 <== mgr.4108 v2:172.21.15.163:6832/1122018204 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251898 (secure 0 0 0) 0x7ff1401f2e40 con 0x7ff11c0762d0 2024-05-27T09:27:32.064 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:32.066 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-05-27T09:27:32.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.065+0000 7ff13f200640 1 -- 172.21.15.114:0/3524228973 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff11c0762d0 msgr2=0x7ff11c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:32.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.065+0000 7ff13f200640 1 --2- 172.21.15.114:0/3524228973 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff11c0762d0 0x7ff11c0787e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff128034000 tx=0x7ff12802e000 comp rx=0 tx=0).stop 2024-05-27T09:27:32.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.065+0000 7ff13f200640 1 -- 172.21.15.114:0/3524228973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff140070060 msgr2=0x7ff1401f34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:32.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.065+0000 7ff13f200640 1 --2- 172.21.15.114:0/3524228973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff140070060 0x7ff1401f34a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff13000c3d0 tx=0x7ff13000c890 comp rx=0 tx=0).stop 2024-05-27T09:27:32.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.065+0000 7ff13f200640 1 -- 172.21.15.114:0/3524228973 shutdown_connections 2024-05-27T09:27:32.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.065+0000 7ff13f200640 1 --2- 172.21.15.114:0/3524228973 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff11c0762d0 0x7ff11c0787e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:32.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.065+0000 7ff13f200640 1 --2- 172.21.15.114:0/3524228973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff140156880 0x7ff140150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:32.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.065+0000 7ff13f200640 1 --2- 172.21.15.114:0/3524228973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff140074a80 0x7ff14014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:32.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.065+0000 7ff13f200640 1 --2- 172.21.15.114:0/3524228973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff140070060 0x7ff1401f34a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:32.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.065+0000 7ff13f200640 1 -- 172.21.15.114:0/3524228973 >> 172.21.15.114:0/3524228973 conn(0x7ff1400ca320 msgr2=0x7ff14014efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:32.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.065+0000 7ff13f200640 1 -- 172.21.15.114:0/3524228973 shutdown_connections 2024-05-27T09:27:32.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.065+0000 7ff13f200640 1 -- 172.21.15.114:0/3524228973 wait complete. 2024-05-27T09:27:32.092 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-05-27T09:27:30.765502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217960,"kb_used_data":1824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756760,"statfs":{"total":773094113280,"available":772870922240,"internally_reserved":0,"allocated":1867776,"data_stored":1766956,"data_compressed":65078,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.230779"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544950+0000","last_change":"2024-05-27T09:27:18.585655+0000","last_active":"2024-05-27T09:27:19.544950+0000","last_peered":"2024-05-27T09:27:19.544950+0000","last_clean":"2024-05-27T09:27:19.544950+0000","last_became_active":"2024-05-27T09:27:18.585169+0000","last_became_peered":"2024-05-27T09:27:18.585169+0000","last_unstale":"2024-05-27T09:27:19.544950+0000","last_undegraded":"2024-05-27T09:27:19.544950+0000","last_fullsized":"2024-05-27T09:27:19.544950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:39:59.034873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835983+0000","last_change":"2024-05-27T09:27:18.566621+0000","last_active":"2024-05-27T09:27:20.835983+0000","last_peered":"2024-05-27T09:27:20.835983+0000","last_clean":"2024-05-27T09:27:20.835983+0000","last_became_active":"2024-05-27T09:27:18.566269+0000","last_became_peered":"2024-05-27T09:27:18.566269+0000","last_unstale":"2024-05-27T09:27:20.835983+0000","last_undegraded":"2024-05-27T09:27:20.835983+0000","last_fullsized":"2024-05-27T09:27:20.835983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:24:24.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835531+0000","last_change":"2024-05-27T09:27:18.560958+0000","last_active":"2024-05-27T09:27:20.835531+0000","last_peered":"2024-05-27T09:27:20.835531+0000","last_clean":"2024-05-27T09:27:20.835531+0000","last_became_active":"2024-05-27T09:27:18.560620+0000","last_became_peered":"2024-05-27T09:27:18.560620+0000","last_unstale":"2024-05-27T09:27:20.835531+0000","last_undegraded":"2024-05-27T09:27:20.835531+0000","last_fullsized":"2024-05-27T09:27:20.835531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:42:38.351682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542664+0000","last_change":"2024-05-27T09:27:18.569804+0000","last_active":"2024-05-27T09:27:19.542664+0000","last_peered":"2024-05-27T09:27:19.542664+0000","last_clean":"2024-05-27T09:27:19.542664+0000","last_became_active":"2024-05-27T09:27:18.569481+0000","last_became_peered":"2024-05-27T09:27:18.569481+0000","last_unstale":"2024-05-27T09:27:19.542664+0000","last_undegraded":"2024-05-27T09:27:19.542664+0000","last_fullsized":"2024-05-27T09:27:19.542664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:55:03.583659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543808+0000","last_change":"2024-05-27T09:27:18.568138+0000","last_active":"2024-05-27T09:27:19.543808+0000","last_peered":"2024-05-27T09:27:19.543808+0000","last_clean":"2024-05-27T09:27:19.543808+0000","last_became_active":"2024-05-27T09:27:18.567795+0000","last_became_peered":"2024-05-27T09:27:18.567795+0000","last_unstale":"2024-05-27T09:27:19.543808+0000","last_undegraded":"2024-05-27T09:27:19.543808+0000","last_fullsized":"2024-05-27T09:27:19.543808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:17:42.005593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.540719+0000","last_change":"2024-05-27T09:27:18.573984+0000","last_active":"2024-05-27T09:27:19.540719+0000","last_peered":"2024-05-27T09:27:19.540719+0000","last_clean":"2024-05-27T09:27:19.540719+0000","last_became_active":"2024-05-27T09:27:18.573576+0000","last_became_peered":"2024-05-27T09:27:18.573576+0000","last_unstale":"2024-05-27T09:27:19.540719+0000","last_undegraded":"2024-05-27T09:27:19.540719+0000","last_fullsized":"2024-05-27T09:27:19.540719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:06.224637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543010+0000","last_change":"2024-05-27T09:27:18.543122+0000","last_active":"2024-05-27T09:27:19.543010+0000","last_peered":"2024-05-27T09:27:19.543010+0000","last_clean":"2024-05-27T09:27:19.543010+0000","last_became_active":"2024-05-27T09:27:18.542593+0000","last_became_peered":"2024-05-27T09:27:18.542593+0000","last_unstale":"2024-05-27T09:27:19.543010+0000","last_undegraded":"2024-05-27T09:27:19.543010+0000","last_fullsized":"2024-05-27T09:27:19.543010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:28:19.579432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835946+0000","last_change":"2024-05-27T09:27:18.562292+0000","last_active":"2024-05-27T09:27:20.835946+0000","last_peered":"2024-05-27T09:27:20.835946+0000","last_clean":"2024-05-27T09:27:20.835946+0000","last_became_active":"2024-05-27T09:27:18.561788+0000","last_became_peered":"2024-05-27T09:27:18.561788+0000","last_unstale":"2024-05-27T09:27:20.835946+0000","last_undegraded":"2024-05-27T09:27:20.835946+0000","last_fullsized":"2024-05-27T09:27:20.835946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:32:39.043268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542996+0000","last_change":"2024-05-27T09:27:18.583770+0000","last_active":"2024-05-27T09:27:19.542996+0000","last_peered":"2024-05-27T09:27:19.542996+0000","last_clean":"2024-05-27T09:27:19.542996+0000","last_became_active":"2024-05-27T09:27:18.582041+0000","last_became_peered":"2024-05-27T09:27:18.582041+0000","last_unstale":"2024-05-27T09:27:19.542996+0000","last_undegraded":"2024-05-27T09:27:19.542996+0000","last_fullsized":"2024-05-27T09:27:19.542996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:19.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.545125+0000","last_change":"2024-05-27T09:27:18.574658+0000","last_active":"2024-05-27T09:27:19.545125+0000","last_peered":"2024-05-27T09:27:19.545125+0000","last_clean":"2024-05-27T09:27:19.545125+0000","last_became_active":"2024-05-27T09:27:18.574409+0000","last_became_peered":"2024-05-27T09:27:18.574409+0000","last_unstale":"2024-05-27T09:27:19.545125+0000","last_undegraded":"2024-05-27T09:27:19.545125+0000","last_fullsized":"2024-05-27T09:27:19.545125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:19.131343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544558+0000","last_change":"2024-05-27T09:27:18.579141+0000","last_active":"2024-05-27T09:27:19.544558+0000","last_peered":"2024-05-27T09:27:19.544558+0000","last_clean":"2024-05-27T09:27:19.544558+0000","last_became_active":"2024-05-27T09:27:18.578679+0000","last_became_peered":"2024-05-27T09:27:18.578679+0000","last_unstale":"2024-05-27T09:27:19.544558+0000","last_undegraded":"2024-05-27T09:27:19.544558+0000","last_fullsized":"2024-05-27T09:27:19.544558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:44.336453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544110+0000","last_change":"2024-05-27T09:27:18.547375+0000","last_active":"2024-05-27T09:27:19.544110+0000","last_peered":"2024-05-27T09:27:19.544110+0000","last_clean":"2024-05-27T09:27:19.544110+0000","last_became_active":"2024-05-27T09:27:18.544328+0000","last_became_peered":"2024-05-27T09:27:18.544328+0000","last_unstale":"2024-05-27T09:27:19.544110+0000","last_undegraded":"2024-05-27T09:27:19.544110+0000","last_fullsized":"2024-05-27T09:27:19.544110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:59:50.499894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836590+0000","last_change":"2024-05-27T09:27:18.561903+0000","last_active":"2024-05-27T09:27:20.836590+0000","last_peered":"2024-05-27T09:27:20.836590+0000","last_clean":"2024-05-27T09:27:20.836590+0000","last_became_active":"2024-05-27T09:27:18.561493+0000","last_became_peered":"2024-05-27T09:27:18.561493+0000","last_unstale":"2024-05-27T09:27:20.836590+0000","last_undegraded":"2024-05-27T09:27:20.836590+0000","last_fullsized":"2024-05-27T09:27:20.836590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:28:31.470269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836378+0000","last_change":"2024-05-27T09:27:18.563168+0000","last_active":"2024-05-27T09:27:20.836378+0000","last_peered":"2024-05-27T09:27:20.836378+0000","last_clean":"2024-05-27T09:27:20.836378+0000","last_became_active":"2024-05-27T09:27:18.562843+0000","last_became_peered":"2024-05-27T09:27:18.562843+0000","last_unstale":"2024-05-27T09:27:20.836378+0000","last_undegraded":"2024-05-27T09:27:20.836378+0000","last_fullsized":"2024-05-27T09:27:20.836378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:33:36.671764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835457+0000","last_change":"2024-05-27T09:27:18.566502+0000","last_active":"2024-05-27T09:27:20.835457+0000","last_peered":"2024-05-27T09:27:20.835457+0000","last_clean":"2024-05-27T09:27:20.835457+0000","last_became_active":"2024-05-27T09:27:18.566108+0000","last_became_peered":"2024-05-27T09:27:18.566108+0000","last_unstale":"2024-05-27T09:27:20.835457+0000","last_undegraded":"2024-05-27T09:27:20.835457+0000","last_fullsized":"2024-05-27T09:27:20.835457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:44:47.447245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836471+0000","last_change":"2024-05-27T09:27:18.582595+0000","last_active":"2024-05-27T09:27:20.836471+0000","last_peered":"2024-05-27T09:27:20.836471+0000","last_clean":"2024-05-27T09:27:20.836471+0000","last_became_active":"2024-05-27T09:27:18.581719+0000","last_became_peered":"2024-05-27T09:27:18.581719+0000","last_unstale":"2024-05-27T09:27:20.836471+0000","last_undegraded":"2024-05-27T09:27:20.836471+0000","last_fullsized":"2024-05-27T09:27:20.836471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:48:19.573654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837234+0000","last_change":"2024-05-27T09:27:18.577565+0000","last_active":"2024-05-27T09:27:20.837234+0000","last_peered":"2024-05-27T09:27:20.837234+0000","last_clean":"2024-05-27T09:27:20.837234+0000","last_became_active":"2024-05-27T09:27:18.577193+0000","last_became_peered":"2024-05-27T09:27:18.577193+0000","last_unstale":"2024-05-27T09:27:20.837234+0000","last_undegraded":"2024-05-27T09:27:20.837234+0000","last_fullsized":"2024-05-27T09:27:20.837234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:43:20.065873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835530+0000","last_change":"2024-05-27T09:27:18.571089+0000","last_active":"2024-05-27T09:27:20.835530+0000","last_peered":"2024-05-27T09:27:20.835530+0000","last_clean":"2024-05-27T09:27:20.835530+0000","last_became_active":"2024-05-27T09:27:18.570712+0000","last_became_peered":"2024-05-27T09:27:18.570712+0000","last_unstale":"2024-05-27T09:27:20.835530+0000","last_undegraded":"2024-05-27T09:27:20.835530+0000","last_fullsized":"2024-05-27T09:27:20.835530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:40:19.310581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837956+0000","last_change":"2024-05-27T09:27:18.572675+0000","last_active":"2024-05-27T09:27:20.837956+0000","last_peered":"2024-05-27T09:27:20.837956+0000","last_clean":"2024-05-27T09:27:20.837956+0000","last_became_active":"2024-05-27T09:27:18.564643+0000","last_became_peered":"2024-05-27T09:27:18.564643+0000","last_unstale":"2024-05-27T09:27:20.837956+0000","last_undegraded":"2024-05-27T09:27:20.837956+0000","last_fullsized":"2024-05-27T09:27:20.837956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:36:15.602241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544179+0000","last_change":"2024-05-27T09:27:18.544554+0000","last_active":"2024-05-27T09:27:19.544179+0000","last_peered":"2024-05-27T09:27:19.544179+0000","last_clean":"2024-05-27T09:27:19.544179+0000","last_became_active":"2024-05-27T09:27:18.543777+0000","last_became_peered":"2024-05-27T09:27:18.543777+0000","last_unstale":"2024-05-27T09:27:19.544179+0000","last_undegraded":"2024-05-27T09:27:19.544179+0000","last_fullsized":"2024-05-27T09:27:19.544179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:35:43.354270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835033+0000","last_change":"2024-05-27T09:27:18.571321+0000","last_active":"2024-05-27T09:27:20.835033+0000","last_peered":"2024-05-27T09:27:20.835033+0000","last_clean":"2024-05-27T09:27:20.835033+0000","last_became_active":"2024-05-27T09:27:18.570948+0000","last_became_peered":"2024-05-27T09:27:18.570948+0000","last_unstale":"2024-05-27T09:27:20.835033+0000","last_undegraded":"2024-05-27T09:27:20.835033+0000","last_fullsized":"2024-05-27T09:27:20.835033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:28:31.470269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539490+0000","last_change":"2024-05-27T09:27:18.544282+0000","last_active":"2024-05-27T09:27:19.539490+0000","last_peered":"2024-05-27T09:27:19.539490+0000","last_clean":"2024-05-27T09:27:19.539490+0000","last_became_active":"2024-05-27T09:27:18.543921+0000","last_became_peered":"2024-05-27T09:27:18.543921+0000","last_unstale":"2024-05-27T09:27:19.539490+0000","last_undegraded":"2024-05-27T09:27:19.539490+0000","last_fullsized":"2024-05-27T09:27:19.539490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:27.225150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835732+0000","last_change":"2024-05-27T09:27:18.575415+0000","last_active":"2024-05-27T09:27:20.835732+0000","last_peered":"2024-05-27T09:27:20.835732+0000","last_clean":"2024-05-27T09:27:20.835732+0000","last_became_active":"2024-05-27T09:27:18.575115+0000","last_became_peered":"2024-05-27T09:27:18.575115+0000","last_unstale":"2024-05-27T09:27:20.835732+0000","last_undegraded":"2024-05-27T09:27:20.835732+0000","last_fullsized":"2024-05-27T09:27:20.835732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:50:48.091125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836496+0000","last_change":"2024-05-27T09:27:18.564422+0000","last_active":"2024-05-27T09:27:20.836496+0000","last_peered":"2024-05-27T09:27:20.836496+0000","last_clean":"2024-05-27T09:27:20.836496+0000","last_became_active":"2024-05-27T09:27:18.563991+0000","last_became_peered":"2024-05-27T09:27:18.563991+0000","last_unstale":"2024-05-27T09:27:20.836496+0000","last_undegraded":"2024-05-27T09:27:20.836496+0000","last_fullsized":"2024-05-27T09:27:20.836496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:19.131343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836032+0000","last_change":"2024-05-27T09:27:18.563557+0000","last_active":"2024-05-27T09:27:20.836032+0000","last_peered":"2024-05-27T09:27:20.836032+0000","last_clean":"2024-05-27T09:27:20.836032+0000","last_became_active":"2024-05-27T09:27:18.562803+0000","last_became_peered":"2024-05-27T09:27:18.562803+0000","last_unstale":"2024-05-27T09:27:20.836032+0000","last_undegraded":"2024-05-27T09:27:20.836032+0000","last_fullsized":"2024-05-27T09:27:20.836032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:47:54.271208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542223+0000","last_change":"2024-05-27T09:27:18.590430+0000","last_active":"2024-05-27T09:27:19.542223+0000","last_peered":"2024-05-27T09:27:19.542223+0000","last_clean":"2024-05-27T09:27:19.542223+0000","last_became_active":"2024-05-27T09:27:18.590083+0000","last_became_peered":"2024-05-27T09:27:18.590083+0000","last_unstale":"2024-05-27T09:27:19.542223+0000","last_undegraded":"2024-05-27T09:27:19.542223+0000","last_fullsized":"2024-05-27T09:27:19.542223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:03.762818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837167+0000","last_change":"2024-05-27T09:27:18.558248+0000","last_active":"2024-05-27T09:27:20.837167+0000","last_peered":"2024-05-27T09:27:20.837167+0000","last_clean":"2024-05-27T09:27:20.837167+0000","last_became_active":"2024-05-27T09:27:18.557953+0000","last_became_peered":"2024-05-27T09:27:18.557953+0000","last_unstale":"2024-05-27T09:27:20.837167+0000","last_undegraded":"2024-05-27T09:27:20.837167+0000","last_fullsized":"2024-05-27T09:27:20.837167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:44:47.447245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836514+0000","last_change":"2024-05-27T09:27:18.563213+0000","last_active":"2024-05-27T09:27:20.836514+0000","last_peered":"2024-05-27T09:27:20.836514+0000","last_clean":"2024-05-27T09:27:20.836514+0000","last_became_active":"2024-05-27T09:27:18.562839+0000","last_became_peered":"2024-05-27T09:27:18.562839+0000","last_unstale":"2024-05-27T09:27:20.836514+0000","last_undegraded":"2024-05-27T09:27:20.836514+0000","last_fullsized":"2024-05-27T09:27:20.836514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:43.851698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836053+0000","last_change":"2024-05-27T09:27:18.562237+0000","last_active":"2024-05-27T09:27:20.836053+0000","last_peered":"2024-05-27T09:27:20.836053+0000","last_clean":"2024-05-27T09:27:20.836053+0000","last_became_active":"2024-05-27T09:27:18.561475+0000","last_became_peered":"2024-05-27T09:27:18.561475+0000","last_unstale":"2024-05-27T09:27:20.836053+0000","last_undegraded":"2024-05-27T09:27:20.836053+0000","last_fullsized":"2024-05-27T09:27:20.836053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:39:58.110527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837611+0000","last_change":"2024-05-27T09:27:18.563469+0000","last_active":"2024-05-27T09:27:20.837611+0000","last_peered":"2024-05-27T09:27:20.837611+0000","last_clean":"2024-05-27T09:27:20.837611+0000","last_became_active":"2024-05-27T09:27:18.563103+0000","last_became_peered":"2024-05-27T09:27:18.563103+0000","last_unstale":"2024-05-27T09:27:20.837611+0000","last_undegraded":"2024-05-27T09:27:20.837611+0000","last_fullsized":"2024-05-27T09:27:20.837611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:53:57.281169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.834984+0000","last_change":"2024-05-27T09:27:18.560551+0000","last_active":"2024-05-27T09:27:20.834984+0000","last_peered":"2024-05-27T09:27:20.834984+0000","last_clean":"2024-05-27T09:27:20.834984+0000","last_became_active":"2024-05-27T09:27:18.560244+0000","last_became_peered":"2024-05-27T09:27:18.560244+0000","last_unstale":"2024-05-27T09:27:20.834984+0000","last_undegraded":"2024-05-27T09:27:20.834984+0000","last_fullsized":"2024-05-27T09:27:20.834984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:00.131180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.538536+0000","last_change":"2024-05-27T09:27:18.568200+0000","last_active":"2024-05-27T09:27:19.538536+0000","last_peered":"2024-05-27T09:27:19.538536+0000","last_clean":"2024-05-27T09:27:19.538536+0000","last_became_active":"2024-05-27T09:27:18.567845+0000","last_became_peered":"2024-05-27T09:27:18.567845+0000","last_unstale":"2024-05-27T09:27:19.538536+0000","last_undegraded":"2024-05-27T09:27:19.538536+0000","last_fullsized":"2024-05-27T09:27:19.538536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:54.726617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.540019+0000","last_change":"2024-05-27T09:27:18.560521+0000","last_active":"2024-05-27T09:27:19.540019+0000","last_peered":"2024-05-27T09:27:19.540019+0000","last_clean":"2024-05-27T09:27:19.540019+0000","last_became_active":"2024-05-27T09:27:18.556156+0000","last_became_peered":"2024-05-27T09:27:18.556156+0000","last_unstale":"2024-05-27T09:27:19.540019+0000","last_undegraded":"2024-05-27T09:27:19.540019+0000","last_fullsized":"2024-05-27T09:27:19.540019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:03.762818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:21.370264+0000","last_change":"2024-05-27T09:27:17.548038+0000","last_active":"2024-05-27T09:27:21.370264+0000","last_peered":"2024-05-27T09:27:21.370264+0000","last_clean":"2024-05-27T09:27:21.370264+0000","last_became_active":"2024-05-27T09:27:17.547653+0000","last_became_peered":"2024-05-27T09:27:17.547653+0000","last_unstale":"2024-05-27T09:27:21.370264+0000","last_undegraded":"2024-05-27T09:27:21.370264+0000","last_fullsized":"2024-05-27T09:27:21.370264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835265+0000","last_change":"2024-05-27T09:27:18.543048+0000","last_active":"2024-05-27T09:27:20.835265+0000","last_peered":"2024-05-27T09:27:20.835265+0000","last_clean":"2024-05-27T09:27:20.835265+0000","last_became_active":"2024-05-27T09:27:18.542715+0000","last_became_peered":"2024-05-27T09:27:18.542715+0000","last_unstale":"2024-05-27T09:27:20.835265+0000","last_undegraded":"2024-05-27T09:27:20.835265+0000","last_fullsized":"2024-05-27T09:27:20.835265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:46:49.444709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837344+0000","last_change":"2024-05-27T09:27:17.570014+0000","last_active":"2024-05-27T09:27:20.837344+0000","last_peered":"2024-05-27T09:27:20.837344+0000","last_clean":"2024-05-27T09:27:20.837344+0000","last_became_active":"2024-05-27T09:27:17.569673+0000","last_became_peered":"2024-05-27T09:27:17.569673+0000","last_unstale":"2024-05-27T09:27:20.837344+0000","last_undegraded":"2024-05-27T09:27:20.837344+0000","last_fullsized":"2024-05-27T09:27:20.837344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:26.210041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836496+0000","last_change":"2024-05-27T09:27:17.565489+0000","last_active":"2024-05-27T09:27:20.836496+0000","last_peered":"2024-05-27T09:27:20.836496+0000","last_clean":"2024-05-27T09:27:20.836496+0000","last_became_active":"2024-05-27T09:27:17.565137+0000","last_became_peered":"2024-05-27T09:27:17.565137+0000","last_unstale":"2024-05-27T09:27:20.836496+0000","last_undegraded":"2024-05-27T09:27:20.836496+0000","last_fullsized":"2024-05-27T09:27:20.836496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836835+0000","last_change":"2024-05-27T09:27:18.565294+0000","last_active":"2024-05-27T09:27:20.836835+0000","last_peered":"2024-05-27T09:27:20.836835+0000","last_clean":"2024-05-27T09:27:20.836835+0000","last_became_active":"2024-05-27T09:27:18.564115+0000","last_became_peered":"2024-05-27T09:27:18.564115+0000","last_unstale":"2024-05-27T09:27:20.836835+0000","last_undegraded":"2024-05-27T09:27:20.836835+0000","last_fullsized":"2024-05-27T09:27:20.836835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:57:20.488849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836662+0000","last_change":"2024-05-27T09:27:17.561204+0000","last_active":"2024-05-27T09:27:20.836662+0000","last_peered":"2024-05-27T09:27:20.836662+0000","last_clean":"2024-05-27T09:27:20.836662+0000","last_became_active":"2024-05-27T09:27:17.560149+0000","last_became_peered":"2024-05-27T09:27:17.560149+0000","last_unstale":"2024-05-27T09:27:20.836662+0000","last_undegraded":"2024-05-27T09:27:20.836662+0000","last_fullsized":"2024-05-27T09:27:20.836662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.284270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836343+0000","last_change":"2024-05-27T09:27:18.542204+0000","last_active":"2024-05-27T09:27:20.836343+0000","last_peered":"2024-05-27T09:27:20.836343+0000","last_clean":"2024-05-27T09:27:20.836343+0000","last_became_active":"2024-05-27T09:27:18.541946+0000","last_became_peered":"2024-05-27T09:27:18.541946+0000","last_unstale":"2024-05-27T09:27:20.836343+0000","last_undegraded":"2024-05-27T09:27:20.836343+0000","last_fullsized":"2024-05-27T09:27:20.836343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:50:48.091125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.541754+0000","last_change":"2024-05-27T09:27:17.561714+0000","last_active":"2024-05-27T09:27:19.541754+0000","last_peered":"2024-05-27T09:27:19.541754+0000","last_clean":"2024-05-27T09:27:19.541754+0000","last_became_active":"2024-05-27T09:27:17.561355+0000","last_became_peered":"2024-05-27T09:27:17.561355+0000","last_unstale":"2024-05-27T09:27:19.541754+0000","last_undegraded":"2024-05-27T09:27:19.541754+0000","last_fullsized":"2024-05-27T09:27:19.541754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:07:02.481451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542128+0000","last_change":"2024-05-27T09:27:18.588199+0000","last_active":"2024-05-27T09:27:19.542128+0000","last_peered":"2024-05-27T09:27:19.542128+0000","last_clean":"2024-05-27T09:27:19.542128+0000","last_became_active":"2024-05-27T09:27:18.587921+0000","last_became_peered":"2024-05-27T09:27:18.587921+0000","last_unstale":"2024-05-27T09:27:19.542128+0000","last_undegraded":"2024-05-27T09:27:19.542128+0000","last_fullsized":"2024-05-27T09:27:19.542128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:17.895211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544807+0000","last_change":"2024-05-27T09:27:17.565854+0000","last_active":"2024-05-27T09:27:19.544807+0000","last_peered":"2024-05-27T09:27:19.544807+0000","last_clean":"2024-05-27T09:27:19.544807+0000","last_became_active":"2024-05-27T09:27:17.565488+0000","last_became_peered":"2024-05-27T09:27:17.565488+0000","last_unstale":"2024-05-27T09:27:19.544807+0000","last_undegraded":"2024-05-27T09:27:19.544807+0000","last_fullsized":"2024-05-27T09:27:19.544807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:53.711508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836966+0000","last_change":"2024-05-27T09:27:18.585592+0000","last_active":"2024-05-27T09:27:20.836966+0000","last_peered":"2024-05-27T09:27:20.836966+0000","last_clean":"2024-05-27T09:27:20.836966+0000","last_became_active":"2024-05-27T09:27:18.585243+0000","last_became_peered":"2024-05-27T09:27:18.585243+0000","last_unstale":"2024-05-27T09:27:20.836966+0000","last_undegraded":"2024-05-27T09:27:20.836966+0000","last_fullsized":"2024-05-27T09:27:20.836966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:28.872653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.547985+0000","last_change":"2024-05-27T09:27:17.548651+0000","last_active":"2024-05-27T09:27:23.547985+0000","last_peered":"2024-05-27T09:27:23.547985+0000","last_clean":"2024-05-27T09:27:23.547985+0000","last_became_active":"2024-05-27T09:27:17.548307+0000","last_became_peered":"2024-05-27T09:27:17.548307+0000","last_unstale":"2024-05-27T09:27:23.547985+0000","last_undegraded":"2024-05-27T09:27:23.547985+0000","last_fullsized":"2024-05-27T09:27:23.547985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:48.905422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837273+0000","last_change":"2024-05-27T09:27:18.582895+0000","last_active":"2024-05-27T09:27:20.837273+0000","last_peered":"2024-05-27T09:27:20.837273+0000","last_clean":"2024-05-27T09:27:20.837273+0000","last_became_active":"2024-05-27T09:27:18.581742+0000","last_became_peered":"2024-05-27T09:27:18.581742+0000","last_unstale":"2024-05-27T09:27:20.837273+0000","last_undegraded":"2024-05-27T09:27:20.837273+0000","last_fullsized":"2024-05-27T09:27:20.837273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:23:26.888905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.560732+0000","last_change":"2024-05-27T09:27:17.556419+0000","last_active":"2024-05-27T09:27:23.560732+0000","last_peered":"2024-05-27T09:27:23.560732+0000","last_clean":"2024-05-27T09:27:23.560732+0000","last_became_active":"2024-05-27T09:27:17.556036+0000","last_became_peered":"2024-05-27T09:27:17.556036+0000","last_unstale":"2024-05-27T09:27:23.560732+0000","last_undegraded":"2024-05-27T09:27:23.560732+0000","last_fullsized":"2024-05-27T09:27:23.560732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:48:20.718085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544281+0000","last_change":"2024-05-27T09:27:18.568187+0000","last_active":"2024-05-27T09:27:19.544281+0000","last_peered":"2024-05-27T09:27:19.544281+0000","last_clean":"2024-05-27T09:27:19.544281+0000","last_became_active":"2024-05-27T09:27:18.565500+0000","last_became_peered":"2024-05-27T09:27:18.565500+0000","last_unstale":"2024-05-27T09:27:19.544281+0000","last_undegraded":"2024-05-27T09:27:19.544281+0000","last_fullsized":"2024-05-27T09:27:19.544281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:16:08.688048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.550202+0000","last_change":"2024-05-27T09:27:17.572168+0000","last_active":"2024-05-27T09:27:23.550202+0000","last_peered":"2024-05-27T09:27:23.550202+0000","last_clean":"2024-05-27T09:27:23.550202+0000","last_became_active":"2024-05-27T09:27:17.571709+0000","last_became_peered":"2024-05-27T09:27:17.571709+0000","last_unstale":"2024-05-27T09:27:23.550202+0000","last_undegraded":"2024-05-27T09:27:23.550202+0000","last_fullsized":"2024-05-27T09:27:23.550202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836574+0000","last_change":"2024-05-27T09:27:18.564930+0000","last_active":"2024-05-27T09:27:20.836574+0000","last_peered":"2024-05-27T09:27:20.836574+0000","last_clean":"2024-05-27T09:27:20.836574+0000","last_became_active":"2024-05-27T09:27:18.564589+0000","last_became_peered":"2024-05-27T09:27:18.564589+0000","last_unstale":"2024-05-27T09:27:20.836574+0000","last_undegraded":"2024-05-27T09:27:20.836574+0000","last_fullsized":"2024-05-27T09:27:20.836574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:21:00.644551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837541+0000","last_change":"2024-05-27T09:27:17.561344+0000","last_active":"2024-05-27T09:27:20.837541+0000","last_peered":"2024-05-27T09:27:20.837541+0000","last_clean":"2024-05-27T09:27:20.837541+0000","last_became_active":"2024-05-27T09:27:17.554860+0000","last_became_peered":"2024-05-27T09:27:17.554860+0000","last_unstale":"2024-05-27T09:27:20.837541+0000","last_undegraded":"2024-05-27T09:27:20.837541+0000","last_fullsized":"2024-05-27T09:27:20.837541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:16.880101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837236+0000","last_change":"2024-05-27T09:27:18.575317+0000","last_active":"2024-05-27T09:27:20.837236+0000","last_peered":"2024-05-27T09:27:20.837236+0000","last_clean":"2024-05-27T09:27:20.837236+0000","last_became_active":"2024-05-27T09:27:18.572024+0000","last_became_peered":"2024-05-27T09:27:18.572024+0000","last_unstale":"2024-05-27T09:27:20.837236+0000","last_undegraded":"2024-05-27T09:27:20.837236+0000","last_fullsized":"2024-05-27T09:27:20.837236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:23:35.418697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543181+0000","last_change":"2024-05-27T09:27:17.575859+0000","last_active":"2024-05-27T09:27:19.543181+0000","last_peered":"2024-05-27T09:27:19.543181+0000","last_clean":"2024-05-27T09:27:19.543181+0000","last_became_active":"2024-05-27T09:27:17.574695+0000","last_became_peered":"2024-05-27T09:27:17.574695+0000","last_unstale":"2024-05-27T09:27:19.543181+0000","last_undegraded":"2024-05-27T09:27:19.543181+0000","last_fullsized":"2024-05-27T09:27:19.543181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:24:19.379819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539060+0000","last_change":"2024-05-27T09:27:18.555137+0000","last_active":"2024-05-27T09:27:19.539060+0000","last_peered":"2024-05-27T09:27:19.539060+0000","last_clean":"2024-05-27T09:27:19.539060+0000","last_became_active":"2024-05-27T09:27:18.554932+0000","last_became_peered":"2024-05-27T09:27:18.554932+0000","last_unstale":"2024-05-27T09:27:19.539060+0000","last_undegraded":"2024-05-27T09:27:19.539060+0000","last_fullsized":"2024-05-27T09:27:19.539060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:06.224637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.556322+0000","last_change":"2024-05-27T09:27:17.547895+0000","last_active":"2024-05-27T09:27:23.556322+0000","last_peered":"2024-05-27T09:27:23.556322+0000","last_clean":"2024-05-27T09:27:23.556322+0000","last_became_active":"2024-05-27T09:27:17.547523+0000","last_became_peered":"2024-05-27T09:27:17.547523+0000","last_unstale":"2024-05-27T09:27:23.556322+0000","last_undegraded":"2024-05-27T09:27:23.556322+0000","last_fullsized":"2024-05-27T09:27:23.556322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836706+0000","last_change":"2024-05-27T09:27:18.584284+0000","last_active":"2024-05-27T09:27:20.836706+0000","last_peered":"2024-05-27T09:27:20.836706+0000","last_clean":"2024-05-27T09:27:20.836706+0000","last_became_active":"2024-05-27T09:27:18.582360+0000","last_became_peered":"2024-05-27T09:27:18.582360+0000","last_unstale":"2024-05-27T09:27:20.836706+0000","last_undegraded":"2024-05-27T09:27:20.836706+0000","last_fullsized":"2024-05-27T09:27:20.836706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:32.535070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.545397+0000","last_change":"2024-05-27T09:27:17.571481+0000","last_active":"2024-05-27T09:27:19.545397+0000","last_peered":"2024-05-27T09:27:19.545397+0000","last_clean":"2024-05-27T09:27:19.545397+0000","last_became_active":"2024-05-27T09:27:17.571146+0000","last_became_peered":"2024-05-27T09:27:17.571146+0000","last_unstale":"2024-05-27T09:27:19.545397+0000","last_undegraded":"2024-05-27T09:27:19.545397+0000","last_fullsized":"2024-05-27T09:27:19.545397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:18.038221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542481+0000","last_change":"2024-05-27T09:27:18.580425+0000","last_active":"2024-05-27T09:27:19.542481+0000","last_peered":"2024-05-27T09:27:19.542481+0000","last_clean":"2024-05-27T09:27:19.542481+0000","last_became_active":"2024-05-27T09:27:18.579440+0000","last_became_peered":"2024-05-27T09:27:18.579440+0000","last_unstale":"2024-05-27T09:27:19.542481+0000","last_undegraded":"2024-05-27T09:27:19.542481+0000","last_fullsized":"2024-05-27T09:27:19.542481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:19.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.554242+0000","last_change":"2024-05-27T09:27:17.551295+0000","last_active":"2024-05-27T09:27:23.554242+0000","last_peered":"2024-05-27T09:27:23.554242+0000","last_clean":"2024-05-27T09:27:23.554242+0000","last_became_active":"2024-05-27T09:27:17.550861+0000","last_became_peered":"2024-05-27T09:27:17.550861+0000","last_unstale":"2024-05-27T09:27:23.554242+0000","last_undegraded":"2024-05-27T09:27:23.554242+0000","last_fullsized":"2024-05-27T09:27:23.554242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:13.955191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.545103+0000","last_change":"2024-05-27T09:27:18.547199+0000","last_active":"2024-05-27T09:27:19.545103+0000","last_peered":"2024-05-27T09:27:19.545103+0000","last_clean":"2024-05-27T09:27:19.545103+0000","last_became_active":"2024-05-27T09:27:18.543683+0000","last_became_peered":"2024-05-27T09:27:18.543683+0000","last_unstale":"2024-05-27T09:27:19.545103+0000","last_undegraded":"2024-05-27T09:27:19.545103+0000","last_fullsized":"2024-05-27T09:27:19.545103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:48:19.573654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:21.370295+0000","last_change":"2024-05-27T09:27:17.559255+0000","last_active":"2024-05-27T09:27:21.370295+0000","last_peered":"2024-05-27T09:27:21.370295+0000","last_clean":"2024-05-27T09:27:21.370295+0000","last_became_active":"2024-05-27T09:27:17.558973+0000","last_became_peered":"2024-05-27T09:27:17.558973+0000","last_unstale":"2024-05-27T09:27:21.370295+0000","last_undegraded":"2024-05-27T09:27:21.370295+0000","last_fullsized":"2024-05-27T09:27:21.370295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544602+0000","last_change":"2024-05-27T09:27:18.557423+0000","last_active":"2024-05-27T09:27:19.544602+0000","last_peered":"2024-05-27T09:27:19.544602+0000","last_clean":"2024-05-27T09:27:19.544602+0000","last_became_active":"2024-05-27T09:27:18.557076+0000","last_became_peered":"2024-05-27T09:27:18.557076+0000","last_unstale":"2024-05-27T09:27:19.544602+0000","last_undegraded":"2024-05-27T09:27:19.544602+0000","last_fullsized":"2024-05-27T09:27:19.544602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:11:04.612363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836934+0000","last_change":"2024-05-27T09:27:17.573700+0000","last_active":"2024-05-27T09:27:20.836934+0000","last_peered":"2024-05-27T09:27:20.836934+0000","last_clean":"2024-05-27T09:27:20.836934+0000","last_became_active":"2024-05-27T09:27:17.573484+0000","last_became_peered":"2024-05-27T09:27:17.573484+0000","last_unstale":"2024-05-27T09:27:20.836934+0000","last_undegraded":"2024-05-27T09:27:20.836934+0000","last_fullsized":"2024-05-27T09:27:20.836934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:48.905422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837609+0000","last_change":"2024-05-27T09:27:18.582777+0000","last_active":"2024-05-27T09:27:20.837609+0000","last_peered":"2024-05-27T09:27:20.837609+0000","last_clean":"2024-05-27T09:27:20.837609+0000","last_became_active":"2024-05-27T09:27:18.574959+0000","last_became_peered":"2024-05-27T09:27:18.574959+0000","last_unstale":"2024-05-27T09:27:20.837609+0000","last_undegraded":"2024-05-27T09:27:20.837609+0000","last_fullsized":"2024-05-27T09:27:20.837609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:40:41.206755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":110,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539210+0000","last_change":"2024-05-27T09:27:12.693423+0000","last_active":"2024-05-27T09:27:19.539210+0000","last_peered":"2024-05-27T09:27:19.539210+0000","last_clean":"2024-05-27T09:27:19.539210+0000","last_became_active":"2024-05-27T09:27:12.690301+0000","last_became_peered":"2024-05-27T09:27:12.690301+0000","last_unstale":"2024-05-27T09:27:19.539210+0000","last_undegraded":"2024-05-27T09:27:19.539210+0000","last_fullsized":"2024-05-27T09:27:19.539210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:11.463226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:11.463226+0000","last_clean_scrub_stamp":"2024-05-27T09:27:11.463226+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:01.810071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837730+0000","last_change":"2024-05-27T09:27:17.567670+0000","last_active":"2024-05-27T09:27:20.837730+0000","last_peered":"2024-05-27T09:27:20.837730+0000","last_clean":"2024-05-27T09:27:20.837730+0000","last_became_active":"2024-05-27T09:27:17.566292+0000","last_became_peered":"2024-05-27T09:27:17.566292+0000","last_unstale":"2024-05-27T09:27:20.837730+0000","last_undegraded":"2024-05-27T09:27:20.837730+0000","last_fullsized":"2024-05-27T09:27:20.837730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:18.038221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836742+0000","last_change":"2024-05-27T09:27:18.571124+0000","last_active":"2024-05-27T09:27:20.836742+0000","last_peered":"2024-05-27T09:27:20.836742+0000","last_clean":"2024-05-27T09:27:20.836742+0000","last_became_active":"2024-05-27T09:27:18.570782+0000","last_became_peered":"2024-05-27T09:27:18.570782+0000","last_unstale":"2024-05-27T09:27:20.836742+0000","last_undegraded":"2024-05-27T09:27:20.836742+0000","last_fullsized":"2024-05-27T09:27:20.836742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:39:58.110527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836667+0000","last_change":"2024-05-27T09:27:17.563259+0000","last_active":"2024-05-27T09:27:20.836667+0000","last_peered":"2024-05-27T09:27:20.836667+0000","last_clean":"2024-05-27T09:27:20.836667+0000","last_became_active":"2024-05-27T09:27:17.563010+0000","last_became_peered":"2024-05-27T09:27:17.563010+0000","last_unstale":"2024-05-27T09:27:20.836667+0000","last_undegraded":"2024-05-27T09:27:20.836667+0000","last_fullsized":"2024-05-27T09:27:20.836667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:53.711508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836371+0000","last_change":"2024-05-27T09:27:18.574466+0000","last_active":"2024-05-27T09:27:20.836371+0000","last_peered":"2024-05-27T09:27:20.836371+0000","last_clean":"2024-05-27T09:27:20.836371+0000","last_became_active":"2024-05-27T09:27:18.574090+0000","last_became_peered":"2024-05-27T09:27:18.574090+0000","last_unstale":"2024-05-27T09:27:20.836371+0000","last_undegraded":"2024-05-27T09:27:20.836371+0000","last_fullsized":"2024-05-27T09:27:20.836371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:41:22.515484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835113+0000","last_change":"2024-05-27T09:27:17.551310+0000","last_active":"2024-05-27T09:27:20.835113+0000","last_peered":"2024-05-27T09:27:20.835113+0000","last_clean":"2024-05-27T09:27:20.835113+0000","last_became_active":"2024-05-27T09:27:17.550982+0000","last_became_peered":"2024-05-27T09:27:17.550982+0000","last_unstale":"2024-05-27T09:27:20.835113+0000","last_undegraded":"2024-05-27T09:27:20.835113+0000","last_fullsized":"2024-05-27T09:27:20.835113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836574+0000","last_change":"2024-05-27T09:27:18.559416+0000","last_active":"2024-05-27T09:27:20.836574+0000","last_peered":"2024-05-27T09:27:20.836574+0000","last_clean":"2024-05-27T09:27:20.836574+0000","last_became_active":"2024-05-27T09:27:18.559017+0000","last_became_peered":"2024-05-27T09:27:18.559017+0000","last_unstale":"2024-05-27T09:27:20.836574+0000","last_undegraded":"2024-05-27T09:27:20.836574+0000","last_fullsized":"2024-05-27T09:27:20.836574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:33:48.400306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837680+0000","last_change":"2024-05-27T09:27:17.563947+0000","last_active":"2024-05-27T09:27:20.837680+0000","last_peered":"2024-05-27T09:27:20.837680+0000","last_clean":"2024-05-27T09:27:20.837680+0000","last_became_active":"2024-05-27T09:27:17.563640+0000","last_became_peered":"2024-05-27T09:27:17.563640+0000","last_unstale":"2024-05-27T09:27:20.837680+0000","last_undegraded":"2024-05-27T09:27:20.837680+0000","last_fullsized":"2024-05-27T09:27:20.837680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:02.747708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543615+0000","last_change":"2024-05-27T09:27:18.575835+0000","last_active":"2024-05-27T09:27:19.543615+0000","last_peered":"2024-05-27T09:27:19.543615+0000","last_clean":"2024-05-27T09:27:19.543615+0000","last_became_active":"2024-05-27T09:27:18.574710+0000","last_became_peered":"2024-05-27T09:27:18.574710+0000","last_unstale":"2024-05-27T09:27:19.543615+0000","last_undegraded":"2024-05-27T09:27:19.543615+0000","last_fullsized":"2024-05-27T09:27:19.543615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:16:00.934068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835781+0000","last_change":"2024-05-27T09:27:17.550749+0000","last_active":"2024-05-27T09:27:20.835781+0000","last_peered":"2024-05-27T09:27:20.835781+0000","last_clean":"2024-05-27T09:27:20.835781+0000","last_became_active":"2024-05-27T09:27:17.550378+0000","last_became_peered":"2024-05-27T09:27:17.550378+0000","last_unstale":"2024-05-27T09:27:20.835781+0000","last_undegraded":"2024-05-27T09:27:20.835781+0000","last_fullsized":"2024-05-27T09:27:20.835781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:20.167629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.541045+0000","last_change":"2024-05-27T09:27:18.567233+0000","last_active":"2024-05-27T09:27:19.541045+0000","last_peered":"2024-05-27T09:27:19.541045+0000","last_clean":"2024-05-27T09:27:19.541045+0000","last_became_active":"2024-05-27T09:27:18.566326+0000","last_became_peered":"2024-05-27T09:27:18.566326+0000","last_unstale":"2024-05-27T09:27:19.541045+0000","last_undegraded":"2024-05-27T09:27:19.541045+0000","last_fullsized":"2024-05-27T09:27:19.541045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:27.225150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837758+0000","last_change":"2024-05-27T09:27:17.557912+0000","last_active":"2024-05-27T09:27:20.837758+0000","last_peered":"2024-05-27T09:27:20.837758+0000","last_clean":"2024-05-27T09:27:20.837758+0000","last_became_active":"2024-05-27T09:27:17.557216+0000","last_became_peered":"2024-05-27T09:27:17.557216+0000","last_unstale":"2024-05-27T09:27:20.837758+0000","last_undegraded":"2024-05-27T09:27:20.837758+0000","last_fullsized":"2024-05-27T09:27:20.837758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:21:22.020724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836847+0000","last_change":"2024-05-27T09:27:18.563649+0000","last_active":"2024-05-27T09:27:20.836847+0000","last_peered":"2024-05-27T09:27:20.836847+0000","last_clean":"2024-05-27T09:27:20.836847+0000","last_became_active":"2024-05-27T09:27:18.562565+0000","last_became_peered":"2024-05-27T09:27:18.562565+0000","last_unstale":"2024-05-27T09:27:20.836847+0000","last_undegraded":"2024-05-27T09:27:20.836847+0000","last_fullsized":"2024-05-27T09:27:20.836847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:46:49.444709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835661+0000","last_change":"2024-05-27T09:27:17.571911+0000","last_active":"2024-05-27T09:27:20.835661+0000","last_peered":"2024-05-27T09:27:20.835661+0000","last_clean":"2024-05-27T09:27:20.835661+0000","last_became_active":"2024-05-27T09:27:17.571551+0000","last_became_peered":"2024-05-27T09:27:17.571551+0000","last_unstale":"2024-05-27T09:27:20.835661+0000","last_undegraded":"2024-05-27T09:27:20.835661+0000","last_fullsized":"2024-05-27T09:27:20.835661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:20.167629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835899+0000","last_change":"2024-05-27T09:27:18.558780+0000","last_active":"2024-05-27T09:27:20.835899+0000","last_peered":"2024-05-27T09:27:20.835899+0000","last_clean":"2024-05-27T09:27:20.835899+0000","last_became_active":"2024-05-27T09:27:18.558463+0000","last_became_peered":"2024-05-27T09:27:18.558463+0000","last_unstale":"2024-05-27T09:27:20.835899+0000","last_undegraded":"2024-05-27T09:27:20.835899+0000","last_fullsized":"2024-05-27T09:27:20.835899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:42:38.351682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.559527+0000","last_change":"2024-05-27T09:27:17.567973+0000","last_active":"2024-05-27T09:27:23.559527+0000","last_peered":"2024-05-27T09:27:23.559527+0000","last_clean":"2024-05-27T09:27:23.559527+0000","last_became_active":"2024-05-27T09:27:17.567650+0000","last_became_peered":"2024-05-27T09:27:17.567650+0000","last_unstale":"2024-05-27T09:27:23.559527+0000","last_undegraded":"2024-05-27T09:27:23.559527+0000","last_fullsized":"2024-05-27T09:27:23.559527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:25:52.924185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835408+0000","last_change":"2024-05-27T09:27:18.554845+0000","last_active":"2024-05-27T09:27:20.835408+0000","last_peered":"2024-05-27T09:27:20.835408+0000","last_clean":"2024-05-27T09:27:20.835408+0000","last_became_active":"2024-05-27T09:27:18.554462+0000","last_became_peered":"2024-05-27T09:27:18.554462+0000","last_unstale":"2024-05-27T09:27:20.835408+0000","last_undegraded":"2024-05-27T09:27:20.835408+0000","last_fullsized":"2024-05-27T09:27:20.835408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:44:47.447245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.541867+0000","last_change":"2024-05-27T09:27:17.570300+0000","last_active":"2024-05-27T09:27:19.541867+0000","last_peered":"2024-05-27T09:27:19.541867+0000","last_clean":"2024-05-27T09:27:19.541867+0000","last_became_active":"2024-05-27T09:27:17.569685+0000","last_became_peered":"2024-05-27T09:27:17.569685+0000","last_unstale":"2024-05-27T09:27:19.541867+0000","last_undegraded":"2024-05-27T09:27:19.541867+0000","last_fullsized":"2024-05-27T09:27:19.541867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:26.210041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836430+0000","last_change":"2024-05-27T09:27:18.562774+0000","last_active":"2024-05-27T09:27:20.836430+0000","last_peered":"2024-05-27T09:27:20.836430+0000","last_clean":"2024-05-27T09:27:20.836430+0000","last_became_active":"2024-05-27T09:27:18.562455+0000","last_became_peered":"2024-05-27T09:27:18.562455+0000","last_unstale":"2024-05-27T09:27:20.836430+0000","last_undegraded":"2024-05-27T09:27:20.836430+0000","last_fullsized":"2024-05-27T09:27:20.836430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:39:59.034873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836403+0000","last_change":"2024-05-27T09:27:17.563172+0000","last_active":"2024-05-27T09:27:20.836403+0000","last_peered":"2024-05-27T09:27:20.836403+0000","last_clean":"2024-05-27T09:27:20.836403+0000","last_became_active":"2024-05-27T09:27:17.562941+0000","last_became_peered":"2024-05-27T09:27:17.562941+0000","last_unstale":"2024-05-27T09:27:20.836403+0000","last_undegraded":"2024-05-27T09:27:20.836403+0000","last_fullsized":"2024-05-27T09:27:20.836403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:51.742078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836477+0000","last_change":"2024-05-27T09:27:18.555617+0000","last_active":"2024-05-27T09:27:20.836477+0000","last_peered":"2024-05-27T09:27:20.836477+0000","last_clean":"2024-05-27T09:27:20.836477+0000","last_became_active":"2024-05-27T09:27:18.555280+0000","last_became_peered":"2024-05-27T09:27:18.555280+0000","last_unstale":"2024-05-27T09:27:20.836477+0000","last_undegraded":"2024-05-27T09:27:20.836477+0000","last_fullsized":"2024-05-27T09:27:20.836477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:32:39.043268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837070+0000","last_change":"2024-05-27T09:27:17.548266+0000","last_active":"2024-05-27T09:27:20.837070+0000","last_peered":"2024-05-27T09:27:20.837070+0000","last_clean":"2024-05-27T09:27:20.837070+0000","last_became_active":"2024-05-27T09:27:17.547938+0000","last_became_peered":"2024-05-27T09:27:17.547938+0000","last_unstale":"2024-05-27T09:27:20.837070+0000","last_undegraded":"2024-05-27T09:27:20.837070+0000","last_fullsized":"2024-05-27T09:27:20.837070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835677+0000","last_change":"2024-05-27T09:27:18.563586+0000","last_active":"2024-05-27T09:27:20.835677+0000","last_peered":"2024-05-27T09:27:20.835677+0000","last_clean":"2024-05-27T09:27:20.835677+0000","last_became_active":"2024-05-27T09:27:18.563227+0000","last_became_peered":"2024-05-27T09:27:18.563227+0000","last_unstale":"2024-05-27T09:27:20.835677+0000","last_undegraded":"2024-05-27T09:27:20.835677+0000","last_fullsized":"2024-05-27T09:27:20.835677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:53.222040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.561110+0000","last_change":"2024-05-27T09:27:17.562200+0000","last_active":"2024-05-27T09:27:23.561110+0000","last_peered":"2024-05-27T09:27:23.561110+0000","last_clean":"2024-05-27T09:27:23.561110+0000","last_became_active":"2024-05-27T09:27:17.561857+0000","last_became_peered":"2024-05-27T09:27:17.561857+0000","last_unstale":"2024-05-27T09:27:23.561110+0000","last_undegraded":"2024-05-27T09:27:23.561110+0000","last_fullsized":"2024-05-27T09:27:23.561110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:05.209528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836936+0000","last_change":"2024-05-27T09:27:18.572676+0000","last_active":"2024-05-27T09:27:20.836936+0000","last_peered":"2024-05-27T09:27:20.836936+0000","last_clean":"2024-05-27T09:27:20.836936+0000","last_became_active":"2024-05-27T09:27:18.572339+0000","last_became_peered":"2024-05-27T09:27:18.572339+0000","last_unstale":"2024-05-27T09:27:20.836936+0000","last_undegraded":"2024-05-27T09:27:20.836936+0000","last_fullsized":"2024-05-27T09:27:20.836936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:09:23.795528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837085+0000","last_change":"2024-05-27T09:27:18.575394+0000","last_active":"2024-05-27T09:27:20.837085+0000","last_peered":"2024-05-27T09:27:20.837085+0000","last_clean":"2024-05-27T09:27:20.837085+0000","last_became_active":"2024-05-27T09:27:18.572307+0000","last_became_peered":"2024-05-27T09:27:18.572307+0000","last_unstale":"2024-05-27T09:27:20.837085+0000","last_undegraded":"2024-05-27T09:27:20.837085+0000","last_fullsized":"2024-05-27T09:27:20.837085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:29:05.116582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.555603+0000","last_change":"2024-05-27T09:27:17.570634+0000","last_active":"2024-05-27T09:27:23.555603+0000","last_peered":"2024-05-27T09:27:23.555603+0000","last_clean":"2024-05-27T09:27:23.555603+0000","last_became_active":"2024-05-27T09:27:17.569866+0000","last_became_peered":"2024-05-27T09:27:17.569866+0000","last_unstale":"2024-05-27T09:27:23.555603+0000","last_undegraded":"2024-05-27T09:27:23.555603+0000","last_fullsized":"2024-05-27T09:27:23.555603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:21:38.268535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539980+0000","last_change":"2024-05-27T09:27:18.549339+0000","last_active":"2024-05-27T09:27:19.539980+0000","last_peered":"2024-05-27T09:27:19.539980+0000","last_clean":"2024-05-27T09:27:19.539980+0000","last_became_active":"2024-05-27T09:27:18.549035+0000","last_became_peered":"2024-05-27T09:27:18.549035+0000","last_unstale":"2024-05-27T09:27:19.539980+0000","last_undegraded":"2024-05-27T09:27:19.539980+0000","last_fullsized":"2024-05-27T09:27:19.539980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:17.895211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.551871+0000","last_change":"2024-05-27T09:27:17.568143+0000","last_active":"2024-05-27T09:27:23.551871+0000","last_peered":"2024-05-27T09:27:23.551871+0000","last_clean":"2024-05-27T09:27:23.551871+0000","last_became_active":"2024-05-27T09:27:17.567235+0000","last_became_peered":"2024-05-27T09:27:17.567235+0000","last_unstale":"2024-05-27T09:27:23.551871+0000","last_undegraded":"2024-05-27T09:27:23.551871+0000","last_fullsized":"2024-05-27T09:27:23.551871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.284270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836871+0000","last_change":"2024-05-27T09:27:18.563270+0000","last_active":"2024-05-27T09:27:20.836871+0000","last_peered":"2024-05-27T09:27:20.836871+0000","last_clean":"2024-05-27T09:27:20.836871+0000","last_became_active":"2024-05-27T09:27:18.562919+0000","last_became_peered":"2024-05-27T09:27:18.562919+0000","last_unstale":"2024-05-27T09:27:20.836871+0000","last_undegraded":"2024-05-27T09:27:20.836871+0000","last_fullsized":"2024-05-27T09:27:20.836871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:55:47.799529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:21.370330+0000","last_change":"2024-05-27T09:27:17.551193+0000","last_active":"2024-05-27T09:27:21.370330+0000","last_peered":"2024-05-27T09:27:21.370330+0000","last_clean":"2024-05-27T09:27:21.370330+0000","last_became_active":"2024-05-27T09:27:17.550849+0000","last_became_peered":"2024-05-27T09:27:17.550849+0000","last_unstale":"2024-05-27T09:27:21.370330+0000","last_undegraded":"2024-05-27T09:27:21.370330+0000","last_fullsized":"2024-05-27T09:27:21.370330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:51.742078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836183+0000","last_change":"2024-05-27T09:27:18.579803+0000","last_active":"2024-05-27T09:27:20.836183+0000","last_peered":"2024-05-27T09:27:20.836183+0000","last_clean":"2024-05-27T09:27:20.836183+0000","last_became_active":"2024-05-27T09:27:18.579574+0000","last_became_peered":"2024-05-27T09:27:18.579574+0000","last_unstale":"2024-05-27T09:27:20.836183+0000","last_undegraded":"2024-05-27T09:27:20.836183+0000","last_fullsized":"2024-05-27T09:27:20.836183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:11:22.320778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.560320+0000","last_change":"2024-05-27T09:27:17.549711+0000","last_active":"2024-05-27T09:27:23.560320+0000","last_peered":"2024-05-27T09:27:23.560320+0000","last_clean":"2024-05-27T09:27:23.560320+0000","last_became_active":"2024-05-27T09:27:17.549291+0000","last_became_peered":"2024-05-27T09:27:17.549291+0000","last_unstale":"2024-05-27T09:27:23.560320+0000","last_undegraded":"2024-05-27T09:27:23.560320+0000","last_fullsized":"2024-05-27T09:27:23.560320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:16:22.057803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65078,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":212992,"data_stored":73257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":74287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":663113,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":72731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":72799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":72675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":663135,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":74959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T09:27:32.095 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-27T09:27:32.096 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-27T09:27:32.096 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-27T09:27:32.096 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-27T09:27:32.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.404+0000 7fd8e4400640 1 Processor -- start 2024-05-27T09:27:32.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.404+0000 7fd8e4400640 1 -- start start 2024-05-27T09:27:32.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.406+0000 7fd8e4400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8dc142720 0x7fd8dc073260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:32.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.406+0000 7fd8e4400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd8dc0c2990 0x7fd8dc0c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:32.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.406+0000 7fd8e4400640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8dc0739a0 con 0x7fd8dc1593b0 2024-05-27T09:27:32.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.406+0000 7fd8e4400640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fd8dc074d20 con 0x7fd8dc0c2990 2024-05-27T09:27:32.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.406+0000 7fd8e4400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fd8dc142b30 con 0x7fd8dc142720 2024-05-27T09:27:32.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.406+0000 7fd8e2400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd8dc0c2990 0x7fd8dc0c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:32.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.406+0000 7fd8e2400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd8dc0c2990 0x7fd8dc0c2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35892/0 (socket says 172.21.15.114:35892) 2024-05-27T09:27:32.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.406+0000 7fd8e2400640 1 -- 172.21.15.114:0/3812181256 learned_addr learned my addr 172.21.15.114:0/3812181256 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:32.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.406+0000 7fd8e1000640 1 --2- 172.21.15.114:0/3812181256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8dc142720 0x7fd8dc073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:32.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.407+0000 7fd8e2400640 1 -- 172.21.15.114:0/3812181256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8dc142720 msgr2=0x7fd8dc073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:32.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.407+0000 7fd8e2400640 1 --2- 172.21.15.114:0/3812181256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8dc142720 0x7fd8dc073260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:32.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.407+0000 7fd8e2400640 1 -- 172.21.15.114:0/3812181256 >> v1:172.21.15.163:6789/0 conn(0x7fd8dc1593b0 legacy=0x7fd8dc15a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:32.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.407+0000 7fd8dbe00640 1 -- 172.21.15.114:0/3812181256 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2816324839 0 0) 0x7fd8dc0739a0 con 0x7fd8dc1593b0 2024-05-27T09:27:32.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.407+0000 7fd8e2400640 1 -- 172.21.15.114:0/3812181256 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8dc073c80 con 0x7fd8dc0c2990 2024-05-27T09:27:32.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.408+0000 7fd8e2400640 1 --2- 172.21.15.114:0/3812181256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8dc0c2990 0x7fd8dc0c2da0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009e80 tx=0x7fd8d402c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bdb351bb98c12ea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:32.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.408+0000 7fd8dbe00640 1 -- 172.21.15.114:0/3812181256 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4035020 con 0x7fd8dc0c2990 2024-05-27T09:27:32.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.408+0000 7fd8dbe00640 1 -- 172.21.15.114:0/3812181256 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8d4031400 con 0x7fd8dc0c2990 2024-05-27T09:27:32.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.408+0000 7fd8dbe00640 1 -- 172.21.15.114:0/3812181256 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d40317d0 con 0x7fd8dc0c2990 2024-05-27T09:27:32.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.409+0000 7fd8e4400640 1 -- 172.21.15.114:0/3812181256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8dc0c2990 msgr2=0x7fd8dc0c2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:32.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.409+0000 7fd8e4400640 1 --2- 172.21.15.114:0/3812181256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8dc0c2990 0x7fd8dc0c2da0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009e80 tx=0x7fd8d402c9f0 comp rx=0 tx=0).stop 2024-05-27T09:27:32.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.409+0000 7fd8e4400640 1 -- 172.21.15.114:0/3812181256 shutdown_connections 2024-05-27T09:27:32.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.409+0000 7fd8e4400640 1 --2- 172.21.15.114:0/3812181256 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8dc0c2990 0x7fd8dc0c2da0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:32.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.409+0000 7fd8e4400640 1 --2- 172.21.15.114:0/3812181256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8dc142720 0x7fd8dc073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:32.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.409+0000 7fd8e4400640 1 -- 172.21.15.114:0/3812181256 >> 172.21.15.114:0/3812181256 conn(0x7fd8dc06f6e0 msgr2=0x7fd8dc070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:32.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.409+0000 7fd8e4400640 1 -- 172.21.15.114:0/3812181256 shutdown_connections 2024-05-27T09:27:32.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.409+0000 7fd8e4400640 1 -- 172.21.15.114:0/3812181256 wait complete. 2024-05-27T09:27:32.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.411+0000 7fd8e4400640 1 Processor -- start 2024-05-27T09:27:32.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.411+0000 7fd8e4400640 1 -- start start 2024-05-27T09:27:32.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.413+0000 7fd8e4400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8dc0c2990 0x7fd8dc15c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:32.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.413+0000 7fd8e4400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8dc142720 0x7fd8dc0ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:32.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.413+0000 7fd8e4400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8dc1593b0 0x7fd8dc0b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:32.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.413+0000 7fd8e4400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd8dc074520 con 0x7fd8dc0c2990 2024-05-27T09:27:32.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.413+0000 7fd8e4400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd8dc074240 con 0x7fd8dc1593b0 2024-05-27T09:27:32.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.413+0000 7fd8e4400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fd8dc074800 con 0x7fd8dc142720 2024-05-27T09:27:32.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.413+0000 7fd8e1a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8dc0c2990 0x7fd8dc15c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:32.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.413+0000 7fd8e1000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8dc142720 0x7fd8dc0ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:32.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.413+0000 7fd8e1a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8dc0c2990 0x7fd8dc15c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35900/0 (socket says 172.21.15.114:35900) 2024-05-27T09:27:32.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.413+0000 7fd8e1a00640 1 -- 172.21.15.114:0/4067522808 learned_addr learned my addr 172.21.15.114:0/4067522808 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:32.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.413+0000 7fd8e2400640 1 --2- 172.21.15.114:0/4067522808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8dc1593b0 0x7fd8dc0b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:32.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.414+0000 7fd8e1a00640 1 -- 172.21.15.114:0/4067522808 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8dc142720 msgr2=0x7fd8dc0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:32.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.414+0000 7fd8e1a00640 1 --2- 172.21.15.114:0/4067522808 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8dc142720 0x7fd8dc0ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:32.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.414+0000 7fd8e1a00640 1 -- 172.21.15.114:0/4067522808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8dc1593b0 msgr2=0x7fd8dc0b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:32.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.414+0000 7fd8e1a00640 1 --2- 172.21.15.114:0/4067522808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8dc1593b0 0x7fd8dc0b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:32.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.414+0000 7fd8e1a00640 1 -- 172.21.15.114:0/4067522808 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8dc0ca1a0 con 0x7fd8dc0c2990 2024-05-27T09:27:32.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.414+0000 7fd8e2400640 1 --2- 172.21.15.114:0/4067522808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8dc1593b0 0x7fd8dc0b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:32.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.414+0000 7fd8e1a00640 1 --2- 172.21.15.114:0/4067522808 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8dc0c2990 0x7fd8dc15c730 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000c3f0 tx=0x7fd8d000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:32.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.414+0000 7fd8d9600640 1 -- 172.21.15.114:0/4067522808 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0017070 con 0x7fd8dc0c2990 2024-05-27T09:27:32.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.414+0000 7fd8d9600640 1 -- 172.21.15.114:0/4067522808 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8d00035f0 con 0x7fd8dc0c2990 2024-05-27T09:27:32.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.414+0000 7fd8e4400640 1 -- 172.21.15.114:0/4067522808 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8dc0ca480 con 0x7fd8dc0c2990 2024-05-27T09:27:32.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.414+0000 7fd8d9600640 1 -- 172.21.15.114:0/4067522808 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d00039e0 con 0x7fd8dc0c2990 2024-05-27T09:27:32.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.414+0000 7fd8e4400640 1 -- 172.21.15.114:0/4067522808 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8dc1f6040 con 0x7fd8dc0c2990 2024-05-27T09:27:32.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.416+0000 7fd8e4400640 1 -- 172.21.15.114:0/4067522808 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8dc1f62a0 con 0x7fd8dc0c2990 2024-05-27T09:27:32.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.417+0000 7fd8d9600640 1 -- 172.21.15.114:0/4067522808 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd8d001a070 con 0x7fd8dc0c2990 2024-05-27T09:27:32.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.418+0000 7fd8d9600640 1 --2- 172.21.15.114:0/4067522808 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd8bc076300 0x7fd8bc078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:32.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.418+0000 7fd8d9600640 1 -- 172.21.15.114:0/4067522808 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd8d0096820 con 0x7fd8dc0c2990 2024-05-27T09:27:32.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.418+0000 7fd8e1000640 1 --2- 172.21.15.114:0/4067522808 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd8bc076300 0x7fd8bc078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:32.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.419+0000 7fd8e1000640 1 --2- 172.21.15.114:0/4067522808 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd8bc076300 0x7fd8bc078810 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc13a0d0 tx=0x7fd8c8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:32.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.422+0000 7fd8d9600640 1 -- 172.21.15.114:0/4067522808 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd8d005fbc0 con 0x7fd8dc0c2990 2024-05-27T09:27:32.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.651+0000 7fd8e4400640 1 -- 172.21.15.114:0/4067522808 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd8dc06a380 con 0x7fd8dc0c2990 2024-05-27T09:27:32.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.652+0000 7fd8d9600640 1 -- 172.21.15.114:0/4067522808 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd8d0068070 con 0x7fd8dc0c2990 2024-05-27T09:27:32.654 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:32.654 INFO:teuthology.orchestra.run.smithi114.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-27T09:27:32.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.655+0000 7fd8e4400640 1 -- 172.21.15.114:0/4067522808 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd8bc076300 msgr2=0x7fd8bc078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:32.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.655+0000 7fd8e4400640 1 --2- 172.21.15.114:0/4067522808 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd8bc076300 0x7fd8bc078810 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc13a0d0 tx=0x7fd8c8009190 comp rx=0 tx=0).stop 2024-05-27T09:27:32.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.655+0000 7fd8e4400640 1 -- 172.21.15.114:0/4067522808 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8dc0c2990 msgr2=0x7fd8dc15c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:32.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.655+0000 7fd8e4400640 1 --2- 172.21.15.114:0/4067522808 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8dc0c2990 0x7fd8dc15c730 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000c3f0 tx=0x7fd8d000c8b0 comp rx=0 tx=0).stop 2024-05-27T09:27:32.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.655+0000 7fd8e4400640 1 -- 172.21.15.114:0/4067522808 shutdown_connections 2024-05-27T09:27:32.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.655+0000 7fd8e4400640 1 --2- 172.21.15.114:0/4067522808 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd8bc076300 0x7fd8bc078810 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:32.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.655+0000 7fd8e4400640 1 --2- 172.21.15.114:0/4067522808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8dc1593b0 0x7fd8dc0b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:32.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.655+0000 7fd8e4400640 1 --2- 172.21.15.114:0/4067522808 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8dc142720 0x7fd8dc0ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:32.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.655+0000 7fd8e4400640 1 --2- 172.21.15.114:0/4067522808 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8dc0c2990 0x7fd8dc15c730 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:32.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.655+0000 7fd8e4400640 1 -- 172.21.15.114:0/4067522808 >> 172.21.15.114:0/4067522808 conn(0x7fd8dc06f6e0 msgr2=0x7fd8dc1410e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:32.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.655+0000 7fd8e4400640 1 -- 172.21.15.114:0/4067522808 shutdown_connections 2024-05-27T09:27:32.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.655+0000 7fd8e4400640 1 -- 172.21.15.114:0/4067522808 wait complete. 2024-05-27T09:27:32.681 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-27T09:27:32.682 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:27:32.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.939+0000 7f916ae00640 1 Processor -- start 2024-05-27T09:27:32.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.939+0000 7f916ae00640 1 -- start start 2024-05-27T09:27:32.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.941+0000 7f916ae00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91641547d0 0x7f91640c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:32.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.941+0000 7f916ae00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f916406fe50 con 0x7f91640741f0 2024-05-27T09:27:32.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.942+0000 7f916ae00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91640700f0 con 0x7f9164074840 2024-05-27T09:27:32.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.942+0000 7f916ae00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f91640703d0 con 0x7f91641547d0 2024-05-27T09:27:32.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.942+0000 7f9169a00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f91640741f0 0x7f916406edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:46692/0 (socket says 172.21.15.114:46692) 2024-05-27T09:27:32.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.942+0000 7f9169a00640 1 -- 172.21.15.114:0/3441372655 learned_addr learned my addr 172.21.15.114:0/3441372655 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:32.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.942+0000 7f9169000640 1 --2- 172.21.15.114:0/3441372655 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91641547d0 0x7f91640c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:32.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.942+0000 7f9169000640 1 -- 172.21.15.114:0/3441372655 >> v1:172.21.15.163:6789/0 conn(0x7f9164074840 legacy=0x7f9164164b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:32.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.942+0000 7f9169000640 1 -- 172.21.15.114:0/3441372655 >> v1:172.21.15.114:6789/0 conn(0x7f91640741f0 legacy=0x7f916406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:32.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.942+0000 7f9169000640 1 -- 172.21.15.114:0/3441372655 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91640731d0 con 0x7f91641547d0 2024-05-27T09:27:32.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.942+0000 7f9163e00640 1 -- 172.21.15.114:0/3441372655 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2473646739 0 0) 0x7f916406fe50 con 0x7f91640741f0 2024-05-27T09:27:32.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.943+0000 7f9169000640 1 --2- 172.21.15.114:0/3441372655 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91641547d0 0x7f91640c63a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9150009750 tx=0x7f915002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f922d3cf56783a2b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:32.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.943+0000 7f9163e00640 1 -- 172.21.15.114:0/3441372655 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9150037070 con 0x7f91641547d0 2024-05-27T09:27:32.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.943+0000 7f9163e00640 1 -- 172.21.15.114:0/3441372655 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f915002ae40 con 0x7f91641547d0 2024-05-27T09:27:32.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.943+0000 7f9163e00640 1 -- 172.21.15.114:0/3441372655 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9150032660 con 0x7f91641547d0 2024-05-27T09:27:32.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.944+0000 7f916ae00640 1 -- 172.21.15.114:0/3441372655 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91641547d0 msgr2=0x7f91640c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:32.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.944+0000 7f916ae00640 1 --2- 172.21.15.114:0/3441372655 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91641547d0 0x7f91640c63a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9150009750 tx=0x7f915002a130 comp rx=0 tx=0).stop 2024-05-27T09:27:32.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.944+0000 7f916ae00640 1 -- 172.21.15.114:0/3441372655 shutdown_connections 2024-05-27T09:27:32.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.944+0000 7f916ae00640 1 --2- 172.21.15.114:0/3441372655 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91641547d0 0x7f91640c63a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:32.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.944+0000 7f916ae00640 1 -- 172.21.15.114:0/3441372655 >> 172.21.15.114:0/3441372655 conn(0x7f91640ca320 msgr2=0x7f91640ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:32.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.944+0000 7f916ae00640 1 -- 172.21.15.114:0/3441372655 shutdown_connections 2024-05-27T09:27:32.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.944+0000 7f916ae00640 1 -- 172.21.15.114:0/3441372655 wait complete. 2024-05-27T09:27:32.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.947+0000 7f916ae00640 1 Processor -- start 2024-05-27T09:27:32.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.947+0000 7f916ae00640 1 -- start start 2024-05-27T09:27:32.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.949+0000 7f916ae00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91640741f0 0x7f91640c3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:32.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.949+0000 7f916ae00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9164074840 0x7f916414fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:32.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.949+0000 7f916ae00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91641547d0 0x7f9164150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:32.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.949+0000 7f916ae00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f916406f500 con 0x7f9164074840 2024-05-27T09:27:32.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.949+0000 7f916ae00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9164073790 con 0x7f91641547d0 2024-05-27T09:27:32.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.949+0000 7f916ae00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f916406f680 con 0x7f91640741f0 2024-05-27T09:27:32.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.949+0000 7f9169000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9164074840 0x7f916414fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:32.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.949+0000 7f9169000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9164074840 0x7f916414fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35904/0 (socket says 172.21.15.114:35904) 2024-05-27T09:27:32.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.949+0000 7f9169000640 1 -- 172.21.15.114:0/3093175529 learned_addr learned my addr 172.21.15.114:0/3093175529 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:32.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.949+0000 7f9169a00640 1 --2- 172.21.15.114:0/3093175529 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91640741f0 0x7f91640c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:32.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.949+0000 7f916a400640 1 --2- 172.21.15.114:0/3093175529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91641547d0 0x7f9164150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:32.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.949+0000 7f9169000640 1 -- 172.21.15.114:0/3093175529 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91640741f0 msgr2=0x7f91640c3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:32.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.949+0000 7f9169000640 1 --2- 172.21.15.114:0/3093175529 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91640741f0 0x7f91640c3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:32.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.949+0000 7f9169000640 1 -- 172.21.15.114:0/3093175529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91641547d0 msgr2=0x7f9164150170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:32.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.949+0000 7f9169000640 1 --2- 172.21.15.114:0/3093175529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91641547d0 0x7f9164150170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:32.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.949+0000 7f9169000640 1 -- 172.21.15.114:0/3093175529 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91640c3dd0 con 0x7f9164074840 2024-05-27T09:27:32.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.950+0000 7f9169000640 1 --2- 172.21.15.114:0/3093175529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9164074840 0x7f916414fc30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9150009870 tx=0x7f91500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:32.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.950+0000 7f9161600640 1 -- 172.21.15.114:0/3093175529 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9150037040 con 0x7f9164074840 2024-05-27T09:27:32.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.950+0000 7f9161600640 1 -- 172.21.15.114:0/3093175529 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9150004030 con 0x7f9164074840 2024-05-27T09:27:32.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.950+0000 7f916ae00640 1 -- 172.21.15.114:0/3093175529 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91640c3740 con 0x7f9164074840 2024-05-27T09:27:32.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.950+0000 7f9161600640 1 -- 172.21.15.114:0/3093175529 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9150004420 con 0x7f9164074840 2024-05-27T09:27:32.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.950+0000 7f916ae00640 1 -- 172.21.15.114:0/3093175529 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9164150780 con 0x7f9164074840 2024-05-27T09:27:32.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.951+0000 7f916ae00640 1 -- 172.21.15.114:0/3093175529 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91640c6030 con 0x7f9164074840 2024-05-27T09:27:32.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.952+0000 7f9161600640 1 -- 172.21.15.114:0/3093175529 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f915003a040 con 0x7f9164074840 2024-05-27T09:27:32.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.953+0000 7f9161600640 1 --2- 172.21.15.114:0/3093175529 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9144076300 0x7f9144078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:32.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.953+0000 7f9161600640 1 -- 172.21.15.114:0/3093175529 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f91500b7c40 con 0x7f9164074840 2024-05-27T09:27:32.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.954+0000 7f9169a00640 1 --2- 172.21.15.114:0/3093175529 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9144076300 0x7f9144078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:32.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.954+0000 7f9169a00640 1 --2- 172.21.15.114:0/3093175529 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9144076300 0x7f9144078810 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9158005880 tx=0x7f9158009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:32.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:32.958+0000 7f9161600640 1 -- 172.21.15.114:0/3093175529 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f91640c6030 con 0x7f9164074840 2024-05-27T09:27:33.026 INFO:tasks.ceph.mds.e.smithi114.stderr:2024-05-27T09:27:33.024+0000 7f4ffb600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-27T09:27:33.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.198+0000 7f916ae00640 1 -- 172.21.15.114:0/3093175529 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91640b2f30 con 0x7f9164074840 2024-05-27T09:27:33.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.199+0000 7f9161600640 1 -- 172.21.15.114:0/3093175529 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+6874 (secure 0 0 0) 0x7f9150042020 con 0x7f9164074840 2024-05-27T09:27:33.201 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:33.201 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:27:28.719478+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"f","rank":2,"incarnation":11,"state":"up:creating","state_seq":1,"addr":"172.21.15.163:6839/444766663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":444766663},{"type":"v1","addr":"172.21.15.163:6839","nonce":444766663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4297]},"id":1}]} 2024-05-27T09:27:33.201 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 11 2024-05-27T09:27:33.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.202+0000 7f916ae00640 1 -- 172.21.15.114:0/3093175529 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9144076300 msgr2=0x7f9144078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:33.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.202+0000 7f916ae00640 1 --2- 172.21.15.114:0/3093175529 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9144076300 0x7f9144078810 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9158005880 tx=0x7f9158009040 comp rx=0 tx=0).stop 2024-05-27T09:27:33.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.202+0000 7f916ae00640 1 -- 172.21.15.114:0/3093175529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9164074840 msgr2=0x7f916414fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:33.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.202+0000 7f916ae00640 1 --2- 172.21.15.114:0/3093175529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9164074840 0x7f916414fc30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9150009870 tx=0x7f91500046c0 comp rx=0 tx=0).stop 2024-05-27T09:27:33.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.202+0000 7f916ae00640 1 -- 172.21.15.114:0/3093175529 shutdown_connections 2024-05-27T09:27:33.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.202+0000 7f916ae00640 1 --2- 172.21.15.114:0/3093175529 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9144076300 0x7f9144078810 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:33.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.202+0000 7f916ae00640 1 --2- 172.21.15.114:0/3093175529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91641547d0 0x7f9164150170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:33.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.202+0000 7f916ae00640 1 --2- 172.21.15.114:0/3093175529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9164074840 0x7f916414fc30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:33.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.202+0000 7f916ae00640 1 --2- 172.21.15.114:0/3093175529 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91640741f0 0x7f91640c3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:33.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.202+0000 7f916ae00640 1 -- 172.21.15.114:0/3093175529 >> 172.21.15.114:0/3093175529 conn(0x7f91640ca320 msgr2=0x7f9164158110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:33.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.202+0000 7f916ae00640 1 -- 172.21.15.114:0/3093175529 shutdown_connections 2024-05-27T09:27:33.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.202+0000 7f916ae00640 1 -- 172.21.15.114:0/3093175529 wait complete. 2024-05-27T09:27:33.230 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:27:33.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.486+0000 7fbd05200640 1 Processor -- start 2024-05-27T09:27:33.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.487+0000 7fbd05200640 1 -- start start 2024-05-27T09:27:33.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.489+0000 7fbd05200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbd000c2a00 0x7fbd00159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:33.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.489+0000 7fbd05200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd000c96d0 con 0x7fbd00159e40 2024-05-27T09:27:33.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.489+0000 7fbd05200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd000c8930 con 0x7fbd001534c0 2024-05-27T09:27:33.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.489+0000 7fbd05200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fbd00159190 con 0x7fbd000c2a00 2024-05-27T09:27:33.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.489+0000 7fbcfea00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbd000c2a00 0x7fbd00159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:33.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.489+0000 7fbcfea00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbd000c2a00 0x7fbd00159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37976/0 (socket says 172.21.15.114:37976) 2024-05-27T09:27:33.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.489+0000 7fbcff400640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fbd001534c0 0x7fbd000c8220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:44766/0 (socket says 172.21.15.114:44766) 2024-05-27T09:27:33.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.489+0000 7fbcfea00640 1 -- 172.21.15.114:0/587001838 learned_addr learned my addr 172.21.15.114:0/587001838 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:33.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.489+0000 7fbcffe00640 1 --1- 172.21.15.114:0/587001838 >> v1:172.21.15.114:6789/0 conn(0x7fbd00159e40 0x7fbd00074140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:46696/0 (socket says 172.21.15.114:46696) 2024-05-27T09:27:33.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.489+0000 7fbcfea00640 1 -- 172.21.15.114:0/587001838 >> v1:172.21.15.163:6789/0 conn(0x7fbd001534c0 legacy=0x7fbd000c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:33.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.490+0000 7fbcfea00640 1 -- 172.21.15.114:0/587001838 >> v1:172.21.15.114:6789/0 conn(0x7fbd00159e40 legacy=0x7fbd00074140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:33.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.490+0000 7fbcfea00640 1 -- 172.21.15.114:0/587001838 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd000c9970 con 0x7fbd000c2a00 2024-05-27T09:27:33.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.490+0000 7fbcfe000640 1 -- 172.21.15.114:0/587001838 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1263022511 0 0) 0x7fbd000c8930 con 0x7fbd001534c0 2024-05-27T09:27:33.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.491+0000 7fbcfea00640 1 --2- 172.21.15.114:0/587001838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd000c2a00 0x7fbd00159900 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbcec009870 tx=0x7fbcec02a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0673cf9120f77e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:33.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.491+0000 7fbcfe000640 1 -- 172.21.15.114:0/587001838 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec037070 con 0x7fbd000c2a00 2024-05-27T09:27:33.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.491+0000 7fbcfe000640 1 -- 172.21.15.114:0/587001838 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcec02ae50 con 0x7fbd000c2a00 2024-05-27T09:27:33.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.491+0000 7fbcfe000640 1 -- 172.21.15.114:0/587001838 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec032860 con 0x7fbd000c2a00 2024-05-27T09:27:33.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.492+0000 7fbd05200640 1 -- 172.21.15.114:0/587001838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd000c2a00 msgr2=0x7fbd00159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:33.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.492+0000 7fbd05200640 1 --2- 172.21.15.114:0/587001838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd000c2a00 0x7fbd00159900 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbcec009870 tx=0x7fbcec02a300 comp rx=0 tx=0).stop 2024-05-27T09:27:33.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.492+0000 7fbd05200640 1 -- 172.21.15.114:0/587001838 shutdown_connections 2024-05-27T09:27:33.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.492+0000 7fbd05200640 1 --2- 172.21.15.114:0/587001838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd000c2a00 0x7fbd00159900 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:33.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.492+0000 7fbd05200640 1 -- 172.21.15.114:0/587001838 >> 172.21.15.114:0/587001838 conn(0x7fbd000ca320 msgr2=0x7fbd000ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:33.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.492+0000 7fbd05200640 1 -- 172.21.15.114:0/587001838 shutdown_connections 2024-05-27T09:27:33.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.492+0000 7fbd05200640 1 -- 172.21.15.114:0/587001838 wait complete. 2024-05-27T09:27:33.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.494+0000 7fbd05200640 1 Processor -- start 2024-05-27T09:27:33.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.494+0000 7fbd05200640 1 -- start start 2024-05-27T09:27:33.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.496+0000 7fbd05200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd000c2a00 0x7fbd000c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:33.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.496+0000 7fbd05200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd001534c0 0x7fbd001402b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:33.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.496+0000 7fbd05200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd00159e40 0x7fbd001407f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:33.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.496+0000 7fbd05200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbd000c9f30 con 0x7fbd00159e40 2024-05-27T09:27:33.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.496+0000 7fbd05200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbd001538d0 con 0x7fbd001534c0 2024-05-27T09:27:33.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.496+0000 7fbd05200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fbd001593e0 con 0x7fbd000c2a00 2024-05-27T09:27:33.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.496+0000 7fbcffe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd00159e40 0x7fbd001407f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:33.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.496+0000 7fbcffe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd00159e40 0x7fbd001407f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35906/0 (socket says 172.21.15.114:35906) 2024-05-27T09:27:33.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.496+0000 7fbcff400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd000c2a00 0x7fbd000c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:33.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.496+0000 7fbcffe00640 1 -- 172.21.15.114:0/3429503186 learned_addr learned my addr 172.21.15.114:0/3429503186 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:33.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.496+0000 7fbcfea00640 1 --2- 172.21.15.114:0/3429503186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd001534c0 0x7fbd001402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:33.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.497+0000 7fbcff400640 1 -- 172.21.15.114:0/3429503186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd001534c0 msgr2=0x7fbd001402b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:33.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.497+0000 7fbcff400640 1 --2- 172.21.15.114:0/3429503186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd001534c0 0x7fbd001402b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:33.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.497+0000 7fbcff400640 1 -- 172.21.15.114:0/3429503186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd00159e40 msgr2=0x7fbd001407f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:33.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.497+0000 7fbcff400640 1 --2- 172.21.15.114:0/3429503186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd00159e40 0x7fbd001407f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:33.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.497+0000 7fbcff400640 1 -- 172.21.15.114:0/3429503186 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd00151220 con 0x7fbd000c2a00 2024-05-27T09:27:33.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.497+0000 7fbcffe00640 1 --2- 172.21.15.114:0/3429503186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd00159e40 0x7fbd001407f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:33.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.497+0000 7fbcff400640 1 --2- 172.21.15.114:0/3429503186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd000c2a00 0x7fbd000c54c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000c390 tx=0x7fbcf000c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:33.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.497+0000 7fbcf7400640 1 -- 172.21.15.114:0/3429503186 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf00074e0 con 0x7fbd000c2a00 2024-05-27T09:27:33.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.497+0000 7fbcf7400640 1 -- 172.21.15.114:0/3429503186 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcf00047e0 con 0x7fbd000c2a00 2024-05-27T09:27:33.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.497+0000 7fbcf7400640 1 -- 172.21.15.114:0/3429503186 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0004c10 con 0x7fbd000c2a00 2024-05-27T09:27:33.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.497+0000 7fbd05200640 1 -- 172.21.15.114:0/3429503186 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd00151500 con 0x7fbd000c2a00 2024-05-27T09:27:33.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.497+0000 7fbd05200640 1 -- 172.21.15.114:0/3429503186 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd000cbaa0 con 0x7fbd000c2a00 2024-05-27T09:27:33.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.498+0000 7fbd05200640 1 -- 172.21.15.114:0/3429503186 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd00050860 con 0x7fbd000c2a00 2024-05-27T09:27:33.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.500+0000 7fbcf7400640 1 -- 172.21.15.114:0/3429503186 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbcf0011070 con 0x7fbd000c2a00 2024-05-27T09:27:33.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.501+0000 7fbcf7400640 1 --2- 172.21.15.114:0/3429503186 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbce00762d0 0x7fbce00787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:33.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.501+0000 7fbcf7400640 1 -- 172.21.15.114:0/3429503186 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbcf0096f90 con 0x7fbd000c2a00 2024-05-27T09:27:33.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.501+0000 7fbcfea00640 1 --2- 172.21.15.114:0/3429503186 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbce00762d0 0x7fbce00787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:33.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.502+0000 7fbcfea00640 1 --2- 172.21.15.114:0/3429503186 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbce00762d0 0x7fbce00787e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbcec034000 tx=0x7fbcec02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:33.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.506+0000 7fbcf7400640 1 -- 172.21.15.114:0/3429503186 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbcf0060290 con 0x7fbd000c2a00 2024-05-27T09:27:33.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.743+0000 7fbd05200640 1 -- 172.21.15.114:0/3429503186 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd000c40f0 con 0x7fbd000c2a00 2024-05-27T09:27:33.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.744+0000 7fbcf7400640 1 -- 172.21.15.114:0/3429503186 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+6874 (secure 0 0 0) 0x7fbcf0064590 con 0x7fbd000c2a00 2024-05-27T09:27:33.746 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:33.746 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":11,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:27:28.719478+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"f","rank":2,"incarnation":11,"state":"up:creating","state_seq":1,"addr":"172.21.15.163:6839/444766663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":444766663},{"type":"v1","addr":"172.21.15.163:6839","nonce":444766663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4297]},"id":1}]} 2024-05-27T09:27:33.746 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 11 2024-05-27T09:27:33.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.747+0000 7fbd05200640 1 -- 172.21.15.114:0/3429503186 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbce00762d0 msgr2=0x7fbce00787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:33.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.747+0000 7fbd05200640 1 --2- 172.21.15.114:0/3429503186 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbce00762d0 0x7fbce00787e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbcec034000 tx=0x7fbcec02e000 comp rx=0 tx=0).stop 2024-05-27T09:27:33.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.747+0000 7fbd05200640 1 -- 172.21.15.114:0/3429503186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd000c2a00 msgr2=0x7fbd000c54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:33.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.747+0000 7fbd05200640 1 --2- 172.21.15.114:0/3429503186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd000c2a00 0x7fbd000c54c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000c390 tx=0x7fbcf000c850 comp rx=0 tx=0).stop 2024-05-27T09:27:33.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.747+0000 7fbd05200640 1 -- 172.21.15.114:0/3429503186 shutdown_connections 2024-05-27T09:27:33.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.747+0000 7fbd05200640 1 --2- 172.21.15.114:0/3429503186 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbce00762d0 0x7fbce00787e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:33.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.747+0000 7fbd05200640 1 --2- 172.21.15.114:0/3429503186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd00159e40 0x7fbd001407f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:33.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.747+0000 7fbd05200640 1 --2- 172.21.15.114:0/3429503186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd001534c0 0x7fbd001402b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:33.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.747+0000 7fbd05200640 1 --2- 172.21.15.114:0/3429503186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd000c2a00 0x7fbd000c54c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:33.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.747+0000 7fbd05200640 1 -- 172.21.15.114:0/3429503186 >> 172.21.15.114:0/3429503186 conn(0x7fbd000ca320 msgr2=0x7fbd000c2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:33.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.748+0000 7fbd05200640 1 -- 172.21.15.114:0/3429503186 shutdown_connections 2024-05-27T09:27:33.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:33.748+0000 7fbd05200640 1 -- 172.21.15.114:0/3429503186 wait complete. 2024-05-27T09:27:33.775 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T09:27:34.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.021+0000 7fe564e00640 1 Processor -- start 2024-05-27T09:27:34.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.022+0000 7fe564e00640 1 -- start start 2024-05-27T09:27:34.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.024+0000 7fe564e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5601534c0 0x7fe5600c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:34.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.024+0000 7fe564e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe5600c8220 0x7fe560159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:34.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.024+0000 7fe564e00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5600c96d0 con 0x7fe5600c8630 2024-05-27T09:27:34.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.024+0000 7fe564e00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fe5600c2f40 con 0x7fe5600c8220 2024-05-27T09:27:34.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.024+0000 7fe564e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fe560159190 con 0x7fe5601534c0 2024-05-27T09:27:34.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.024+0000 7fe55ea00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe5600c8220 0x7fe560159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:34.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.024+0000 7fe55ea00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe5600c8220 0x7fe560159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35910/0 (socket says 172.21.15.114:35910) 2024-05-27T09:27:34.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.024+0000 7fe55ea00640 1 -- 172.21.15.114:0/1834342161 learned_addr learned my addr 172.21.15.114:0/1834342161 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:34.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.024+0000 7fe55f400640 1 --2- 172.21.15.114:0/1834342161 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5601534c0 0x7fe5600c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:34.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.024+0000 7fe55ea00640 1 -- 172.21.15.114:0/1834342161 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5601534c0 msgr2=0x7fe5600c2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:34.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.024+0000 7fe55ea00640 1 --2- 172.21.15.114:0/1834342161 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5601534c0 0x7fe5600c2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:34.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.024+0000 7fe55ea00640 1 -- 172.21.15.114:0/1834342161 >> v1:172.21.15.163:6789/0 conn(0x7fe5600c8630 legacy=0x7fe560074140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:34.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.024+0000 7fe55ea00640 1 -- 172.21.15.114:0/1834342161 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5600c9970 con 0x7fe5600c8220 2024-05-27T09:27:34.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.025+0000 7fe55ea00640 1 --2- 172.21.15.114:0/1834342161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5600c8220 0x7fe560159900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe548009870 tx=0x7fe54802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f625fa5a9625c4d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:34.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.026+0000 7fe55e000640 1 -- 172.21.15.114:0/1834342161 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548037070 con 0x7fe5600c8220 2024-05-27T09:27:34.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.026+0000 7fe55e000640 1 -- 172.21.15.114:0/1834342161 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe548032440 con 0x7fe5600c8220 2024-05-27T09:27:34.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.026+0000 7fe55e000640 1 -- 172.21.15.114:0/1834342161 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548032810 con 0x7fe5600c8220 2024-05-27T09:27:34.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.026+0000 7fe564e00640 1 -- 172.21.15.114:0/1834342161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5600c8220 msgr2=0x7fe560159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:34.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.026+0000 7fe564e00640 1 --2- 172.21.15.114:0/1834342161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5600c8220 0x7fe560159900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe548009870 tx=0x7fe54802a250 comp rx=0 tx=0).stop 2024-05-27T09:27:34.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.026+0000 7fe564e00640 1 -- 172.21.15.114:0/1834342161 shutdown_connections 2024-05-27T09:27:34.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.026+0000 7fe564e00640 1 --2- 172.21.15.114:0/1834342161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5600c8220 0x7fe560159900 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:34.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.026+0000 7fe564e00640 1 --2- 172.21.15.114:0/1834342161 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5601534c0 0x7fe5600c2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:34.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.026+0000 7fe564e00640 1 -- 172.21.15.114:0/1834342161 >> 172.21.15.114:0/1834342161 conn(0x7fe5600ca320 msgr2=0x7fe5600ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:34.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.027+0000 7fe564e00640 1 -- 172.21.15.114:0/1834342161 shutdown_connections 2024-05-27T09:27:34.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.027+0000 7fe564e00640 1 -- 172.21.15.114:0/1834342161 wait complete. 2024-05-27T09:27:34.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.029+0000 7fe564e00640 1 Processor -- start 2024-05-27T09:27:34.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.029+0000 7fe564e00640 1 -- start start 2024-05-27T09:27:34.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.030+0000 7fe564e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5600c8220 0x7fe5600c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:34.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.030+0000 7fe564e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5600c8630 0x7fe56014a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:34.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.030+0000 7fe564e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5601534c0 0x7fe560148d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:34.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.030+0000 7fe564e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe5600c9f30 con 0x7fe5600c8630 2024-05-27T09:27:34.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.030+0000 7fe564e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe5601538d0 con 0x7fe5601534c0 2024-05-27T09:27:34.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.030+0000 7fe564e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fe5601593e0 con 0x7fe5600c8220 2024-05-27T09:27:34.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.030+0000 7fe55ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5600c8630 0x7fe56014a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:34.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.031+0000 7fe55ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5600c8630 0x7fe56014a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35922/0 (socket says 172.21.15.114:35922) 2024-05-27T09:27:34.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.031+0000 7fe55ea00640 1 -- 172.21.15.114:0/1718656726 learned_addr learned my addr 172.21.15.114:0/1718656726 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:34.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.031+0000 7fe55fe00640 1 --2- 172.21.15.114:0/1718656726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5601534c0 0x7fe560148d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:34.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.031+0000 7fe55f400640 1 --2- 172.21.15.114:0/1718656726 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5600c8220 0x7fe5600c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:34.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.031+0000 7fe55ea00640 1 -- 172.21.15.114:0/1718656726 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5600c8220 msgr2=0x7fe5600c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:34.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.031+0000 7fe55ea00640 1 --2- 172.21.15.114:0/1718656726 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5600c8220 0x7fe5600c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:34.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.031+0000 7fe55ea00640 1 -- 172.21.15.114:0/1718656726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5601534c0 msgr2=0x7fe560148d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:34.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.031+0000 7fe55ea00640 1 --2- 172.21.15.114:0/1718656726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5601534c0 0x7fe560148d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:34.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.031+0000 7fe55ea00640 1 -- 172.21.15.114:0/1718656726 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5601f6370 con 0x7fe5600c8630 2024-05-27T09:27:34.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.031+0000 7fe55ea00640 1 --2- 172.21.15.114:0/1718656726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5600c8630 0x7fe56014a450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe548000c00 tx=0x7fe5480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:34.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.031+0000 7fe557400640 1 -- 172.21.15.114:0/1718656726 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548037040 con 0x7fe5600c8630 2024-05-27T09:27:34.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.032+0000 7fe557400640 1 -- 172.21.15.114:0/1718656726 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe548042070 con 0x7fe5600c8630 2024-05-27T09:27:34.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.032+0000 7fe564e00640 1 -- 172.21.15.114:0/1718656726 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe560150440 con 0x7fe5600c8630 2024-05-27T09:27:34.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.032+0000 7fe557400640 1 -- 172.21.15.114:0/1718656726 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548030880 con 0x7fe5600c8630 2024-05-27T09:27:34.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.032+0000 7fe564e00640 1 -- 172.21.15.114:0/1718656726 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5601f6610 con 0x7fe5600c8630 2024-05-27T09:27:34.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.033+0000 7fe564e00640 1 -- 172.21.15.114:0/1718656726 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe56014a990 con 0x7fe5600c8630 2024-05-27T09:27:34.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.034+0000 7fe557400640 1 -- 172.21.15.114:0/1718656726 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe548030a20 con 0x7fe5600c8630 2024-05-27T09:27:34.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.035+0000 7fe557400640 1 --2- 172.21.15.114:0/1718656726 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe53c076300 0x7fe53c078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:34.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.035+0000 7fe557400640 1 -- 172.21.15.114:0/1718656726 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe5480b7d80 con 0x7fe5600c8630 2024-05-27T09:27:34.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.035+0000 7fe55f400640 1 --2- 172.21.15.114:0/1718656726 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe53c076300 0x7fe53c078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:34.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.036+0000 7fe55f400640 1 --2- 172.21.15.114:0/1718656726 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe53c076300 0x7fe53c078810 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe550011340 tx=0x7fe55000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:34.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.040+0000 7fe557400640 1 -- 172.21.15.114:0/1718656726 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe548080ff0 con 0x7fe5600c8630 2024-05-27T09:27:34.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.230+0000 7fe564e00640 1 -- 172.21.15.114:0/1718656726 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe560050860 con 0x7fe5600c8630 2024-05-27T09:27:34.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.233+0000 7fe557400640 1 -- 172.21.15.114:0/1718656726 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16645 (secure 0 0 0) 0x7fe5480852f0 con 0x7fe5600c8630 2024-05-27T09:27:34.234 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:34.234 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"fsid":"2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48","created":"2024-05-27T09:27:00.307278+0000","modified":"2024-05-27T09:27:19.530054+0000","last_up_change":"2024-05-27T09:27:11.463226+0000","last_in_change":"2024-05-27T09:27:02.972880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T09:27:10.796859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T09:27:16.063199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T09:27:17.064857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3561c750-1d63-42f6-a6ae-0ee63721e15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6817","nonce":1960550710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6819","nonce":1960550710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6824","nonce":1960550710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6821","nonce":1960550710}]},"public_addr":"172.21.15.114:6817/1960550710","cluster_addr":"172.21.15.114:6819/1960550710","heartbeat_back_addr":"172.21.15.114:6824/1960550710","heartbeat_front_addr":"172.21.15.114:6821/1960550710","state":["exists","up"]},{"osd":1,"uuid":"9a407d04-38c0-4676-928f-d04e8116bd19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6825","nonce":326580057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6827","nonce":326580057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6831","nonce":326580057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6829","nonce":326580057}]},"public_addr":"172.21.15.114:6825/326580057","cluster_addr":"172.21.15.114:6827/326580057","heartbeat_back_addr":"172.21.15.114:6831/326580057","heartbeat_front_addr":"172.21.15.114:6829/326580057","state":["exists","up"]},{"osd":2,"uuid":"255a1cc6-9357-43f7-9db5-d760f4806ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6809","nonce":865568619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6811","nonce":865568619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6815","nonce":865568619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6813","nonce":865568619}]},"public_addr":"172.21.15.114:6809/865568619","cluster_addr":"172.21.15.114:6811/865568619","heartbeat_back_addr":"172.21.15.114:6815/865568619","heartbeat_front_addr":"172.21.15.114:6813/865568619","state":["exists","up"]},{"osd":3,"uuid":"cb4d7692-9a17-41b9-927e-d8eac3e217c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6801","nonce":1601792164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6803","nonce":1601792164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6807","nonce":1601792164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6805","nonce":1601792164}]},"public_addr":"172.21.15.114:6801/1601792164","cluster_addr":"172.21.15.114:6803/1601792164","heartbeat_back_addr":"172.21.15.114:6807/1601792164","heartbeat_front_addr":"172.21.15.114:6805/1601792164","state":["exists","up"]},{"osd":4,"uuid":"cdef1940-3453-4fbd-b0f1-459f5f5c7142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6817","nonce":1982246015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6819","nonce":1982246015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6823","nonce":1982246015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6821","nonce":1982246015}]},"public_addr":"172.21.15.163:6817/1982246015","cluster_addr":"172.21.15.163:6819/1982246015","heartbeat_back_addr":"172.21.15.163:6823/1982246015","heartbeat_front_addr":"172.21.15.163:6821/1982246015","state":["exists","up"]},{"osd":5,"uuid":"220a7b32-abcc-492f-bca8-4615cc2ea111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6801","nonce":1695294004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6803","nonce":1695294004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6807","nonce":1695294004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6805","nonce":1695294004}]},"public_addr":"172.21.15.163:6801/1695294004","cluster_addr":"172.21.15.163:6803/1695294004","heartbeat_back_addr":"172.21.15.163:6807/1695294004","heartbeat_front_addr":"172.21.15.163:6805/1695294004","state":["exists","up"]},{"osd":6,"uuid":"43a74d13-d03f-493a-a6c0-e73b0a056c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6809","nonce":3649718505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6811","nonce":3649718505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6815","nonce":3649718505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6813","nonce":3649718505}]},"public_addr":"172.21.15.163:6809/3649718505","cluster_addr":"172.21.15.163:6811/3649718505","heartbeat_back_addr":"172.21.15.163:6815/3649718505","heartbeat_front_addr":"172.21.15.163:6813/3649718505","state":["exists","up"]},{"osd":7,"uuid":"134629c2-70af-4b84-83a6-7ddf53681baa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6825","nonce":2539451311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6827","nonce":2539451311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6831","nonce":2539451311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6829","nonce":2539451311}]},"public_addr":"172.21.15.163:6825/2539451311","cluster_addr":"172.21.15.163:6827/2539451311","heartbeat_back_addr":"172.21.15.163:6831/2539451311","heartbeat_front_addr":"172.21.15.163:6829/2539451311","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.388942+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.515675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.646761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.400212+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.220072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.079531+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.180419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.175008+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T09:27:34.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.235+0000 7fe564e00640 1 -- 172.21.15.114:0/1718656726 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe53c076300 msgr2=0x7fe53c078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:34.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.235+0000 7fe564e00640 1 --2- 172.21.15.114:0/1718656726 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe53c076300 0x7fe53c078810 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe550011340 tx=0x7fe55000b040 comp rx=0 tx=0).stop 2024-05-27T09:27:34.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.235+0000 7fe564e00640 1 -- 172.21.15.114:0/1718656726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5600c8630 msgr2=0x7fe56014a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:34.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.235+0000 7fe564e00640 1 --2- 172.21.15.114:0/1718656726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5600c8630 0x7fe56014a450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe548000c00 tx=0x7fe5480046c0 comp rx=0 tx=0).stop 2024-05-27T09:27:34.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.235+0000 7fe564e00640 1 -- 172.21.15.114:0/1718656726 shutdown_connections 2024-05-27T09:27:34.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.235+0000 7fe564e00640 1 --2- 172.21.15.114:0/1718656726 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe53c076300 0x7fe53c078810 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:34.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.235+0000 7fe564e00640 1 --2- 172.21.15.114:0/1718656726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5601534c0 0x7fe560148d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:34.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.235+0000 7fe564e00640 1 --2- 172.21.15.114:0/1718656726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5600c8630 0x7fe56014a450 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:34.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.235+0000 7fe564e00640 1 --2- 172.21.15.114:0/1718656726 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5600c8220 0x7fe5600c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:34.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.235+0000 7fe564e00640 1 -- 172.21.15.114:0/1718656726 >> 172.21.15.114:0/1718656726 conn(0x7fe5600ca320 msgr2=0x7fe56014c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:34.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.235+0000 7fe564e00640 1 -- 172.21.15.114:0/1718656726 shutdown_connections 2024-05-27T09:27:34.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.236+0000 7fe564e00640 1 -- 172.21.15.114:0/1718656726 wait complete. 2024-05-27T09:27:34.259 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:27:34.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.501+0000 7f5ccae00640 1 Processor -- start 2024-05-27T09:27:34.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.502+0000 7f5ccae00640 1 -- start start 2024-05-27T09:27:34.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.504+0000 7f5ccae00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cc4074860 0x7f5cc40c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:34.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.504+0000 7f5ccae00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5cc4160b40 0x7f5cc4160f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:34.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.504+0000 7f5ccae00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cc41615d0 con 0x7f5cc4164db0 2024-05-27T09:27:34.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.504+0000 7f5ccae00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f5cc40cade0 con 0x7f5cc4160b40 2024-05-27T09:27:34.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.504+0000 7f5ccae00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f5cc40c3d60 con 0x7f5cc4074860 2024-05-27T09:27:34.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.504+0000 7f5cc3e00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f5cc4164db0 0x7f5cc415aac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:46700/0 (socket says 172.21.15.114:46700) 2024-05-27T09:27:34.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.504+0000 7f5cc3e00640 1 -- 172.21.15.114:0/3530474490 learned_addr learned my addr 172.21.15.114:0/3530474490 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:34.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.504+0000 7f5cc3400640 1 --2- 172.21.15.114:0/3530474490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cc4074860 0x7f5cc40c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:34.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.504+0000 7f5cc8e00640 1 --2- 172.21.15.114:0/3530474490 >> v2:172.21.15.114:3300/0 conn(0x7f5cc4160b40 0x7f5cc4160f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:34.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.504+0000 7f5cc3400640 1 -- 172.21.15.114:0/3530474490 >> v2:172.21.15.114:3300/0 conn(0x7f5cc4160b40 msgr2=0x7f5cc4160f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:34.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.504+0000 7f5cc3400640 1 --2- 172.21.15.114:0/3530474490 >> v2:172.21.15.114:3300/0 conn(0x7f5cc4160b40 0x7f5cc4160f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:34.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.504+0000 7f5cc3400640 1 -- 172.21.15.114:0/3530474490 >> v1:172.21.15.114:6789/0 conn(0x7f5cc4164db0 legacy=0x7f5cc415aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:34.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.504+0000 7f5cc2a00640 1 -- 172.21.15.114:0/3530474490 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4259981002 0 0) 0x7f5cc41615d0 con 0x7f5cc4164db0 2024-05-27T09:27:34.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.504+0000 7f5cc3400640 1 -- 172.21.15.114:0/3530474490 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc4161890 con 0x7f5cc4074860 2024-05-27T09:27:34.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.505+0000 7f5cc3400640 1 --2- 172.21.15.114:0/3530474490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cc4074860 0x7f5cc40c3820 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009d90 tx=0x7f5cb402a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8fe7a9a324f4b1c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:34.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.506+0000 7f5cc2a00640 1 -- 172.21.15.114:0/3530474490 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4007740 con 0x7f5cc4074860 2024-05-27T09:27:34.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.506+0000 7f5cc2a00640 1 -- 172.21.15.114:0/3530474490 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cb40078e0 con 0x7f5cc4074860 2024-05-27T09:27:34.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.506+0000 7f5cc2a00640 1 -- 172.21.15.114:0/3530474490 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4007cb0 con 0x7f5cc4074860 2024-05-27T09:27:34.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.506+0000 7f5ccae00640 1 -- 172.21.15.114:0/3530474490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cc4074860 msgr2=0x7f5cc40c3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:34.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.506+0000 7f5ccae00640 1 --2- 172.21.15.114:0/3530474490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cc4074860 0x7f5cc40c3820 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009d90 tx=0x7f5cb402a6f0 comp rx=0 tx=0).stop 2024-05-27T09:27:34.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.507+0000 7f5ccae00640 1 -- 172.21.15.114:0/3530474490 shutdown_connections 2024-05-27T09:27:34.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.507+0000 7f5ccae00640 1 --2- 172.21.15.114:0/3530474490 >> v2:172.21.15.114:3300/0 conn(0x7f5cc4160b40 0x7f5cc4160f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:34.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.507+0000 7f5ccae00640 1 --2- 172.21.15.114:0/3530474490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cc4074860 0x7f5cc40c3820 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:34.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.507+0000 7f5ccae00640 1 -- 172.21.15.114:0/3530474490 >> 172.21.15.114:0/3530474490 conn(0x7f5cc406f6e0 msgr2=0x7f5cc4070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:34.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.507+0000 7f5ccae00640 1 -- 172.21.15.114:0/3530474490 shutdown_connections 2024-05-27T09:27:34.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.507+0000 7f5ccae00640 1 -- 172.21.15.114:0/3530474490 wait complete. 2024-05-27T09:27:34.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.509+0000 7f5ccae00640 1 Processor -- start 2024-05-27T09:27:34.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.509+0000 7f5ccae00640 1 -- start start 2024-05-27T09:27:34.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.510+0000 7f5ccae00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cc4074860 0x7f5cc415df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:34.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.510+0000 7f5ccae00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cc4160b40 0x7f5cc40cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:34.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.510+0000 7f5ccae00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cc4164db0 0x7f5cc40c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:34.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.510+0000 7f5ccae00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc4164890 con 0x7f5cc4074860 2024-05-27T09:27:34.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.510+0000 7f5ccae00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc41645b0 con 0x7f5cc4164db0 2024-05-27T09:27:34.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.510+0000 7f5ccae00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f5cc4073310 con 0x7f5cc4160b40 2024-05-27T09:27:34.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.511+0000 7f5cc3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cc4074860 0x7f5cc415df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:34.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.511+0000 7f5cc3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cc4074860 0x7f5cc415df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35936/0 (socket says 172.21.15.114:35936) 2024-05-27T09:27:34.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.511+0000 7f5cc3e00640 1 -- 172.21.15.114:0/967509568 learned_addr learned my addr 172.21.15.114:0/967509568 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:34.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.511+0000 7f5cc3400640 1 --2- 172.21.15.114:0/967509568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cc4160b40 0x7f5cc40cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:34.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.511+0000 7f5cc8e00640 1 --2- 172.21.15.114:0/967509568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cc4164db0 0x7f5cc40c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:34.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.511+0000 7f5cc3e00640 1 -- 172.21.15.114:0/967509568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cc4160b40 msgr2=0x7f5cc40cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:34.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.511+0000 7f5cc3e00640 1 --2- 172.21.15.114:0/967509568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cc4160b40 0x7f5cc40cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:34.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.511+0000 7f5cc3e00640 1 -- 172.21.15.114:0/967509568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cc4164db0 msgr2=0x7f5cc40c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:34.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.511+0000 7f5cc3e00640 1 --2- 172.21.15.114:0/967509568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cc4164db0 0x7f5cc40c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:34.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.511+0000 7f5cc3e00640 1 -- 172.21.15.114:0/967509568 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc41f6520 con 0x7f5cc4074860 2024-05-27T09:27:34.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.511+0000 7f5cc3e00640 1 --2- 172.21.15.114:0/967509568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cc4074860 0x7f5cc415df20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00c410 tx=0x7f5cac00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:34.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.512+0000 7f5cbbe00640 1 -- 172.21.15.114:0/967509568 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0074e0 con 0x7f5cc4074860 2024-05-27T09:27:34.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.512+0000 7f5cbbe00640 1 -- 172.21.15.114:0/967509568 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cac0047e0 con 0x7f5cc4074860 2024-05-27T09:27:34.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.512+0000 7f5ccae00640 1 -- 172.21.15.114:0/967509568 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc41f67e0 con 0x7f5cc4074860 2024-05-27T09:27:34.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.512+0000 7f5cbbe00640 1 -- 172.21.15.114:0/967509568 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac004bd0 con 0x7f5cc4074860 2024-05-27T09:27:34.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.512+0000 7f5ccae00640 1 -- 172.21.15.114:0/967509568 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc41f6da0 con 0x7f5cc4074860 2024-05-27T09:27:34.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.513+0000 7f5ccae00640 1 -- 172.21.15.114:0/967509568 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc406a380 con 0x7f5cc4074860 2024-05-27T09:27:34.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.514+0000 7f5cbbe00640 1 -- 172.21.15.114:0/967509568 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5cac011070 con 0x7f5cc4074860 2024-05-27T09:27:34.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.515+0000 7f5cbbe00640 1 --2- 172.21.15.114:0/967509568 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5ca00762d0 0x7f5ca00787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:34.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.515+0000 7f5cbbe00640 1 -- 172.21.15.114:0/967509568 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5cac097230 con 0x7f5cc4074860 2024-05-27T09:27:34.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.515+0000 7f5cc3400640 1 --2- 172.21.15.114:0/967509568 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5ca00762d0 0x7f5ca00787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:34.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.516+0000 7f5cc3400640 1 --2- 172.21.15.114:0/967509568 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5ca00762d0 0x7f5ca00787e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4034000 tx=0x7f5cb402f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:34.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.520+0000 7f5cbbe00640 1 -- 172.21.15.114:0/967509568 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5cac060410 con 0x7f5cc4074860 2024-05-27T09:27:34.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.767+0000 7f5ccae00640 1 -- 172.21.15.114:0/967509568 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cc4073e30 con 0x7f5cc4074860 2024-05-27T09:27:34.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.768+0000 7f5cbbe00640 1 -- 172.21.15.114:0/967509568 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+6877 (secure 0 0 0) 0x7f5cac064710 con 0x7f5cc4074860 2024-05-27T09:27:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:34.770 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:27:33.841352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"f","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6839/444766663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":444766663},{"type":"v1","addr":"172.21.15.163:6839","nonce":444766663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4279,4297,4300]},"id":1}]} 2024-05-27T09:27:34.770 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 12 2024-05-27T09:27:34.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.771+0000 7f5ccae00640 1 -- 172.21.15.114:0/967509568 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5ca00762d0 msgr2=0x7f5ca00787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:34.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.771+0000 7f5ccae00640 1 --2- 172.21.15.114:0/967509568 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5ca00762d0 0x7f5ca00787e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4034000 tx=0x7f5cb402f000 comp rx=0 tx=0).stop 2024-05-27T09:27:34.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.771+0000 7f5ccae00640 1 -- 172.21.15.114:0/967509568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cc4074860 msgr2=0x7f5cc415df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:34.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.771+0000 7f5ccae00640 1 --2- 172.21.15.114:0/967509568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cc4074860 0x7f5cc415df20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00c410 tx=0x7f5cac00c8d0 comp rx=0 tx=0).stop 2024-05-27T09:27:34.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.771+0000 7f5ccae00640 1 -- 172.21.15.114:0/967509568 shutdown_connections 2024-05-27T09:27:34.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.771+0000 7f5ccae00640 1 --2- 172.21.15.114:0/967509568 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5ca00762d0 0x7f5ca00787e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:34.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.771+0000 7f5ccae00640 1 --2- 172.21.15.114:0/967509568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cc4164db0 0x7f5cc40c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:34.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.771+0000 7f5ccae00640 1 --2- 172.21.15.114:0/967509568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cc4160b40 0x7f5cc40cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:34.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.771+0000 7f5ccae00640 1 --2- 172.21.15.114:0/967509568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cc4074860 0x7f5cc415df20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:34.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.771+0000 7f5ccae00640 1 -- 172.21.15.114:0/967509568 >> 172.21.15.114:0/967509568 conn(0x7f5cc406f6e0 msgr2=0x7f5cc415fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:34.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.771+0000 7f5ccae00640 1 -- 172.21.15.114:0/967509568 shutdown_connections 2024-05-27T09:27:34.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:34.771+0000 7f5ccae00640 1 -- 172.21.15.114:0/967509568 wait complete. 2024-05-27T09:27:35.400 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 12, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-27T09:27:18.079214+0000', 'modified': '2024-05-27T09:27:33.841352+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4300, 'mds_1': 4297, 'mds_2': 4279}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4279': {'gid': 4279, 'name': 'f', 'rank': 2, 'incarnation': 11, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.163:6839/444766663', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6837', 'nonce': 444766663}, {'type': 'v1', 'addr': '172.21.15.163:6839', 'nonce': 444766663}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4297': {'gid': 4297, 'name': 'c', 'rank': 1, 'incarnation': 9, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.114:6834/657027519', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6832', 'nonce': 657027519}, {'type': 'v1', 'addr': '172.21.15.114:6834', 'nonce': 657027519}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4300': {'gid': 4300, 'name': 'e', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.114:6837/1448950017', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6836', 'nonce': 1448950017}, {'type': 'v1', 'addr': '172.21.15.114:6837', 'nonce': 1448950017}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4300, 'qdb_cluster': [4279, 4297, 4300]} 2024-05-27T09:27:35.401 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-27T09:27:35.401 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-05-27T09:27:35.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.682+0000 7f9c81800640 1 Processor -- start 2024-05-27T09:27:35.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.682+0000 7f9c81800640 1 -- start start 2024-05-27T09:27:35.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.684+0000 7f9c81800640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9c7c1547d0 0x7f9c7c0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:35.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.684+0000 7f9c81800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c7c074840 0x7f9c7c15ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:35.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.684+0000 7f9c81800640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c7c06fe50 con 0x7f9c7c0741f0 2024-05-27T09:27:35.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.684+0000 7f9c81800640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f9c7c0703d0 con 0x7f9c7c1547d0 2024-05-27T09:27:35.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.684+0000 7f9c81800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f9c7c15b260 con 0x7f9c7c074840 2024-05-27T09:27:35.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.684+0000 7f9c7be00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f9c7c0741f0 0x7f9c7c06edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:46904/0 (socket says 172.21.15.114:46904) 2024-05-27T09:27:35.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.684+0000 7f9c7be00640 1 -- 172.21.15.114:0/1422511796 learned_addr learned my addr 172.21.15.114:0/1422511796 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:35.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.685+0000 7f9c7b400640 1 --2- 172.21.15.114:0/1422511796 >> v2:172.21.15.163:3300/0 conn(0x7f9c7c1547d0 0x7f9c7c0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:35.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.685+0000 7f9c80e00640 1 --2- 172.21.15.114:0/1422511796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c7c074840 0x7f9c7c15ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:35.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.685+0000 7f9c7aa00640 1 -- 172.21.15.114:0/1422511796 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 960375556 0 0) 0x7f9c7c06fe50 con 0x7f9c7c0741f0 2024-05-27T09:27:35.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.685+0000 7f9c7aa00640 1 -- 172.21.15.114:0/1422511796 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c60003520 con 0x7f9c7c0741f0 2024-05-27T09:27:35.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.685+0000 7f9c7b400640 1 -- 172.21.15.114:0/1422511796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c7c074840 msgr2=0x7f9c7c15ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:35.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.685+0000 7f9c7b400640 1 --2- 172.21.15.114:0/1422511796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c7c074840 0x7f9c7c15ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:35.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.685+0000 7f9c7b400640 1 -- 172.21.15.114:0/1422511796 >> v1:172.21.15.114:6789/0 conn(0x7f9c7c0741f0 legacy=0x7f9c7c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:35.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.685+0000 7f9c7b400640 1 -- 172.21.15.114:0/1422511796 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7c0700f0 con 0x7f9c7c1547d0 2024-05-27T09:27:35.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.685+0000 7f9c80e00640 1 --2- 172.21.15.114:0/1422511796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c7c074840 0x7f9c7c15ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:35.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.686+0000 7f9c7b400640 1 --2- 172.21.15.114:0/1422511796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c7c1547d0 0x7f9c7c0c63a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9c680097a0 tx=0x7f9c6802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44e3ad29eb6d97c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:35.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.686+0000 7f9c7aa00640 1 -- 172.21.15.114:0/1422511796 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68037070 con 0x7f9c7c1547d0 2024-05-27T09:27:35.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.687+0000 7f9c7aa00640 1 -- 172.21.15.114:0/1422511796 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c68032440 con 0x7f9c7c1547d0 2024-05-27T09:27:35.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.687+0000 7f9c7aa00640 1 -- 172.21.15.114:0/1422511796 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c680327f0 con 0x7f9c7c1547d0 2024-05-27T09:27:35.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.687+0000 7f9c81800640 1 -- 172.21.15.114:0/1422511796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c7c1547d0 msgr2=0x7f9c7c0c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:35.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.687+0000 7f9c81800640 1 --2- 172.21.15.114:0/1422511796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c7c1547d0 0x7f9c7c0c63a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9c680097a0 tx=0x7f9c6802a210 comp rx=0 tx=0).stop 2024-05-27T09:27:35.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.687+0000 7f9c81800640 1 -- 172.21.15.114:0/1422511796 shutdown_connections 2024-05-27T09:27:35.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.687+0000 7f9c81800640 1 --2- 172.21.15.114:0/1422511796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c7c074840 0x7f9c7c15ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:35.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.687+0000 7f9c81800640 1 --2- 172.21.15.114:0/1422511796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c7c1547d0 0x7f9c7c0c63a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:35.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.687+0000 7f9c81800640 1 -- 172.21.15.114:0/1422511796 >> 172.21.15.114:0/1422511796 conn(0x7f9c7c0ca320 msgr2=0x7f9c7c0ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:35.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.687+0000 7f9c81800640 1 -- 172.21.15.114:0/1422511796 shutdown_connections 2024-05-27T09:27:35.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.688+0000 7f9c81800640 1 -- 172.21.15.114:0/1422511796 wait complete. 2024-05-27T09:27:35.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.690+0000 7f9c81800640 1 Processor -- start 2024-05-27T09:27:35.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.690+0000 7f9c81800640 1 -- start start 2024-05-27T09:27:35.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.691+0000 7f9c81800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c7c0741f0 0x7f9c7c1566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:35.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.691+0000 7f9c81800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c7c074840 0x7f9c7c155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:35.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.691+0000 7f9c81800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c7c1547d0 0x7f9c7c0cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:35.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.691+0000 7f9c81800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7c073790 con 0x7f9c7c0741f0 2024-05-27T09:27:35.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.691+0000 7f9c81800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7c0734b0 con 0x7f9c7c074840 2024-05-27T09:27:35.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.691+0000 7f9c81800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f9c7c06f500 con 0x7f9c7c1547d0 2024-05-27T09:27:35.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.692+0000 7f9c7be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c7c0741f0 0x7f9c7c1566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:35.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.692+0000 7f9c7be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c7c0741f0 0x7f9c7c1566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47796/0 (socket says 172.21.15.114:47796) 2024-05-27T09:27:35.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.692+0000 7f9c7be00640 1 -- 172.21.15.114:0/926864122 learned_addr learned my addr 172.21.15.114:0/926864122 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:35.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.692+0000 7f9c7b400640 1 --2- 172.21.15.114:0/926864122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c7c074840 0x7f9c7c155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:35.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.692+0000 7f9c80e00640 1 --2- 172.21.15.114:0/926864122 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c7c1547d0 0x7f9c7c0cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:35.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.692+0000 7f9c7be00640 1 -- 172.21.15.114:0/926864122 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c7c1547d0 msgr2=0x7f9c7c0cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:35.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.692+0000 7f9c7be00640 1 --2- 172.21.15.114:0/926864122 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c7c1547d0 0x7f9c7c0cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:35.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.692+0000 7f9c7be00640 1 -- 172.21.15.114:0/926864122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c7c074840 msgr2=0x7f9c7c155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:35.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.692+0000 7f9c7be00640 1 --2- 172.21.15.114:0/926864122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c7c074840 0x7f9c7c155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:35.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.692+0000 7f9c7be00640 1 -- 172.21.15.114:0/926864122 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7c156be0 con 0x7f9c7c0741f0 2024-05-27T09:27:35.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.692+0000 7f9c7be00640 1 --2- 172.21.15.114:0/926864122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c7c0741f0 0x7f9c7c1566a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c00c410 tx=0x7f9c6c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:35.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.693+0000 7f9c73e00640 1 -- 172.21.15.114:0/926864122 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c017070 con 0x7f9c7c0741f0 2024-05-27T09:27:35.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.693+0000 7f9c73e00640 1 -- 172.21.15.114:0/926864122 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c6c005ce0 con 0x7f9c7c0741f0 2024-05-27T09:27:35.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.693+0000 7f9c81800640 1 -- 172.21.15.114:0/926864122 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c7c0c8e20 con 0x7f9c7c0741f0 2024-05-27T09:27:35.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.693+0000 7f9c73e00640 1 -- 172.21.15.114:0/926864122 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c0072d0 con 0x7f9c7c0741f0 2024-05-27T09:27:35.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.693+0000 7f9c81800640 1 -- 172.21.15.114:0/926864122 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c7c14e170 con 0x7f9c7c0741f0 2024-05-27T09:27:35.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.695+0000 7f9c73e00640 1 -- 172.21.15.114:0/926864122 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9c6c01a070 con 0x7f9c7c0741f0 2024-05-27T09:27:35.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.696+0000 7f9c73e00640 1 --2- 172.21.15.114:0/926864122 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9c5c0762d0 0x7f9c5c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:35.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.696+0000 7f9c73e00640 1 -- 172.21.15.114:0/926864122 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9c6c096880 con 0x7f9c7c0741f0 2024-05-27T09:27:35.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.696+0000 7f9c7b400640 1 --2- 172.21.15.114:0/926864122 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9c5c0762d0 0x7f9c5c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:35.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.697+0000 7f9c89dc2740 1 Processor -- start 2024-05-27T09:27:35.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.697+0000 7f9c7b400640 1 --2- 172.21.15.114:0/926864122 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9c5c0762d0 0x7f9c5c0787e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9c68034000 tx=0x7f9c6802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:35.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.697+0000 7f9c89dc2740 1 -- start start 2024-05-27T09:27:35.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.698+0000 7f9c89dc2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422e4270 0x562142685fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:35.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.698+0000 7f9c89dc2740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5621422bb3b0 0x5621426c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:35.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.698+0000 7f9c89dc2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422c3590 0x5621426aa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:35.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.698+0000 7f9c89dc2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5621425f7df0 con 0x5621422c3590 2024-05-27T09:27:35.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.698+0000 7f9c89dc2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5621425e7990 con 0x5621422e4270 2024-05-27T09:27:35.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.698+0000 7f9c89dc2740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x5621425eb9d0 con 0x5621422bb3b0 2024-05-27T09:27:35.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.699+0000 7f9c80e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422c3590 0x5621426aa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:35.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.699+0000 7f9c7b400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5621422bb3b0 0x5621426c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:35.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.699+0000 7f9c7be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422e4270 0x562142685fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:35.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.699+0000 7f9c80e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422c3590 0x5621426aa740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47800/0 (socket says 172.21.15.114:47800) 2024-05-27T09:27:35.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.699+0000 7f9c80e00640 1 -- 172.21.15.114:0/2608756770 learned_addr learned my addr 172.21.15.114:0/2608756770 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:35.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.699+0000 7f9c80e00640 1 -- 172.21.15.114:0/2608756770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5621422bb3b0 msgr2=0x5621426c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:35.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.699+0000 7f9c80e00640 1 --2- 172.21.15.114:0/2608756770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5621422bb3b0 0x5621426c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:35.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.699+0000 7f9c80e00640 1 -- 172.21.15.114:0/2608756770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422e4270 msgr2=0x562142685fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:35.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.699+0000 7f9c80e00640 1 --2- 172.21.15.114:0/2608756770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422e4270 0x562142685fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:35.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.699+0000 7f9c80e00640 1 -- 172.21.15.114:0/2608756770 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621425889b0 con 0x5621422c3590 2024-05-27T09:27:35.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.699+0000 7f9c7be00640 1 --2- 172.21.15.114:0/2608756770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422e4270 0x562142685fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:35.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.700+0000 7f9c80e00640 1 --2- 172.21.15.114:0/2608756770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422c3590 0x5621426aa740 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9c74011390 tx=0x7f9c74009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f111a1d1b4eb47d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:35.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.700+0000 7f9c70c00640 1 -- 172.21.15.114:0/2608756770 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74003b90 con 0x5621422c3590 2024-05-27T09:27:35.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.700+0000 7f9c70c00640 1 -- 172.21.15.114:0/2608756770 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c74003d30 con 0x5621422c3590 2024-05-27T09:27:35.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.700+0000 7f9c70c00640 1 -- 172.21.15.114:0/2608756770 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74015940 con 0x5621422c3590 2024-05-27T09:27:35.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.700+0000 7f9c89dc2740 1 -- 172.21.15.114:0/2608756770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422c3590 msgr2=0x5621426aa740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:35.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.701+0000 7f9c89dc2740 1 --2- 172.21.15.114:0/2608756770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422c3590 0x5621426aa740 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9c74011390 tx=0x7f9c74009750 comp rx=0 tx=0).stop 2024-05-27T09:27:35.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.701+0000 7f9c89dc2740 1 -- 172.21.15.114:0/2608756770 shutdown_connections 2024-05-27T09:27:35.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.701+0000 7f9c89dc2740 1 --2- 172.21.15.114:0/2608756770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422c3590 0x5621426aa740 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:35.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.701+0000 7f9c89dc2740 1 --2- 172.21.15.114:0/2608756770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5621422bb3b0 0x5621426c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:35.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.701+0000 7f9c89dc2740 1 --2- 172.21.15.114:0/2608756770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422e4270 0x562142685fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:35.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.701+0000 7f9c89dc2740 1 -- 172.21.15.114:0/2608756770 >> 172.21.15.114:0/2608756770 conn(0x5621422d83c0 msgr2=0x562142684ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:35.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.701+0000 7f9c89dc2740 1 -- 172.21.15.114:0/2608756770 shutdown_connections 2024-05-27T09:27:35.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.701+0000 7f9c89dc2740 1 -- 172.21.15.114:0/2608756770 wait complete. 2024-05-27T09:27:35.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.702+0000 7f9c89dc2740 1 -- start start 2024-05-27T09:27:35.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.702+0000 7f9c89dc2740 1 Processor -- start 2024-05-27T09:27:35.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.703+0000 7f9c89dc2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422bb3b0 0x5621426c04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:35.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.703+0000 7f9c89dc2740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5621422c3590 0x5621426bf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:35.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.703+0000 7f9c89dc2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422e4270 0x5621426cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:35.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.704+0000 7f9c89dc2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5621422cd850 con 0x5621422e4270 2024-05-27T09:27:35.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.704+0000 7f9c89dc2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56214258ff20 con 0x5621422bb3b0 2024-05-27T09:27:35.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.704+0000 7f9c89dc2740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x5621426bfa50 con 0x5621422c3590 2024-05-27T09:27:35.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.704+0000 7f9c89dc2740 10 client.0 _ll_register_callbacks cb 0x5621425e1ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T09:27:35.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.704+0000 7f9c80e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422e4270 0x5621426cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:35.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.704+0000 7f9c80e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422e4270 0x5621426cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47804/0 (socket says 172.21.15.114:47804) 2024-05-27T09:27:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.704+0000 7f9c80e00640 1 -- 172.21.15.114:0/2556992165 learned_addr learned my addr 172.21.15.114:0/2556992165 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.704+0000 7f9c7b400640 1 --2- 172.21.15.114:0/2556992165 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5621422c3590 0x5621426bf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.704+0000 7f9c7be00640 1 --2- 172.21.15.114:0/2556992165 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422bb3b0 0x5621426c04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.704+0000 7f9c80e00640 1 -- 172.21.15.114:0/2556992165 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5621422c3590 msgr2=0x5621426bf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.704+0000 7f9c80e00640 1 --2- 172.21.15.114:0/2556992165 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5621422c3590 0x5621426bf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.704+0000 7f9c80e00640 1 -- 172.21.15.114:0/2556992165 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422bb3b0 msgr2=0x5621426c04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.704+0000 7f9c80e00640 1 --2- 172.21.15.114:0/2556992165 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422bb3b0 0x5621426c04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.704+0000 7f9c80e00640 1 -- 172.21.15.114:0/2556992165 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562142580d00 con 0x5621422e4270 2024-05-27T09:27:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.705+0000 7f9c80e00640 1 --2- 172.21.15.114:0/2556992165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422e4270 0x5621426cb3e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9c74011ce0 tx=0x7f9c74004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc80e60fb300234c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.705+0000 7f9c65600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-05-27T09:27:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.705+0000 7f9c65600640 1 -- 172.21.15.114:0/2556992165 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74009d80 con 0x5621422e4270 2024-05-27T09:27:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.705+0000 7f9c65600640 1 -- 172.21.15.114:0/2556992165 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9c74015bf0 con 0x5621422e4270 2024-05-27T09:27:35.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.705+0000 7f9c89dc2740 1 -- 172.21.15.114:0/2556992165 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c74003d30 con 0x5621422e4270 2024-05-27T09:27:35.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.705+0000 7f9c65600640 1 -- 172.21.15.114:0/2556992165 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) ==== 24+0+0 (secure 0 0 0) 0x7f9c74015df0 con 0x5621422e4270 2024-05-27T09:27:35.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.705+0000 7f9c89dc2740 10 client.4518 fetch_fsmap learned FSMap version 12 2024-05-27T09:27:35.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.705+0000 7f9c89dc2740 1 -- 172.21.15.114:0/2556992165 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) -- 0x5621422fc2d0 con 0x5621422e4270 2024-05-27T09:27:35.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.705+0000 7f9c65600640 1 -- 172.21.15.114:0/2556992165 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c74003d30 con 0x5621422e4270 2024-05-27T09:27:35.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.705+0000 7f9c65600640 1 -- 172.21.15.114:0/2556992165 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 12) ==== 4744+0+0 (secure 0 0 0) 0x7f9c74030050 con 0x5621422e4270 2024-05-27T09:27:35.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.706+0000 7f9c89dc2740 10 client.4518 fetch_fsmap finished waiting for FSMap version 12 2024-05-27T09:27:35.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.706+0000 7f9c89dc2740 10 client.4518 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-27T09:27:35.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.706+0000 7f9c89dc2740 20 client.4518 populate_metadata read hostname 'smithi114' 2024-05-27T09:27:35.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.706+0000 7f9c89dc2740 1 --2- 172.21.15.114:0/2556992165 >> [v2:172.21.15.163:6837/444766663,v1:172.21.15.163:6839/444766663] conn(0x5621422a2e40 0x5621426e4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:35.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.706+0000 7f9c89dc2740 4 client.4518 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T09:27:35.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.706+0000 7f9c89dc2740 1 -- 172.21.15.114:0/2556992165 --> [v2:172.21.15.163:6837/444766663,v1:172.21.15.163:6839/444766663] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621425f9d40 con 0x5621422a2e40 2024-05-27T09:27:35.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.706+0000 7f9c7be00640 1 --2- 172.21.15.114:0/2556992165 >> [v2:172.21.15.163:6837/444766663,v1:172.21.15.163:6839/444766663] conn(0x5621422a2e40 0x5621426e4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:35.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.706+0000 7f9c7be00640 1 --2- 172.21.15.114:0/2556992165 >> [v2:172.21.15.163:6837/444766663,v1:172.21.15.163:6839/444766663] conn(0x5621422a2e40 0x5621426e4fc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f747d7f53e53955 server_cookie=29839f4a5aaac7 in_seq=0 out_seq=0 2024-05-27T09:27:35.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:35.706+0000 7f9c65600640 10 client.4518 ms_handle_connect on v2:172.21.15.163:6837/444766663 2024-05-27T09:27:38.026 INFO:tasks.ceph.mds.e.smithi114.stderr:2024-05-27T09:27:38.025+0000 7f4ffb600640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-27T09:27:38.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.845+0000 7f9c65600640 1 -- 172.21.15.114:0/2556992165 <== mds.2 v2:172.21.15.163:6837/444766663 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5621425f9d40 con 0x5621422a2e40 2024-05-27T09:27:38.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.845+0000 7f9c65600640 10 client.4518 handle_command_reply: tid=0 2024-05-27T09:27:38.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.845+0000 7f9c89dc2740 1 client.4518 shutdown 2024-05-27T09:27:38.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.846+0000 7f9c89dc2740 1 -- 172.21.15.114:0/2556992165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422e4270 msgr2=0x5621426cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:38.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.846+0000 7f9c89dc2740 1 --2- 172.21.15.114:0/2556992165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422e4270 0x5621426cb3e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9c74011ce0 tx=0x7f9c74004ab0 comp rx=0 tx=0).stop 2024-05-27T09:27:38.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.847+0000 7f9c89dc2740 1 -- 172.21.15.114:0/2556992165 shutdown_connections 2024-05-27T09:27:38.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.847+0000 7f9c89dc2740 1 --2- 172.21.15.114:0/2556992165 >> [v2:172.21.15.163:6837/444766663,v1:172.21.15.163:6839/444766663] conn(0x5621422a2e40 0x5621426e4fc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:38.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.847+0000 7f9c89dc2740 1 --2- 172.21.15.114:0/2556992165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422e4270 0x5621426cb3e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:38.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.847+0000 7f9c89dc2740 1 --2- 172.21.15.114:0/2556992165 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5621422c3590 0x5621426bf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:38.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.847+0000 7f9c89dc2740 1 --2- 172.21.15.114:0/2556992165 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422bb3b0 0x5621426c04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:38.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.847+0000 7f9c89dc2740 1 -- 172.21.15.114:0/2556992165 >> 172.21.15.114:0/2556992165 conn(0x5621422d83c0 msgr2=0x562142684490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:38.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.847+0000 7f9c65600640 0 client.4518 ms_handle_reset on v2:172.21.15.163:6837/444766663 2024-05-27T09:27:38.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.847+0000 7f9c89dc2740 1 -- 172.21.15.114:0/2556992165 shutdown_connections 2024-05-27T09:27:38.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.847+0000 7f9c89dc2740 1 -- 172.21.15.114:0/2556992165 wait complete. 2024-05-27T09:27:38.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.848+0000 7f9c89dc2740 20 client.4518 trim_cache size 0 max 16384 2024-05-27T09:27:38.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.879+0000 7f9c89dc2740 1 Processor -- start 2024-05-27T09:27:38.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.880+0000 7f9c89dc2740 1 -- start start 2024-05-27T09:27:38.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.882+0000 7f9c89dc2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422bb3b0 0x5621426bf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:38.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.882+0000 7f9c89dc2740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5621422c3590 0x5621426bffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:38.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.882+0000 7f9c89dc2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422e4270 0x5621426c04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:38.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.883+0000 7f9c89dc2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7c070cc0 con 0x5621422e4270 2024-05-27T09:27:38.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.883+0000 7f9c89dc2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5621426c2620 con 0x5621422bb3b0 2024-05-27T09:27:38.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.883+0000 7f9c89dc2740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x5621426bf7e0 con 0x5621422c3590 2024-05-27T09:27:38.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.883+0000 7f9c7be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422bb3b0 0x5621426bf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:38.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.883+0000 7f9c7b400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5621422c3590 0x5621426bffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:38.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.883+0000 7f9c7be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422bb3b0 0x5621426bf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:44546/0 (socket says 172.21.15.114:44546) 2024-05-27T09:27:38.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.883+0000 7f9c7be00640 1 -- 172.21.15.114:0/4233059559 learned_addr learned my addr 172.21.15.114:0/4233059559 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:38.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.883+0000 7f9c80e00640 1 --2- 172.21.15.114:0/4233059559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422e4270 0x5621426c04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:38.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.884+0000 7f9c7b400640 1 -- 172.21.15.114:0/4233059559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422bb3b0 msgr2=0x5621426bf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:38.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.884+0000 7f9c7b400640 1 --2- 172.21.15.114:0/4233059559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422bb3b0 0x5621426bf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:38.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.884+0000 7f9c7b400640 1 -- 172.21.15.114:0/4233059559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422e4270 msgr2=0x5621426c04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:38.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.884+0000 7f9c7b400640 1 --2- 172.21.15.114:0/4233059559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422e4270 0x5621426c04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:38.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.884+0000 7f9c7b400640 1 -- 172.21.15.114:0/4233059559 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621423c66b0 con 0x5621422c3590 2024-05-27T09:27:38.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.884+0000 7f9c7be00640 1 --2- 172.21.15.114:0/4233059559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422bb3b0 0x5621426bf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:38.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.884+0000 7f9c80e00640 1 --2- 172.21.15.114:0/4233059559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422e4270 0x5621426c04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:27:38.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.884+0000 7f9c7b400640 1 --2- 172.21.15.114:0/4233059559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5621422c3590 0x5621426bffb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c070020 tx=0x7f9c6802c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ed1706bb5133b7d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:38.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.884+0000 7f9c67e00640 1 -- 172.21.15.114:0/4233059559 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6804f030 con 0x5621422c3590 2024-05-27T09:27:38.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.885+0000 7f9c67e00640 1 -- 172.21.15.114:0/4233059559 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c68003ca0 con 0x5621422c3590 2024-05-27T09:27:38.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.885+0000 7f9c67e00640 1 -- 172.21.15.114:0/4233059559 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68037040 con 0x5621422c3590 2024-05-27T09:27:38.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.886+0000 7f9c89dc2740 1 -- 172.21.15.114:0/4233059559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5621422c3590 msgr2=0x5621426bffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:38.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.886+0000 7f9c89dc2740 1 --2- 172.21.15.114:0/4233059559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5621422c3590 0x5621426bffb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c070020 tx=0x7f9c6802c000 comp rx=0 tx=0).stop 2024-05-27T09:27:38.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.886+0000 7f9c89dc2740 1 -- 172.21.15.114:0/4233059559 shutdown_connections 2024-05-27T09:27:38.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.886+0000 7f9c89dc2740 1 --2- 172.21.15.114:0/4233059559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422e4270 0x5621426c04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:38.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.886+0000 7f9c89dc2740 1 --2- 172.21.15.114:0/4233059559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5621422c3590 0x5621426bffb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:38.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.886+0000 7f9c89dc2740 1 --2- 172.21.15.114:0/4233059559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422bb3b0 0x5621426bf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:38.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.886+0000 7f9c89dc2740 1 -- 172.21.15.114:0/4233059559 >> 172.21.15.114:0/4233059559 conn(0x5621422d83c0 msgr2=0x56214267bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:38.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.887+0000 7f9c89dc2740 1 -- 172.21.15.114:0/4233059559 shutdown_connections 2024-05-27T09:27:38.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.887+0000 7f9c89dc2740 1 -- 172.21.15.114:0/4233059559 wait complete. 2024-05-27T09:27:38.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.888+0000 7f9c89dc2740 1 -- start start 2024-05-27T09:27:38.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.888+0000 7f9c89dc2740 1 Processor -- start 2024-05-27T09:27:38.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.889+0000 7f9c89dc2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422bb3b0 0x5621426dbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:38.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.889+0000 7f9c89dc2740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5621422c3590 0x5621426d8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:38.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.890+0000 7f9c89dc2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422e4270 0x56214273e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:38.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.890+0000 7f9c89dc2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5621426ca910 con 0x5621422e4270 2024-05-27T09:27:38.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.890+0000 7f9c89dc2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5621426ca2c0 con 0x5621422bb3b0 2024-05-27T09:27:38.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.890+0000 7f9c89dc2740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x5621426d9c30 con 0x5621422c3590 2024-05-27T09:27:38.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.890+0000 7f9c89dc2740 10 client.0 _ll_register_callbacks cb 0x7f9c68035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T09:27:38.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.890+0000 7f9c80e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422e4270 0x56214273e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:38.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.890+0000 7f9c80e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422e4270 0x56214273e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47830/0 (socket says 172.21.15.114:47830) 2024-05-27T09:27:38.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.890+0000 7f9c80e00640 1 -- 172.21.15.114:0/3564228028 learned_addr learned my addr 172.21.15.114:0/3564228028 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:38.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.890+0000 7f9c7b400640 1 --2- 172.21.15.114:0/3564228028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5621422c3590 0x5621426d8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:38.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.890+0000 7f9c7be00640 1 --2- 172.21.15.114:0/3564228028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422bb3b0 0x5621426dbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:38.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.890+0000 7f9c80e00640 1 -- 172.21.15.114:0/3564228028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5621422c3590 msgr2=0x5621426d8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:38.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.890+0000 7f9c80e00640 1 --2- 172.21.15.114:0/3564228028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5621422c3590 0x5621426d8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:38.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.890+0000 7f9c80e00640 1 -- 172.21.15.114:0/3564228028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422bb3b0 msgr2=0x5621426dbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:38.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.890+0000 7f9c80e00640 1 --2- 172.21.15.114:0/3564228028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422bb3b0 0x5621426dbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:38.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.890+0000 7f9c80e00640 1 -- 172.21.15.114:0/3564228028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621426d9990 con 0x5621422e4270 2024-05-27T09:27:38.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.891+0000 7f9c80e00640 1 --2- 172.21.15.114:0/3564228028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422e4270 0x56214273e300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9c74011ce0 tx=0x7f9c7400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e967a3a480b63c9a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:38.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.891+0000 7f9c65600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-05-27T09:27:38.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.891+0000 7f9c65600640 1 -- 172.21.15.114:0/3564228028 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74021040 con 0x5621422e4270 2024-05-27T09:27:38.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.891+0000 7f9c65600640 1 -- 172.21.15.114:0/3564228028 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9c74015bf0 con 0x5621422e4270 2024-05-27T09:27:38.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.891+0000 7f9c89dc2740 1 -- 172.21.15.114:0/3564228028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c68003ca0 con 0x5621422e4270 2024-05-27T09:27:38.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.891+0000 7f9c65600640 1 -- 172.21.15.114:0/3564228028 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) ==== 24+0+0 (secure 0 0 0) 0x7f9c74040050 con 0x5621422e4270 2024-05-27T09:27:38.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.892+0000 7f9c89dc2740 10 client.4524 fetch_fsmap learned FSMap version 12 2024-05-27T09:27:38.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.892+0000 7f9c89dc2740 1 -- 172.21.15.114:0/3564228028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) -- 0x562142681060 con 0x5621422e4270 2024-05-27T09:27:38.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.892+0000 7f9c65600640 1 -- 172.21.15.114:0/3564228028 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c68003ca0 con 0x5621422e4270 2024-05-27T09:27:38.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.892+0000 7f9c65600640 1 -- 172.21.15.114:0/3564228028 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 12) ==== 4744+0+0 (secure 0 0 0) 0x7f9c74043020 con 0x5621422e4270 2024-05-27T09:27:38.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.892+0000 7f9c89dc2740 10 client.4524 fetch_fsmap finished waiting for FSMap version 12 2024-05-27T09:27:38.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.892+0000 7f9c89dc2740 10 client.4524 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-27T09:27:38.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.892+0000 7f9c89dc2740 20 client.4524 populate_metadata read hostname 'smithi114' 2024-05-27T09:27:38.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.892+0000 7f9c89dc2740 1 --2- 172.21.15.114:0/3564228028 >> [v2:172.21.15.163:6837/444766663,v1:172.21.15.163:6839/444766663] conn(0x5621426863c0 0x562142750ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:38.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.892+0000 7f9c89dc2740 4 client.4524 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T09:27:38.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.892+0000 7f9c89dc2740 1 -- 172.21.15.114:0/3564228028 --> [v2:172.21.15.163:6837/444766663,v1:172.21.15.163:6839/444766663] -- command(tid 0: {"prefix": "status"}) -- 0x5621425f3520 con 0x5621426863c0 2024-05-27T09:27:38.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.893+0000 7f9c7be00640 1 --2- 172.21.15.114:0/3564228028 >> [v2:172.21.15.163:6837/444766663,v1:172.21.15.163:6839/444766663] conn(0x5621426863c0 0x562142750ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:38.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.893+0000 7f9c7be00640 1 --2- 172.21.15.114:0/3564228028 >> [v2:172.21.15.163:6837/444766663,v1:172.21.15.163:6839/444766663] conn(0x5621426863c0 0x562142750ea0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b6cc8eaf844af7d server_cookie=6ec39a7a1c98cf20 in_seq=0 out_seq=0 2024-05-27T09:27:38.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.893+0000 7f9c65600640 10 client.4524 ms_handle_connect on v2:172.21.15.163:6837/444766663 2024-05-27T09:27:38.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.893+0000 7f9c65600640 1 -- 172.21.15.114:0/3564228028 <== mds.2 v2:172.21.15.163:6837/444766663 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x5621425f3520 con 0x5621426863c0 2024-05-27T09:27:38.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.893+0000 7f9c65600640 10 client.4524 handle_command_reply: tid=0 2024-05-27T09:27:38.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.893+0000 7f9c89dc2740 1 client.4524 shutdown 2024-05-27T09:27:38.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.894+0000 7f9c89dc2740 1 -- 172.21.15.114:0/3564228028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422e4270 msgr2=0x56214273e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:38.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.894+0000 7f9c89dc2740 1 --2- 172.21.15.114:0/3564228028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422e4270 0x56214273e300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9c74011ce0 tx=0x7f9c7400b040 comp rx=0 tx=0).stop 2024-05-27T09:27:38.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.894+0000 7f9c89dc2740 1 -- 172.21.15.114:0/3564228028 shutdown_connections 2024-05-27T09:27:38.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.894+0000 7f9c89dc2740 1 --2- 172.21.15.114:0/3564228028 >> [v2:172.21.15.163:6837/444766663,v1:172.21.15.163:6839/444766663] conn(0x5621426863c0 0x562142750ea0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:38.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.894+0000 7f9c89dc2740 1 --2- 172.21.15.114:0/3564228028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621422e4270 0x56214273e300 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:38.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.894+0000 7f9c89dc2740 1 --2- 172.21.15.114:0/3564228028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5621422c3590 0x5621426d8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:38.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.894+0000 7f9c89dc2740 1 --2- 172.21.15.114:0/3564228028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5621422bb3b0 0x5621426dbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:38.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.894+0000 7f9c65600640 0 client.4524 ms_handle_reset on v2:172.21.15.163:6837/444766663 2024-05-27T09:27:38.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.894+0000 7f9c89dc2740 1 -- 172.21.15.114:0/3564228028 >> 172.21.15.114:0/3564228028 conn(0x5621422d83c0 msgr2=0x56214267bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:38.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.894+0000 7f9c89dc2740 1 -- 172.21.15.114:0/3564228028 shutdown_connections 2024-05-27T09:27:38.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.894+0000 7f9c89dc2740 1 -- 172.21.15.114:0/3564228028 wait complete. 2024-05-27T09:27:38.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.894+0000 7f9c89dc2740 20 client.4524 trim_cache size 0 max 16384 2024-05-27T09:27:38.898 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-05-27T09:27:38.899 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48", 2024-05-27T09:27:38.899 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 2, 2024-05-27T09:27:38.899 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 4279, 2024-05-27T09:27:38.899 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-05-27T09:27:38.899 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-05-27T09:27:38.899 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-05-27T09:27:38.899 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 10.170580419, 2024-05-27T09:27:38.899 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 12, 2024-05-27T09:27:38.899 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 21, 2024-05-27T09:27:38.899 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 21, 2024-05-27T09:27:38.899 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 26.893960818 2024-05-27T09:27:38.899 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-05-27T09:27:38.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.895+0000 7f9c81800640 1 -- 172.21.15.114:0/926864122 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9c5c0762d0 msgr2=0x7f9c5c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:38.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.895+0000 7f9c81800640 1 --2- 172.21.15.114:0/926864122 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9c5c0762d0 0x7f9c5c0787e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9c68034000 tx=0x7f9c6802e000 comp rx=0 tx=0).stop 2024-05-27T09:27:38.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.895+0000 7f9c81800640 1 -- 172.21.15.114:0/926864122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c7c0741f0 msgr2=0x7f9c7c1566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:38.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.895+0000 7f9c81800640 1 --2- 172.21.15.114:0/926864122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c7c0741f0 0x7f9c7c1566a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c00c410 tx=0x7f9c6c00c810 comp rx=0 tx=0).stop 2024-05-27T09:27:38.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.895+0000 7f9c81800640 1 -- 172.21.15.114:0/926864122 shutdown_connections 2024-05-27T09:27:38.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.895+0000 7f9c81800640 1 --2- 172.21.15.114:0/926864122 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9c5c0762d0 0x7f9c5c0787e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:38.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.895+0000 7f9c81800640 1 --2- 172.21.15.114:0/926864122 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c7c1547d0 0x7f9c7c0cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:38.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.895+0000 7f9c81800640 1 --2- 172.21.15.114:0/926864122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c7c074840 0x7f9c7c155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:38.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.895+0000 7f9c81800640 1 --2- 172.21.15.114:0/926864122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c7c0741f0 0x7f9c7c1566a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:38.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.895+0000 7f9c81800640 1 -- 172.21.15.114:0/926864122 >> 172.21.15.114:0/926864122 conn(0x7f9c7c0ca320 msgr2=0x7f9c7c0c4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:38.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.895+0000 7f9c81800640 1 -- 172.21.15.114:0/926864122 shutdown_connections 2024-05-27T09:27:38.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:38.895+0000 7f9c81800640 1 -- 172.21.15.114:0/926864122 wait complete. 2024-05-27T09:27:38.925 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-27T09:27:39.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.197+0000 7fbc7f800640 1 Processor -- start 2024-05-27T09:27:39.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.198+0000 7fbc7f800640 1 -- start start 2024-05-27T09:27:39.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.200+0000 7fbc7f800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbc78074a60 0x7fbc78154820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.200+0000 7fbc7f800640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc7815a810 con 0x7fbc78073260 2024-05-27T09:27:39.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.200+0000 7fbc7f800640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc7815aaf0 con 0x7fbc780ca9e0 2024-05-27T09:27:39.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.200+0000 7fbc7f800640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fbc7815db60 con 0x7fbc78074a60 2024-05-27T09:27:39.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.200+0000 7fbc6fe00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fbc78073260 0x7fbc780ca2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:46918/0 (socket says 172.21.15.114:46918) 2024-05-27T09:27:39.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.200+0000 7fbc6fe00640 1 -- 172.21.15.114:0/2168155747 learned_addr learned my addr 172.21.15.114:0/2168155747 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:39.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.200+0000 7fbc7d800640 1 --1- 172.21.15.114:0/2168155747 >> v1:172.21.15.163:6789/0 conn(0x7fbc780ca9e0 0x7fbc7815dff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:59138/0 (socket says 172.21.15.114:59138) 2024-05-27T09:27:39.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.200+0000 7fbc7ce00640 1 --2- 172.21.15.114:0/2168155747 >> v2:172.21.15.114:3300/0 conn(0x7fbc78074a60 0x7fbc78154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.201+0000 7fbc77e00640 1 -- 172.21.15.114:0/2168155747 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4240170271 0 0) 0x7fbc7815aaf0 con 0x7fbc780ca9e0 2024-05-27T09:27:39.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.201+0000 7fbc77e00640 1 -- 172.21.15.114:0/2168155747 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc5c003520 con 0x7fbc780ca9e0 2024-05-27T09:27:39.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.201+0000 7fbc77e00640 1 -- 172.21.15.114:0/2168155747 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2022482872 0 0) 0x7fbc7815a810 con 0x7fbc78073260 2024-05-27T09:27:39.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.201+0000 7fbc77e00640 1 -- 172.21.15.114:0/2168155747 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc7815aaf0 con 0x7fbc78073260 2024-05-27T09:27:39.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.201+0000 7fbc7ce00640 1 -- 172.21.15.114:0/2168155747 >> v1:172.21.15.163:6789/0 conn(0x7fbc780ca9e0 legacy=0x7fbc7815dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:39.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.201+0000 7fbc7ce00640 1 -- 172.21.15.114:0/2168155747 >> v1:172.21.15.114:6789/0 conn(0x7fbc78073260 legacy=0x7fbc780ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:39.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.201+0000 7fbc7ce00640 1 -- 172.21.15.114:0/2168155747 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7815add0 con 0x7fbc78074a60 2024-05-27T09:27:39.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.203+0000 7fbc7ce00640 1 --2- 172.21.15.114:0/2168155747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc78074a60 0x7fbc78154820 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbc680097a0 tx=0x7fbc6802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f71f5cba8225a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:39.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.203+0000 7fbc77e00640 1 -- 172.21.15.114:0/2168155747 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68037070 con 0x7fbc78074a60 2024-05-27T09:27:39.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.203+0000 7fbc77e00640 1 -- 172.21.15.114:0/2168155747 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc68032640 con 0x7fbc78074a60 2024-05-27T09:27:39.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.203+0000 7fbc77e00640 1 -- 172.21.15.114:0/2168155747 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc680329f0 con 0x7fbc78074a60 2024-05-27T09:27:39.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.204+0000 7fbc7f800640 1 -- 172.21.15.114:0/2168155747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc78074a60 msgr2=0x7fbc78154820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:39.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.204+0000 7fbc7f800640 1 --2- 172.21.15.114:0/2168155747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc78074a60 0x7fbc78154820 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbc680097a0 tx=0x7fbc6802a2c0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.204+0000 7fbc7f800640 1 -- 172.21.15.114:0/2168155747 shutdown_connections 2024-05-27T09:27:39.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.204+0000 7fbc7f800640 1 --2- 172.21.15.114:0/2168155747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc78074a60 0x7fbc78154820 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.204+0000 7fbc7f800640 1 -- 172.21.15.114:0/2168155747 >> 172.21.15.114:0/2168155747 conn(0x7fbc7806f6e0 msgr2=0x7fbc78070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:39.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.204+0000 7fbc7f800640 1 -- 172.21.15.114:0/2168155747 shutdown_connections 2024-05-27T09:27:39.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.204+0000 7fbc7f800640 1 -- 172.21.15.114:0/2168155747 wait complete. 2024-05-27T09:27:39.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.206+0000 7fbc7f800640 1 Processor -- start 2024-05-27T09:27:39.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.206+0000 7fbc7f800640 1 -- start start 2024-05-27T09:27:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.208+0000 7fbc7f800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc78073260 0x7fbc7815cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.208+0000 7fbc7f800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc78074a60 0x7fbc780c2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.208+0000 7fbc7f800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc780ca9e0 0x7fbc780c3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.208+0000 7fbc7f800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7815a4e0 con 0x7fbc78073260 2024-05-27T09:27:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.208+0000 7fbc7f800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7815b390 con 0x7fbc78074a60 2024-05-27T09:27:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.208+0000 7fbc7f800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fbc7815e700 con 0x7fbc780ca9e0 2024-05-27T09:27:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.208+0000 7fbc7ce00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc78073260 0x7fbc7815cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.208+0000 7fbc7d800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc780ca9e0 0x7fbc780c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.208+0000 7fbc6fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc78074a60 0x7fbc780c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.208+0000 7fbc7ce00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc78073260 0x7fbc7815cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47848/0 (socket says 172.21.15.114:47848) 2024-05-27T09:27:39.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.208+0000 7fbc7ce00640 1 -- 172.21.15.114:0/2750573797 learned_addr learned my addr 172.21.15.114:0/2750573797 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:39.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.208+0000 7fbc7ce00640 1 -- 172.21.15.114:0/2750573797 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc780ca9e0 msgr2=0x7fbc780c3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:39.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.208+0000 7fbc7ce00640 1 --2- 172.21.15.114:0/2750573797 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc780ca9e0 0x7fbc780c3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.209+0000 7fbc7ce00640 1 -- 172.21.15.114:0/2750573797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc78074a60 msgr2=0x7fbc780c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:39.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.209+0000 7fbc7ce00640 1 --2- 172.21.15.114:0/2750573797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc78074a60 0x7fbc780c2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.209+0000 7fbc7ce00640 1 -- 172.21.15.114:0/2750573797 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc780c5e80 con 0x7fbc78073260 2024-05-27T09:27:39.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.209+0000 7fbc7ce00640 1 --2- 172.21.15.114:0/2750573797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc78073260 0x7fbc7815cae0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbc68000c00 tx=0x7fbc680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:39.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.209+0000 7fbc75600640 1 -- 172.21.15.114:0/2750573797 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68037040 con 0x7fbc78073260 2024-05-27T09:27:39.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.209+0000 7fbc75600640 1 -- 172.21.15.114:0/2750573797 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc68042070 con 0x7fbc78073260 2024-05-27T09:27:39.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.209+0000 7fbc7f800640 1 -- 172.21.15.114:0/2750573797 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc780c6160 con 0x7fbc78073260 2024-05-27T09:27:39.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.209+0000 7fbc75600640 1 -- 172.21.15.114:0/2750573797 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc680309b0 con 0x7fbc78073260 2024-05-27T09:27:39.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.209+0000 7fbc7f800640 1 -- 172.21.15.114:0/2750573797 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc781e4530 con 0x7fbc78073260 2024-05-27T09:27:39.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.212+0000 7fbc75600640 1 -- 172.21.15.114:0/2750573797 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbc68030b50 con 0x7fbc78073260 2024-05-27T09:27:39.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.213+0000 7fbc75600640 1 --2- 172.21.15.114:0/2750573797 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbc540762d0 0x7fbc540787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.213+0000 7fbc75600640 1 -- 172.21.15.114:0/2750573797 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbc680b7e90 con 0x7fbc78073260 2024-05-27T09:27:39.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.213+0000 7fbc6fe00640 1 --2- 172.21.15.114:0/2750573797 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbc540762d0 0x7fbc540787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.213+0000 7fbc86873740 1 Processor -- start 2024-05-27T09:27:39.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.214+0000 7fbc86873740 1 -- start start 2024-05-27T09:27:39.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.214+0000 7fbc6fe00640 1 --2- 172.21.15.114:0/2750573797 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbc540762d0 0x7fbc540787e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbc78153e10 tx=0x7fbc60009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:39.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.215+0000 7fbc86873740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18afab270 0x55d18b34ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.215+0000 7fbc86873740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d18af823b0 0x55d18b38c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.215+0000 7fbc86873740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af8a590 0x55d18b3715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.215+0000 7fbc86873740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d18b2bedf0 con 0x55d18af8a590 2024-05-27T09:27:39.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.215+0000 7fbc86873740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d18b2ae990 con 0x55d18af823b0 2024-05-27T09:27:39.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.215+0000 7fbc86873740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x55d18b2b29d0 con 0x55d18afab270 2024-05-27T09:27:39.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.215+0000 7fbc7ce00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18afab270 0x55d18b34ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.215+0000 7fbc6fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d18af823b0 0x55d18b38c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.215+0000 7fbc7ce00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18afab270 0x55d18b34ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:35652/0 (socket says 172.21.15.114:35652) 2024-05-27T09:27:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.215+0000 7fbc7ce00640 1 -- 172.21.15.114:0/2168448315 learned_addr learned my addr 172.21.15.114:0/2168448315 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.216+0000 7fbc7d800640 1 --2- 172.21.15.114:0/2168448315 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af8a590 0x55d18b3715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.216+0000 7fbc7ce00640 1 -- 172.21.15.114:0/2168448315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d18af823b0 msgr2=0x55d18b38c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.216+0000 7fbc7ce00640 1 --2- 172.21.15.114:0/2168448315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d18af823b0 0x55d18b38c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.216+0000 7fbc7ce00640 1 -- 172.21.15.114:0/2168448315 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af8a590 msgr2=0x55d18b3715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.216+0000 7fbc7ce00640 1 --2- 172.21.15.114:0/2168448315 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af8a590 0x55d18b3715b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.216+0000 7fbc7ce00640 1 -- 172.21.15.114:0/2168448315 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d18b24f9b0 con 0x55d18afab270 2024-05-27T09:27:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.216+0000 7fbc7ce00640 1 --2- 172.21.15.114:0/2168448315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18afab270 0x55d18b34ce40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbc6807d780 tx=0x7fbc68080900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e28260e4b148aa07 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.216+0000 7fbc6d600640 1 -- 172.21.15.114:0/2168448315 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68081190 con 0x55d18afab270 2024-05-27T09:27:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.216+0000 7fbc6d600640 1 -- 172.21.15.114:0/2168448315 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc68085490 con 0x55d18afab270 2024-05-27T09:27:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.216+0000 7fbc6d600640 1 -- 172.21.15.114:0/2168448315 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68089eb0 con 0x55d18afab270 2024-05-27T09:27:39.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.217+0000 7fbc86873740 1 -- 172.21.15.114:0/2168448315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18afab270 msgr2=0x55d18b34ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:39.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.217+0000 7fbc86873740 1 --2- 172.21.15.114:0/2168448315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18afab270 0x55d18b34ce40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbc6807d780 tx=0x7fbc68080900 comp rx=0 tx=0).stop 2024-05-27T09:27:39.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.217+0000 7fbc86873740 1 -- 172.21.15.114:0/2168448315 shutdown_connections 2024-05-27T09:27:39.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.217+0000 7fbc86873740 1 --2- 172.21.15.114:0/2168448315 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af8a590 0x55d18b3715b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.217+0000 7fbc86873740 1 --2- 172.21.15.114:0/2168448315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d18af823b0 0x55d18b38c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.217+0000 7fbc86873740 1 --2- 172.21.15.114:0/2168448315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18afab270 0x55d18b34ce40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.217+0000 7fbc86873740 1 -- 172.21.15.114:0/2168448315 >> 172.21.15.114:0/2168448315 conn(0x55d18af9f3c0 msgr2=0x55d18b34bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.217+0000 7fbc86873740 1 -- 172.21.15.114:0/2168448315 shutdown_connections 2024-05-27T09:27:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.217+0000 7fbc86873740 1 -- 172.21.15.114:0/2168448315 wait complete. 2024-05-27T09:27:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.218+0000 7fbc86873740 1 -- start start 2024-05-27T09:27:39.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.219+0000 7fbc86873740 1 Processor -- start 2024-05-27T09:27:39.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.220+0000 7fbc86873740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18af823b0 0x55d18b3862e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.220+0000 7fbc86873740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af8a590 0x55d18b39da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.220+0000 7fbc86873740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d18afab270 0x55d18b374830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.220+0000 7fbc86873740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d18af94850 con 0x55d18af8a590 2024-05-27T09:27:39.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.220+0000 7fbc86873740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d18b256f20 con 0x55d18afab270 2024-05-27T09:27:39.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.220+0000 7fbc86873740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x55d18b39e430 con 0x55d18af823b0 2024-05-27T09:27:39.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.220+0000 7fbc86873740 10 client.0 _ll_register_callbacks cb 0x55d18b2a8ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T09:27:39.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.220+0000 7fbc6fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af8a590 0x55d18b39da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.220+0000 7fbc6fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af8a590 0x55d18b39da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47868/0 (socket says 172.21.15.114:47868) 2024-05-27T09:27:39.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.220+0000 7fbc6fe00640 1 -- 172.21.15.114:0/874441085 learned_addr learned my addr 172.21.15.114:0/874441085 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:39.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.221+0000 7fbc7ce00640 1 --2- 172.21.15.114:0/874441085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18af823b0 0x55d18b3862e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.221+0000 7fbc7d800640 1 --2- 172.21.15.114:0/874441085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d18afab270 0x55d18b374830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.221+0000 7fbc6fe00640 1 -- 172.21.15.114:0/874441085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18af823b0 msgr2=0x55d18b3862e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:39.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.221+0000 7fbc6fe00640 1 --2- 172.21.15.114:0/874441085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18af823b0 0x55d18b3862e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.221+0000 7fbc6fe00640 1 -- 172.21.15.114:0/874441085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d18afab270 msgr2=0x55d18b374830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:39.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.221+0000 7fbc6fe00640 1 --2- 172.21.15.114:0/874441085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d18afab270 0x55d18b374830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.221+0000 7fbc6fe00640 1 -- 172.21.15.114:0/874441085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d18b247d00 con 0x55d18af8a590 2024-05-27T09:27:39.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.221+0000 7fbc7ce00640 1 --2- 172.21.15.114:0/874441085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18af823b0 0x55d18b3862e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.221+0000 7fbc6fe00640 1 --2- 172.21.15.114:0/874441085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af8a590 0x55d18b39da20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbc600146d0 tx=0x7fbc600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2391f978fadbafc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.221+0000 7fbc66000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-05-27T09:27:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.222+0000 7fbc66000640 1 -- 172.21.15.114:0/874441085 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60025020 con 0x55d18af8a590 2024-05-27T09:27:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.222+0000 7fbc66000640 1 -- 172.21.15.114:0/874441085 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbc60021610 con 0x55d18af8a590 2024-05-27T09:27:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.222+0000 7fbc86873740 1 -- 172.21.15.114:0/874441085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc68085490 con 0x55d18af8a590 2024-05-27T09:27:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.222+0000 7fbc66000640 1 -- 172.21.15.114:0/874441085 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) ==== 24+0+0 (secure 0 0 0) 0x7fbc6002f050 con 0x55d18af8a590 2024-05-27T09:27:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.222+0000 7fbc86873740 10 client.4539 fetch_fsmap learned FSMap version 12 2024-05-27T09:27:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.222+0000 7fbc86873740 1 -- 172.21.15.114:0/874441085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) -- 0x55d18afc32d0 con 0x55d18af8a590 2024-05-27T09:27:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.222+0000 7fbc66000640 1 -- 172.21.15.114:0/874441085 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc68085490 con 0x55d18af8a590 2024-05-27T09:27:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.222+0000 7fbc66000640 1 -- 172.21.15.114:0/874441085 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 12) ==== 4744+0+0 (secure 0 0 0) 0x7fbc60032020 con 0x55d18af8a590 2024-05-27T09:27:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.222+0000 7fbc86873740 10 client.4539 fetch_fsmap finished waiting for FSMap version 12 2024-05-27T09:27:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.222+0000 7fbc86873740 10 client.4539 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-27T09:27:39.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.222+0000 7fbc86873740 20 client.4539 populate_metadata read hostname 'smithi114' 2024-05-27T09:27:39.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.222+0000 7fbc86873740 1 --2- 172.21.15.114:0/874441085 >> [v2:172.21.15.114:6832/657027519,v1:172.21.15.114:6834/657027519] conn(0x55d18af69e40 0x55d18b3ac180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.222+0000 7fbc86873740 4 client.4539 mds_command: new command op to 4297 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T09:27:39.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.222+0000 7fbc86873740 1 -- 172.21.15.114:0/874441085 --> [v2:172.21.15.114:6832/657027519,v1:172.21.15.114:6834/657027519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d18b2c0d40 con 0x55d18af69e40 2024-05-27T09:27:39.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.223+0000 7fbc7ce00640 1 --2- 172.21.15.114:0/874441085 >> [v2:172.21.15.114:6832/657027519,v1:172.21.15.114:6834/657027519] conn(0x55d18af69e40 0x55d18b3ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.223+0000 7fbc7ce00640 1 --2- 172.21.15.114:0/874441085 >> [v2:172.21.15.114:6832/657027519,v1:172.21.15.114:6834/657027519] conn(0x55d18af69e40 0x55d18b3ac180 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb77a141881c82a1 server_cookie=6fc2de77862d2fed in_seq=0 out_seq=0 2024-05-27T09:27:39.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.223+0000 7fbc66000640 10 client.4539 ms_handle_connect on v2:172.21.15.114:6832/657027519 2024-05-27T09:27:39.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.224+0000 7fbc66000640 1 -- 172.21.15.114:0/874441085 <== mds.1 v2:172.21.15.114:6832/657027519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d18b2c0d40 con 0x55d18af69e40 2024-05-27T09:27:39.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.224+0000 7fbc66000640 10 client.4539 handle_command_reply: tid=0 2024-05-27T09:27:39.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.224+0000 7fbc86873740 1 client.4539 shutdown 2024-05-27T09:27:39.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.225+0000 7fbc86873740 1 -- 172.21.15.114:0/874441085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af8a590 msgr2=0x55d18b39da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:39.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.225+0000 7fbc86873740 1 --2- 172.21.15.114:0/874441085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af8a590 0x55d18b39da20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbc600146d0 tx=0x7fbc600057f0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.225+0000 7fbc86873740 1 -- 172.21.15.114:0/874441085 shutdown_connections 2024-05-27T09:27:39.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.225+0000 7fbc86873740 1 --2- 172.21.15.114:0/874441085 >> [v2:172.21.15.114:6832/657027519,v1:172.21.15.114:6834/657027519] conn(0x55d18af69e40 0x55d18b3ac180 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.225+0000 7fbc86873740 1 --2- 172.21.15.114:0/874441085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d18afab270 0x55d18b374830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.225+0000 7fbc86873740 1 --2- 172.21.15.114:0/874441085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af8a590 0x55d18b39da20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.225+0000 7fbc86873740 1 --2- 172.21.15.114:0/874441085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18af823b0 0x55d18b3862e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.225+0000 7fbc86873740 1 -- 172.21.15.114:0/874441085 >> 172.21.15.114:0/874441085 conn(0x55d18af9f3c0 msgr2=0x55d18b346650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:39.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.225+0000 7fbc66000640 0 client.4539 ms_handle_reset on v2:172.21.15.114:6832/657027519 2024-05-27T09:27:39.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.225+0000 7fbc86873740 1 -- 172.21.15.114:0/874441085 shutdown_connections 2024-05-27T09:27:39.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.225+0000 7fbc86873740 1 -- 172.21.15.114:0/874441085 wait complete. 2024-05-27T09:27:39.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.225+0000 7fbc86873740 20 client.4539 trim_cache size 0 max 16384 2024-05-27T09:27:39.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.246+0000 7fbc86873740 1 Processor -- start 2024-05-27T09:27:39.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.246+0000 7fbc86873740 1 -- start start 2024-05-27T09:27:39.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.247+0000 7fbc86873740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af823b0 0x55d18b387310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.247+0000 7fbc86873740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d18af8a590 0x55d18b34cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.247+0000 7fbc86873740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18afab270 0x55d18b34d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.247+0000 7fbc86873740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781e8e30 con 0x55d18af823b0 2024-05-27T09:27:39.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.247+0000 7fbc86873740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d18b3a1840 con 0x55d18af8a590 2024-05-27T09:27:39.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.247+0000 7fbc86873740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x55d18b395c30 con 0x55d18afab270 2024-05-27T09:27:39.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.247+0000 7fbc7ce00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af823b0 0x55d18b387310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.247+0000 7fbc7ce00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af823b0 0x55d18b387310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47880/0 (socket says 172.21.15.114:47880) 2024-05-27T09:27:39.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.247+0000 7fbc7ce00640 1 -- 172.21.15.114:0/511890318 learned_addr learned my addr 172.21.15.114:0/511890318 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:39.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.248+0000 7fbc7d800640 1 --2- 172.21.15.114:0/511890318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18afab270 0x55d18b34d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.248+0000 7fbc6fe00640 1 --2- 172.21.15.114:0/511890318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d18af8a590 0x55d18b34cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.248+0000 7fbc7ce00640 1 -- 172.21.15.114:0/511890318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18afab270 msgr2=0x55d18b34d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:39.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.248+0000 7fbc7ce00640 1 --2- 172.21.15.114:0/511890318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18afab270 0x55d18b34d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.248+0000 7fbc7ce00640 1 -- 172.21.15.114:0/511890318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d18af8a590 msgr2=0x55d18b34cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:39.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.248+0000 7fbc7ce00640 1 --2- 172.21.15.114:0/511890318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d18af8a590 0x55d18b34cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.248+0000 7fbc7ce00640 1 -- 172.21.15.114:0/511890318 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d18b08d6b0 con 0x55d18af823b0 2024-05-27T09:27:39.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.248+0000 7fbc7ce00640 1 --2- 172.21.15.114:0/511890318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af823b0 0x55d18b387310 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbc6807d210 tx=0x7fbc680b7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928e72b341e16443 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:39.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.248+0000 7fbc6cc00640 1 -- 172.21.15.114:0/511890318 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68085490 con 0x55d18af823b0 2024-05-27T09:27:39.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.248+0000 7fbc6cc00640 1 -- 172.21.15.114:0/511890318 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc68081190 con 0x55d18af823b0 2024-05-27T09:27:39.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.248+0000 7fbc6cc00640 1 -- 172.21.15.114:0/511890318 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc680815f0 con 0x55d18af823b0 2024-05-27T09:27:39.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.249+0000 7fbc86873740 1 -- 172.21.15.114:0/511890318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af823b0 msgr2=0x55d18b387310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:39.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.249+0000 7fbc86873740 1 --2- 172.21.15.114:0/511890318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af823b0 0x55d18b387310 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbc6807d210 tx=0x7fbc680b7000 comp rx=0 tx=0).stop 2024-05-27T09:27:39.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.249+0000 7fbc86873740 1 -- 172.21.15.114:0/511890318 shutdown_connections 2024-05-27T09:27:39.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.249+0000 7fbc86873740 1 --2- 172.21.15.114:0/511890318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18afab270 0x55d18b34d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.249+0000 7fbc86873740 1 --2- 172.21.15.114:0/511890318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d18af8a590 0x55d18b34cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.249+0000 7fbc86873740 1 --2- 172.21.15.114:0/511890318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af823b0 0x55d18b387310 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.249+0000 7fbc86873740 1 -- 172.21.15.114:0/511890318 >> 172.21.15.114:0/511890318 conn(0x55d18af9f3c0 msgr2=0x55d18b346650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:39.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.249+0000 7fbc86873740 1 -- 172.21.15.114:0/511890318 shutdown_connections 2024-05-27T09:27:39.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.249+0000 7fbc86873740 1 -- 172.21.15.114:0/511890318 wait complete. 2024-05-27T09:27:39.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.250+0000 7fbc86873740 1 -- start start 2024-05-27T09:27:39.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.250+0000 7fbc86873740 1 Processor -- start 2024-05-27T09:27:39.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.252+0000 7fbc86873740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18af823b0 0x55d18b34b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.252+0000 7fbc86873740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af8a590 0x55d18b40e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.252+0000 7fbc86873740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d18afab270 0x55d18b36ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.252+0000 7fbc86873740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d18b375000 con 0x55d18af8a590 2024-05-27T09:27:39.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.252+0000 7fbc86873740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d18b389ce0 con 0x55d18afab270 2024-05-27T09:27:39.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.252+0000 7fbc86873740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x55d18b38ab20 con 0x55d18af823b0 2024-05-27T09:27:39.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.252+0000 7fbc86873740 10 client.0 _ll_register_callbacks cb 0x7fbc60001d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T09:27:39.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.252+0000 7fbc6fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af8a590 0x55d18b40e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.252+0000 7fbc7ce00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18af823b0 0x55d18b34b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.252+0000 7fbc6fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af8a590 0x55d18b40e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47882/0 (socket says 172.21.15.114:47882) 2024-05-27T09:27:39.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.252+0000 7fbc6fe00640 1 -- 172.21.15.114:0/2468956308 learned_addr learned my addr 172.21.15.114:0/2468956308 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:39.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.252+0000 7fbc7d800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d18afab270 0x55d18b36ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.253+0000 7fbc6fe00640 1 -- 172.21.15.114:0/2468956308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18af823b0 msgr2=0x55d18b34b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:39.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.253+0000 7fbc6fe00640 1 --2- 172.21.15.114:0/2468956308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18af823b0 0x55d18b34b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.253+0000 7fbc6fe00640 1 -- 172.21.15.114:0/2468956308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d18afab270 msgr2=0x55d18b36ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:39.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.253+0000 7fbc6fe00640 1 --2- 172.21.15.114:0/2468956308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d18afab270 0x55d18b36ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.253+0000 7fbc6fe00640 1 -- 172.21.15.114:0/2468956308 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d18b40ebc0 con 0x55d18af8a590 2024-05-27T09:27:39.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.253+0000 7fbc7ce00640 1 --2- 172.21.15.114:0/2468956308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18af823b0 0x55d18b34b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:39.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.253+0000 7fbc6fe00640 1 --2- 172.21.15.114:0/2468956308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af8a590 0x55d18b40e680 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55d18b38b9e0 tx=0x7fbc60021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5074a44ef208402 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:39.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.253+0000 7fbc66000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-05-27T09:27:39.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.253+0000 7fbc66000640 1 -- 172.21.15.114:0/2468956308 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60025070 con 0x55d18af8a590 2024-05-27T09:27:39.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.253+0000 7fbc66000640 1 -- 172.21.15.114:0/2468956308 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbc6002a3f0 con 0x55d18af8a590 2024-05-27T09:27:39.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.253+0000 7fbc86873740 1 -- 172.21.15.114:0/2468956308 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc68081190 con 0x55d18af8a590 2024-05-27T09:27:39.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.253+0000 7fbc66000640 1 -- 172.21.15.114:0/2468956308 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) ==== 24+0+0 (secure 0 0 0) 0x7fbc60021e40 con 0x55d18af8a590 2024-05-27T09:27:39.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.253+0000 7fbc86873740 10 client.4545 fetch_fsmap learned FSMap version 12 2024-05-27T09:27:39.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.253+0000 7fbc86873740 1 -- 172.21.15.114:0/2468956308 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) -- 0x55d18b376700 con 0x55d18af8a590 2024-05-27T09:27:39.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.254+0000 7fbc66000640 1 -- 172.21.15.114:0/2468956308 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc68081190 con 0x55d18af8a590 2024-05-27T09:27:39.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.254+0000 7fbc66000640 1 -- 172.21.15.114:0/2468956308 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 12) ==== 4744+0+0 (secure 0 0 0) 0x7fbc60044080 con 0x55d18af8a590 2024-05-27T09:27:39.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.254+0000 7fbc86873740 10 client.4545 fetch_fsmap finished waiting for FSMap version 12 2024-05-27T09:27:39.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.254+0000 7fbc86873740 10 client.4545 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-27T09:27:39.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.254+0000 7fbc86873740 20 client.4545 populate_metadata read hostname 'smithi114' 2024-05-27T09:27:39.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.254+0000 7fbc86873740 1 --2- 172.21.15.114:0/2468956308 >> [v2:172.21.15.114:6832/657027519,v1:172.21.15.114:6834/657027519] conn(0x55d18b405170 0x55d18b417740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.254+0000 7fbc86873740 4 client.4545 mds_command: new command op to 4297 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T09:27:39.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.254+0000 7fbc86873740 1 -- 172.21.15.114:0/2468956308 --> [v2:172.21.15.114:6832/657027519,v1:172.21.15.114:6834/657027519] -- command(tid 0: {"prefix": "status"}) -- 0x55d18b2ba520 con 0x55d18b405170 2024-05-27T09:27:39.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.254+0000 7fbc7ce00640 1 --2- 172.21.15.114:0/2468956308 >> [v2:172.21.15.114:6832/657027519,v1:172.21.15.114:6834/657027519] conn(0x55d18b405170 0x55d18b417740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.255+0000 7fbc66000640 10 client.4545 ms_handle_connect on v2:172.21.15.114:6832/657027519 2024-05-27T09:27:39.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.255+0000 7fbc7ce00640 1 --2- 172.21.15.114:0/2468956308 >> [v2:172.21.15.114:6832/657027519,v1:172.21.15.114:6834/657027519] conn(0x55d18b405170 0x55d18b417740 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a15a77386f2ad0e2 server_cookie=75113cb1a90e2d6 in_seq=0 out_seq=0 2024-05-27T09:27:39.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.255+0000 7fbc66000640 1 -- 172.21.15.114:0/2468956308 <== mds.1 v2:172.21.15.114:6832/657027519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55d18b2ba520 con 0x55d18b405170 2024-05-27T09:27:39.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.255+0000 7fbc66000640 10 client.4545 handle_command_reply: tid=0 2024-05-27T09:27:39.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.255+0000 7fbc86873740 1 client.4545 shutdown 2024-05-27T09:27:39.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.255+0000 7fbc86873740 1 -- 172.21.15.114:0/2468956308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af8a590 msgr2=0x55d18b40e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:39.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.255+0000 7fbc86873740 1 --2- 172.21.15.114:0/2468956308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af8a590 0x55d18b40e680 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55d18b38b9e0 tx=0x7fbc60021000 comp rx=0 tx=0).stop 2024-05-27T09:27:39.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.255+0000 7fbc86873740 1 -- 172.21.15.114:0/2468956308 shutdown_connections 2024-05-27T09:27:39.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.255+0000 7fbc86873740 1 --2- 172.21.15.114:0/2468956308 >> [v2:172.21.15.114:6832/657027519,v1:172.21.15.114:6834/657027519] conn(0x55d18b405170 0x55d18b417740 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.255+0000 7fbc86873740 1 --2- 172.21.15.114:0/2468956308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d18afab270 0x55d18b36ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.255+0000 7fbc86873740 1 --2- 172.21.15.114:0/2468956308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d18af8a590 0x55d18b40e680 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.255+0000 7fbc86873740 1 --2- 172.21.15.114:0/2468956308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d18af823b0 0x55d18b34b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.255+0000 7fbc66000640 0 client.4545 ms_handle_reset on v2:172.21.15.114:6832/657027519 2024-05-27T09:27:39.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.255+0000 7fbc86873740 1 -- 172.21.15.114:0/2468956308 >> 172.21.15.114:0/2468956308 conn(0x55d18af9f3c0 msgr2=0x55d18b375d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:39.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.255+0000 7fbc86873740 1 -- 172.21.15.114:0/2468956308 shutdown_connections 2024-05-27T09:27:39.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.255+0000 7fbc86873740 1 -- 172.21.15.114:0/2468956308 wait complete. 2024-05-27T09:27:39.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.256+0000 7fbc86873740 20 client.4545 trim_cache size 0 max 16384 2024-05-27T09:27:39.261 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-05-27T09:27:39.261 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48", 2024-05-27T09:27:39.261 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 1, 2024-05-27T09:27:39.261 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 4297, 2024-05-27T09:27:39.261 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-05-27T09:27:39.261 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-05-27T09:27:39.261 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-05-27T09:27:39.261 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 15.651754239000001, 2024-05-27T09:27:39.261 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 12, 2024-05-27T09:27:39.261 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 21, 2024-05-27T09:27:39.261 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 21, 2024-05-27T09:27:39.261 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 27.150600162 2024-05-27T09:27:39.261 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-05-27T09:27:39.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.256+0000 7fbc7f800640 1 -- 172.21.15.114:0/2750573797 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbc540762d0 msgr2=0x7fbc540787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:39.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.256+0000 7fbc7f800640 1 --2- 172.21.15.114:0/2750573797 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbc540762d0 0x7fbc540787e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbc78153e10 tx=0x7fbc60009040 comp rx=0 tx=0).stop 2024-05-27T09:27:39.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.256+0000 7fbc7f800640 1 -- 172.21.15.114:0/2750573797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc78073260 msgr2=0x7fbc7815cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:39.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.256+0000 7fbc7f800640 1 --2- 172.21.15.114:0/2750573797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc78073260 0x7fbc7815cae0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbc68000c00 tx=0x7fbc680046c0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.256+0000 7fbc7f800640 1 -- 172.21.15.114:0/2750573797 shutdown_connections 2024-05-27T09:27:39.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.256+0000 7fbc7f800640 1 --2- 172.21.15.114:0/2750573797 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbc540762d0 0x7fbc540787e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.256+0000 7fbc7f800640 1 --2- 172.21.15.114:0/2750573797 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc780ca9e0 0x7fbc780c3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.256+0000 7fbc7f800640 1 --2- 172.21.15.114:0/2750573797 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc78074a60 0x7fbc780c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.256+0000 7fbc7f800640 1 --2- 172.21.15.114:0/2750573797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc78073260 0x7fbc7815cae0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.256+0000 7fbc7f800640 1 -- 172.21.15.114:0/2750573797 >> 172.21.15.114:0/2750573797 conn(0x7fbc7806f6e0 msgr2=0x7fbc78158aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:39.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.256+0000 7fbc7f800640 1 -- 172.21.15.114:0/2750573797 shutdown_connections 2024-05-27T09:27:39.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.256+0000 7fbc7f800640 1 -- 172.21.15.114:0/2750573797 wait complete. 2024-05-27T09:27:39.283 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-05-27T09:27:39.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.554+0000 7fa2da600640 1 Processor -- start 2024-05-27T09:27:39.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.554+0000 7fa2da600640 1 -- start start 2024-05-27T09:27:39.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.556+0000 7fa2da600640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa2d4074a60 0x7fa2d4154820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.557+0000 7fa2da600640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa2d4073260 0x7fa2d4073670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.557+0000 7fa2da600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2d40ca2d0 0x7fa2d40ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.557+0000 7fa2da600640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa2d415db60 con 0x7fa2d4073260 2024-05-27T09:27:39.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.557+0000 7fa2da600640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fa2d40cac20 con 0x7fa2d4074a60 2024-05-27T09:27:39.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.557+0000 7fa2da600640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d415a810 con 0x7fa2d40ca2d0 2024-05-27T09:27:39.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.557+0000 7fa2d3400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa2d4074a60 0x7fa2d4154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.557+0000 7fa2d2a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa2d4073260 0x7fa2d4073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.557+0000 7fa2d3400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa2d4074a60 0x7fa2d4154820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:44618/0 (socket says 172.21.15.114:44618) 2024-05-27T09:27:39.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.557+0000 7fa2d3400640 1 -- 172.21.15.114:0/970465712 learned_addr learned my addr 172.21.15.114:0/970465712 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:39.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.557+0000 7fa2d3e00640 1 --2- 172.21.15.114:0/970465712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2d40ca2d0 0x7fa2d40ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.557+0000 7fa2d3400640 1 -- 172.21.15.114:0/970465712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2d40ca2d0 msgr2=0x7fa2d40ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:39.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.557+0000 7fa2d3400640 1 --2- 172.21.15.114:0/970465712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2d40ca2d0 0x7fa2d40ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.557+0000 7fa2d3400640 1 -- 172.21.15.114:0/970465712 >> v2:172.21.15.114:3300/0 conn(0x7fa2d4073260 msgr2=0x7fa2d4073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:39.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.557+0000 7fa2d3400640 1 --2- 172.21.15.114:0/970465712 >> v2:172.21.15.114:3300/0 conn(0x7fa2d4073260 0x7fa2d4073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.557+0000 7fa2d3400640 1 -- 172.21.15.114:0/970465712 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d415aaf0 con 0x7fa2d4074a60 2024-05-27T09:27:39.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.559+0000 7fa2d3400640 1 --2- 172.21.15.114:0/970465712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2d4074a60 0x7fa2d4154820 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa2d40c4ce0 tx=0x7fa2c402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80ae601ded993090 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:39.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.559+0000 7fa2d2000640 1 -- 172.21.15.114:0/970465712 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4037070 con 0x7fa2d4074a60 2024-05-27T09:27:39.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.559+0000 7fa2d2000640 1 -- 172.21.15.114:0/970465712 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c4032440 con 0x7fa2d4074a60 2024-05-27T09:27:39.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.559+0000 7fa2d2000640 1 -- 172.21.15.114:0/970465712 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4032810 con 0x7fa2d4074a60 2024-05-27T09:27:39.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.560+0000 7fa2da600640 1 -- 172.21.15.114:0/970465712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2d4074a60 msgr2=0x7fa2d4154820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:39.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.560+0000 7fa2da600640 1 --2- 172.21.15.114:0/970465712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2d4074a60 0x7fa2d4154820 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa2d40c4ce0 tx=0x7fa2c402a270 comp rx=0 tx=0).stop 2024-05-27T09:27:39.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.560+0000 7fa2da600640 1 -- 172.21.15.114:0/970465712 shutdown_connections 2024-05-27T09:27:39.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.560+0000 7fa2da600640 1 --2- 172.21.15.114:0/970465712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2d40ca2d0 0x7fa2d40ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.560+0000 7fa2da600640 1 --2- 172.21.15.114:0/970465712 >> v2:172.21.15.114:3300/0 conn(0x7fa2d4073260 0x7fa2d4073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.560+0000 7fa2da600640 1 --2- 172.21.15.114:0/970465712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2d4074a60 0x7fa2d4154820 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.560+0000 7fa2da600640 1 -- 172.21.15.114:0/970465712 >> 172.21.15.114:0/970465712 conn(0x7fa2d406f6e0 msgr2=0x7fa2d4070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:39.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.560+0000 7fa2da600640 1 -- 172.21.15.114:0/970465712 shutdown_connections 2024-05-27T09:27:39.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.560+0000 7fa2da600640 1 -- 172.21.15.114:0/970465712 wait complete. 2024-05-27T09:27:39.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.563+0000 7fa2da600640 1 Processor -- start 2024-05-27T09:27:39.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.563+0000 7fa2da600640 1 -- start start 2024-05-27T09:27:39.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.565+0000 7fa2da600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2d4073260 0x7fa2d41557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.565+0000 7fa2da600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2d4074a60 0x7fa2d41f42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.565+0000 7fa2da600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2d40ca2d0 0x7fa2d4148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.565+0000 7fa2da600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d415b390 con 0x7fa2d4073260 2024-05-27T09:27:39.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.565+0000 7fa2da600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d415b0b0 con 0x7fa2d4074a60 2024-05-27T09:27:39.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.565+0000 7fa2da600640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d415a4e0 con 0x7fa2d40ca2d0 2024-05-27T09:27:39.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.565+0000 7fa2d3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2d4073260 0x7fa2d41557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.566+0000 7fa2d3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2d4073260 0x7fa2d41557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47900/0 (socket says 172.21.15.114:47900) 2024-05-27T09:27:39.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.566+0000 7fa2d3400640 1 -- 172.21.15.114:0/3041948855 learned_addr learned my addr 172.21.15.114:0/3041948855 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:39.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.566+0000 7fa2d2a00640 1 --2- 172.21.15.114:0/3041948855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2d4074a60 0x7fa2d41f42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.566+0000 7fa2d3e00640 1 --2- 172.21.15.114:0/3041948855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2d40ca2d0 0x7fa2d4148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.566+0000 7fa2d3400640 1 -- 172.21.15.114:0/3041948855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2d40ca2d0 msgr2=0x7fa2d4148d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:39.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.566+0000 7fa2d3400640 1 --2- 172.21.15.114:0/3041948855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2d40ca2d0 0x7fa2d4148d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.566+0000 7fa2d3400640 1 -- 172.21.15.114:0/3041948855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2d4074a60 msgr2=0x7fa2d41f42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:39.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.566+0000 7fa2d3400640 1 --2- 172.21.15.114:0/3041948855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2d4074a60 0x7fa2d41f42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.566+0000 7fa2d3400640 1 -- 172.21.15.114:0/3041948855 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d41f4810 con 0x7fa2d4073260 2024-05-27T09:27:39.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.566+0000 7fa2d3400640 1 --2- 172.21.15.114:0/3041948855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2d4073260 0x7fa2d41557b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa2c402a2b0 tx=0x7fa2c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:39.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.567+0000 7fa2cb400640 1 -- 172.21.15.114:0/3041948855 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4041020 con 0x7fa2d4073260 2024-05-27T09:27:39.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.567+0000 7fa2cb400640 1 -- 172.21.15.114:0/3041948855 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c403d620 con 0x7fa2d4073260 2024-05-27T09:27:39.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.567+0000 7fa2cb400640 1 -- 172.21.15.114:0/3041948855 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4037040 con 0x7fa2d4073260 2024-05-27T09:27:39.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.567+0000 7fa2da600640 1 -- 172.21.15.114:0/3041948855 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d4155f70 con 0x7fa2d4073260 2024-05-27T09:27:39.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.567+0000 7fa2da600640 1 -- 172.21.15.114:0/3041948855 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d41f4a20 con 0x7fa2d4073260 2024-05-27T09:27:39.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.569+0000 7fa2cb400640 1 -- 172.21.15.114:0/3041948855 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa2c4044020 con 0x7fa2d4073260 2024-05-27T09:27:39.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.570+0000 7fa2cb400640 1 --2- 172.21.15.114:0/3041948855 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa2b40762d0 0x7fa2b40787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.571+0000 7fa2cb400640 1 -- 172.21.15.114:0/3041948855 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa2c40b7be0 con 0x7fa2d4073260 2024-05-27T09:27:39.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.571+0000 7fa2d2a00640 1 --2- 172.21.15.114:0/3041948855 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa2b40762d0 0x7fa2b40787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.571+0000 7fa2e11fd740 1 Processor -- start 2024-05-27T09:27:39.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.571+0000 7fa2e11fd740 1 -- start start 2024-05-27T09:27:39.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.572+0000 7fa2d2a00640 1 --2- 172.21.15.114:0/3041948855 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa2b40762d0 0x7fa2b40787e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4149730 tx=0x7fa2c0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:39.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.573+0000 7fa2e11fd740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5ae5270 0x560fc5e81e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.573+0000 7fa2e11fd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560fc5abc3b0 0x560fc5ec6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.573+0000 7fa2e11fd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5ac4590 0x560fc5ec65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.573+0000 7fa2e11fd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560fc5df8df0 con 0x560fc5ac4590 2024-05-27T09:27:39.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.573+0000 7fa2e11fd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560fc5de8990 con 0x560fc5abc3b0 2024-05-27T09:27:39.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.573+0000 7fa2e11fd740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x560fc5dec9d0 con 0x560fc5ae5270 2024-05-27T09:27:39.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.573+0000 7fa2d3400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5ae5270 0x560fc5e81e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.573+0000 7fa2d3400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5ae5270 0x560fc5e81e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:35696/0 (socket says 172.21.15.114:35696) 2024-05-27T09:27:39.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.573+0000 7fa2d3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5ac4590 0x560fc5ec65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.573+0000 7fa2d3400640 1 -- 172.21.15.114:0/194439968 learned_addr learned my addr 172.21.15.114:0/194439968 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:39.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.573+0000 7fa2d2a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560fc5abc3b0 0x560fc5ec6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.573+0000 7fa2d3400640 1 -- 172.21.15.114:0/194439968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560fc5abc3b0 msgr2=0x560fc5ec6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:39.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.573+0000 7fa2d3400640 1 --2- 172.21.15.114:0/194439968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560fc5abc3b0 0x560fc5ec6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.573+0000 7fa2d3400640 1 -- 172.21.15.114:0/194439968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5ac4590 msgr2=0x560fc5ec65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:39.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.573+0000 7fa2d3400640 1 --2- 172.21.15.114:0/194439968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5ac4590 0x560fc5ec65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.573+0000 7fa2d3400640 1 -- 172.21.15.114:0/194439968 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fc5d899b0 con 0x560fc5ae5270 2024-05-27T09:27:39.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.574+0000 7fa2d3400640 1 --2- 172.21.15.114:0/194439968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5ae5270 0x560fc5e81e00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa2c407d4d0 tx=0x7fa2c4080710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75cc23c203505d27 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:39.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.574+0000 7fa2bfe00640 1 -- 172.21.15.114:0/194439968 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c409a390 con 0x560fc5ae5270 2024-05-27T09:27:39.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.574+0000 7fa2bfe00640 1 -- 172.21.15.114:0/194439968 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c40851e0 con 0x560fc5ae5270 2024-05-27T09:27:39.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.574+0000 7fa2bfe00640 1 -- 172.21.15.114:0/194439968 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4089990 con 0x560fc5ae5270 2024-05-27T09:27:39.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.575+0000 7fa2e11fd740 1 -- 172.21.15.114:0/194439968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5ae5270 msgr2=0x560fc5e81e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:39.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.575+0000 7fa2e11fd740 1 --2- 172.21.15.114:0/194439968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5ae5270 0x560fc5e81e00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa2c407d4d0 tx=0x7fa2c4080710 comp rx=0 tx=0).stop 2024-05-27T09:27:39.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.575+0000 7fa2e11fd740 1 -- 172.21.15.114:0/194439968 shutdown_connections 2024-05-27T09:27:39.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.575+0000 7fa2e11fd740 1 --2- 172.21.15.114:0/194439968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5ac4590 0x560fc5ec65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.575+0000 7fa2e11fd740 1 --2- 172.21.15.114:0/194439968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560fc5abc3b0 0x560fc5ec6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.575+0000 7fa2e11fd740 1 --2- 172.21.15.114:0/194439968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5ae5270 0x560fc5e81e00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.575+0000 7fa2e11fd740 1 -- 172.21.15.114:0/194439968 >> 172.21.15.114:0/194439968 conn(0x560fc5ad93c0 msgr2=0x560fc5e85d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:39.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.575+0000 7fa2e11fd740 1 -- 172.21.15.114:0/194439968 shutdown_connections 2024-05-27T09:27:39.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.575+0000 7fa2e11fd740 1 -- 172.21.15.114:0/194439968 wait complete. 2024-05-27T09:27:39.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.576+0000 7fa2e11fd740 1 -- start start 2024-05-27T09:27:39.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.576+0000 7fa2e11fd740 1 Processor -- start 2024-05-27T09:27:39.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.578+0000 7fa2e11fd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5abc3b0 0x560fc5ec7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.578+0000 7fa2e11fd740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5ac4590 0x560fc5ec0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.578+0000 7fa2e11fd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560fc5ae5270 0x560fc5eabda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.578+0000 7fa2e11fd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560fc5ace850 con 0x560fc5abc3b0 2024-05-27T09:27:39.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.578+0000 7fa2e11fd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560fc5d90f20 con 0x560fc5ae5270 2024-05-27T09:27:39.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.578+0000 7fa2e11fd740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x560fc5e867d0 con 0x560fc5ac4590 2024-05-27T09:27:39.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.578+0000 7fa2e11fd740 10 client.0 _ll_register_callbacks cb 0x560fc5de2ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T09:27:39.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.578+0000 7fa2d3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5abc3b0 0x560fc5ec7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.578+0000 7fa2d3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5abc3b0 0x560fc5ec7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47916/0 (socket says 172.21.15.114:47916) 2024-05-27T09:27:39.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.578+0000 7fa2d3400640 1 -- 172.21.15.114:0/2484684341 learned_addr learned my addr 172.21.15.114:0/2484684341 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:39.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.578+0000 7fa2d3e00640 1 --2- 172.21.15.114:0/2484684341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560fc5ae5270 0x560fc5eabda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.578+0000 7fa2d2a00640 1 --2- 172.21.15.114:0/2484684341 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5ac4590 0x560fc5ec0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.578+0000 7fa2d3400640 1 -- 172.21.15.114:0/2484684341 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5ac4590 msgr2=0x560fc5ec0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:39.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.578+0000 7fa2d3400640 1 --2- 172.21.15.114:0/2484684341 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5ac4590 0x560fc5ec0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.578+0000 7fa2d3400640 1 -- 172.21.15.114:0/2484684341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560fc5ae5270 msgr2=0x560fc5eabda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:39.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.578+0000 7fa2d3400640 1 --2- 172.21.15.114:0/2484684341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560fc5ae5270 0x560fc5eabda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.578+0000 7fa2d3400640 1 -- 172.21.15.114:0/2484684341 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fc5d81d00 con 0x560fc5abc3b0 2024-05-27T09:27:39.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.579+0000 7fa2d3400640 1 --2- 172.21.15.114:0/2484684341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5abc3b0 0x560fc5ec7350 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4000c00 tx=0x7fa2c40095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c573d40358fc2cf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:39.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.579+0000 7fa2bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-05-27T09:27:39.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.579+0000 7fa2bcc00640 1 -- 172.21.15.114:0/2484684341 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c409a390 con 0x560fc5abc3b0 2024-05-27T09:27:39.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.579+0000 7fa2bcc00640 1 -- 172.21.15.114:0/2484684341 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa2c40873f0 con 0x560fc5abc3b0 2024-05-27T09:27:39.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.579+0000 7fa2e11fd740 1 -- 172.21.15.114:0/2484684341 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2c40851e0 con 0x560fc5abc3b0 2024-05-27T09:27:39.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.579+0000 7fa2bcc00640 1 -- 172.21.15.114:0/2484684341 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) ==== 24+0+0 (secure 0 0 0) 0x7fa2c4089c00 con 0x560fc5abc3b0 2024-05-27T09:27:39.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.579+0000 7fa2e11fd740 10 client.4557 fetch_fsmap learned FSMap version 12 2024-05-27T09:27:39.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.579+0000 7fa2e11fd740 1 -- 172.21.15.114:0/2484684341 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) -- 0x560fc5afd2d0 con 0x560fc5abc3b0 2024-05-27T09:27:39.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.579+0000 7fa2bcc00640 1 -- 172.21.15.114:0/2484684341 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c4089e50 con 0x560fc5abc3b0 2024-05-27T09:27:39.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.580+0000 7fa2bcc00640 1 -- 172.21.15.114:0/2484684341 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 12) ==== 4744+0+0 (secure 0 0 0) 0x7fa2c403a030 con 0x560fc5abc3b0 2024-05-27T09:27:39.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.580+0000 7fa2e11fd740 10 client.4557 fetch_fsmap finished waiting for FSMap version 12 2024-05-27T09:27:39.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.580+0000 7fa2e11fd740 10 client.4557 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-27T09:27:39.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.580+0000 7fa2e11fd740 20 client.4557 populate_metadata read hostname 'smithi114' 2024-05-27T09:27:39.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.580+0000 7fa2e11fd740 1 --2- 172.21.15.114:0/2484684341 >> [v2:172.21.15.114:6836/1448950017,v1:172.21.15.114:6837/1448950017] conn(0x560fc5aa3e40 0x560fc5ec2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.580+0000 7fa2e11fd740 4 client.4557 mds_command: new command op to 4300 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T09:27:39.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.580+0000 7fa2e11fd740 1 -- 172.21.15.114:0/2484684341 --> [v2:172.21.15.114:6836/1448950017,v1:172.21.15.114:6837/1448950017] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fc5dfad40 con 0x560fc5aa3e40 2024-05-27T09:27:39.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.580+0000 7fa2d2a00640 1 --2- 172.21.15.114:0/2484684341 >> [v2:172.21.15.114:6836/1448950017,v1:172.21.15.114:6837/1448950017] conn(0x560fc5aa3e40 0x560fc5ec2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.581+0000 7fa2d2a00640 1 --2- 172.21.15.114:0/2484684341 >> [v2:172.21.15.114:6836/1448950017,v1:172.21.15.114:6837/1448950017] conn(0x560fc5aa3e40 0x560fc5ec2170 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9da1c46b58801e3 server_cookie=c8e0bdee922b50fb in_seq=0 out_seq=0 2024-05-27T09:27:39.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.581+0000 7fa2bcc00640 10 client.4557 ms_handle_connect on v2:172.21.15.114:6836/1448950017 2024-05-27T09:27:39.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.581+0000 7fa2bcc00640 1 -- 172.21.15.114:0/2484684341 <== mds.0 v2:172.21.15.114:6836/1448950017 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560fc5dfad40 con 0x560fc5aa3e40 2024-05-27T09:27:39.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.581+0000 7fa2bcc00640 10 client.4557 handle_command_reply: tid=0 2024-05-27T09:27:39.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.582+0000 7fa2e11fd740 1 client.4557 shutdown 2024-05-27T09:27:39.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.582+0000 7fa2e11fd740 1 -- 172.21.15.114:0/2484684341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5abc3b0 msgr2=0x560fc5ec7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:39.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.582+0000 7fa2e11fd740 1 --2- 172.21.15.114:0/2484684341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5abc3b0 0x560fc5ec7350 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4000c00 tx=0x7fa2c40095f0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.582+0000 7fa2e11fd740 1 -- 172.21.15.114:0/2484684341 shutdown_connections 2024-05-27T09:27:39.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.582+0000 7fa2e11fd740 1 --2- 172.21.15.114:0/2484684341 >> [v2:172.21.15.114:6836/1448950017,v1:172.21.15.114:6837/1448950017] conn(0x560fc5aa3e40 0x560fc5ec2170 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.582+0000 7fa2e11fd740 1 --2- 172.21.15.114:0/2484684341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560fc5ae5270 0x560fc5eabda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.582+0000 7fa2e11fd740 1 --2- 172.21.15.114:0/2484684341 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5ac4590 0x560fc5ec0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.582+0000 7fa2e11fd740 1 --2- 172.21.15.114:0/2484684341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5abc3b0 0x560fc5ec7350 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.582+0000 7fa2bcc00640 0 client.4557 ms_handle_reset on v2:172.21.15.114:6836/1448950017 2024-05-27T09:27:39.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.582+0000 7fa2e11fd740 1 -- 172.21.15.114:0/2484684341 >> 172.21.15.114:0/2484684341 conn(0x560fc5ad93c0 msgr2=0x560fc5e869d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:39.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.582+0000 7fa2e11fd740 1 -- 172.21.15.114:0/2484684341 shutdown_connections 2024-05-27T09:27:39.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.582+0000 7fa2e11fd740 1 -- 172.21.15.114:0/2484684341 wait complete. 2024-05-27T09:27:39.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.582+0000 7fa2e11fd740 20 client.4557 trim_cache size 0 max 16384 2024-05-27T09:27:39.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.603+0000 7fa2e11fd740 1 Processor -- start 2024-05-27T09:27:39.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.603+0000 7fa2e11fd740 1 -- start start 2024-05-27T09:27:39.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.605+0000 7fa2e11fd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560fc5abc3b0 0x560fc5e86dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.605+0000 7fa2e11fd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5ac4590 0x560fc5e81b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.605+0000 7fa2e11fd740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5ae5270 0x560fc5e82070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.605+0000 7fa2e11fd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d40c44b0 con 0x560fc5ac4590 2024-05-27T09:27:39.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.605+0000 7fa2e11fd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560fc5dec9d0 con 0x560fc5abc3b0 2024-05-27T09:27:39.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.605+0000 7fa2e11fd740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x560fc5df8df0 con 0x560fc5ae5270 2024-05-27T09:27:39.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.605+0000 7fa2d2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5ac4590 0x560fc5e81b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.605+0000 7fa2d2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5ac4590 0x560fc5e81b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47924/0 (socket says 172.21.15.114:47924) 2024-05-27T09:27:39.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.605+0000 7fa2d2a00640 1 -- 172.21.15.114:0/1599921610 learned_addr learned my addr 172.21.15.114:0/1599921610 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:39.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.605+0000 7fa2d3400640 1 --2- 172.21.15.114:0/1599921610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560fc5abc3b0 0x560fc5e86dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.605+0000 7fa2d3e00640 1 --2- 172.21.15.114:0/1599921610 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5ae5270 0x560fc5e82070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.605+0000 7fa2d2a00640 1 -- 172.21.15.114:0/1599921610 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5ae5270 msgr2=0x560fc5e82070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:39.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.605+0000 7fa2d2a00640 1 --2- 172.21.15.114:0/1599921610 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5ae5270 0x560fc5e82070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.605+0000 7fa2d2a00640 1 -- 172.21.15.114:0/1599921610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560fc5abc3b0 msgr2=0x560fc5e86dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:39.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.605+0000 7fa2d2a00640 1 --2- 172.21.15.114:0/1599921610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560fc5abc3b0 0x560fc5e86dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.605+0000 7fa2d2a00640 1 -- 172.21.15.114:0/1599921610 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fc5bc76b0 con 0x560fc5ac4590 2024-05-27T09:27:39.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.605+0000 7fa2d3400640 1 --2- 172.21.15.114:0/1599921610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560fc5abc3b0 0x560fc5e86dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:27:39.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.606+0000 7fa2d2a00640 1 --2- 172.21.15.114:0/1599921610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5ac4590 0x560fc5e81b30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4073210 tx=0x7fa2c0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7b0f7e6a8093e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:39.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.606+0000 7fa2bf400640 1 -- 172.21.15.114:0/1599921610 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c00036d0 con 0x560fc5ac4590 2024-05-27T09:27:39.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.606+0000 7fa2bf400640 1 -- 172.21.15.114:0/1599921610 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c0032070 con 0x560fc5ac4590 2024-05-27T09:27:39.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.606+0000 7fa2bf400640 1 -- 172.21.15.114:0/1599921610 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0003a70 con 0x560fc5ac4590 2024-05-27T09:27:39.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.607+0000 7fa2e11fd740 1 -- 172.21.15.114:0/1599921610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5ac4590 msgr2=0x560fc5e81b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:39.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.607+0000 7fa2e11fd740 1 --2- 172.21.15.114:0/1599921610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5ac4590 0x560fc5e81b30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4073210 tx=0x7fa2c0019000 comp rx=0 tx=0).stop 2024-05-27T09:27:39.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.607+0000 7fa2e11fd740 1 -- 172.21.15.114:0/1599921610 shutdown_connections 2024-05-27T09:27:39.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.607+0000 7fa2e11fd740 1 --2- 172.21.15.114:0/1599921610 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5ae5270 0x560fc5e82070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.607+0000 7fa2e11fd740 1 --2- 172.21.15.114:0/1599921610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5ac4590 0x560fc5e81b30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.607+0000 7fa2e11fd740 1 --2- 172.21.15.114:0/1599921610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560fc5abc3b0 0x560fc5e86dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.607+0000 7fa2e11fd740 1 -- 172.21.15.114:0/1599921610 >> 172.21.15.114:0/1599921610 conn(0x560fc5ad93c0 msgr2=0x560fc5ebd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:39.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.607+0000 7fa2e11fd740 1 -- 172.21.15.114:0/1599921610 shutdown_connections 2024-05-27T09:27:39.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.607+0000 7fa2e11fd740 1 -- 172.21.15.114:0/1599921610 wait complete. 2024-05-27T09:27:39.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.608+0000 7fa2e11fd740 1 -- start start 2024-05-27T09:27:39.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.608+0000 7fa2e11fd740 1 Processor -- start 2024-05-27T09:27:39.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.610+0000 7fa2e11fd740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5abc3b0 0x560fc5e81f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.610+0000 7fa2e11fd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560fc5ac4590 0x560fc5eaf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.610+0000 7fa2e11fd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5ae5270 0x560fc5f4d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.610+0000 7fa2e11fd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560fc5eac9d0 con 0x560fc5ae5270 2024-05-27T09:27:39.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.610+0000 7fa2e11fd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560fc5edf340 con 0x560fc5ac4590 2024-05-27T09:27:39.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.610+0000 7fa2e11fd740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x560fc5ebf920 con 0x560fc5abc3b0 2024-05-27T09:27:39.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.610+0000 7fa2e11fd740 10 client.0 _ll_register_callbacks cb 0x7fa2cc009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T09:27:39.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.610+0000 7fa2d3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5ae5270 0x560fc5f4d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.610+0000 7fa2d3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5ae5270 0x560fc5f4d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47938/0 (socket says 172.21.15.114:47938) 2024-05-27T09:27:39.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.610+0000 7fa2d3e00640 1 -- 172.21.15.114:0/117667230 learned_addr learned my addr 172.21.15.114:0/117667230 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:39.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.610+0000 7fa2d2a00640 1 --2- 172.21.15.114:0/117667230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560fc5ac4590 0x560fc5eaf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.610+0000 7fa2d3400640 1 --2- 172.21.15.114:0/117667230 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5abc3b0 0x560fc5e81f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.610+0000 7fa2d3e00640 1 -- 172.21.15.114:0/117667230 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5abc3b0 msgr2=0x560fc5e81f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:39.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.610+0000 7fa2d3e00640 1 --2- 172.21.15.114:0/117667230 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5abc3b0 0x560fc5e81f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.610+0000 7fa2d3e00640 1 -- 172.21.15.114:0/117667230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560fc5ac4590 msgr2=0x560fc5eaf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:39.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.610+0000 7fa2d3e00640 1 --2- 172.21.15.114:0/117667230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560fc5ac4590 0x560fc5eaf360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.610+0000 7fa2d3e00640 1 -- 172.21.15.114:0/117667230 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fc5ea8bf0 con 0x560fc5ae5270 2024-05-27T09:27:39.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.611+0000 7fa2d3400640 1 --2- 172.21.15.114:0/117667230 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5abc3b0 0x560fc5e81f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:39.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.611+0000 7fa2d2a00640 1 --2- 172.21.15.114:0/117667230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560fc5ac4590 0x560fc5eaf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:39.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.611+0000 7fa2d3e00640 1 --2- 172.21.15.114:0/117667230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5ae5270 0x560fc5f4d190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc017f60 tx=0x7fa2cc017f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63131c186ae814ce server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:39.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.611+0000 7fa2bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-05-27T09:27:39.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.611+0000 7fa2bcc00640 1 -- 172.21.15.114:0/117667230 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc014590 con 0x560fc5ae5270 2024-05-27T09:27:39.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.611+0000 7fa2bcc00640 1 -- 172.21.15.114:0/117667230 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa2cc014730 con 0x560fc5ae5270 2024-05-27T09:27:39.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.611+0000 7fa2e11fd740 1 -- 172.21.15.114:0/117667230 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2c0032070 con 0x560fc5ae5270 2024-05-27T09:27:39.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.611+0000 7fa2bcc00640 1 -- 172.21.15.114:0/117667230 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) ==== 24+0+0 (secure 0 0 0) 0x7fa2cc02c050 con 0x560fc5ae5270 2024-05-27T09:27:39.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.611+0000 7fa2e11fd740 10 client.4563 fetch_fsmap learned FSMap version 12 2024-05-27T09:27:39.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.611+0000 7fa2e11fd740 1 -- 172.21.15.114:0/117667230 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) -- 0x560fc5ebe0b0 con 0x560fc5ae5270 2024-05-27T09:27:39.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.612+0000 7fa2bcc00640 1 -- 172.21.15.114:0/117667230 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c0032070 con 0x560fc5ae5270 2024-05-27T09:27:39.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.612+0000 7fa2bcc00640 1 -- 172.21.15.114:0/117667230 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 12) ==== 4744+0+0 (secure 0 0 0) 0x7fa2cc02f020 con 0x560fc5ae5270 2024-05-27T09:27:39.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.612+0000 7fa2e11fd740 10 client.4563 fetch_fsmap finished waiting for FSMap version 12 2024-05-27T09:27:39.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.612+0000 7fa2e11fd740 10 client.4563 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-27T09:27:39.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.612+0000 7fa2e11fd740 20 client.4563 populate_metadata read hostname 'smithi114' 2024-05-27T09:27:39.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.612+0000 7fa2e11fd740 1 --2- 172.21.15.114:0/117667230 >> [v2:172.21.15.114:6836/1448950017,v1:172.21.15.114:6837/1448950017] conn(0x560fc5e850b0 0x560fc5f59580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:39.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.612+0000 7fa2e11fd740 4 client.4563 mds_command: new command op to 4300 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T09:27:39.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.612+0000 7fa2e11fd740 1 -- 172.21.15.114:0/117667230 --> [v2:172.21.15.114:6836/1448950017,v1:172.21.15.114:6837/1448950017] -- command(tid 0: {"prefix": "status"}) -- 0x560fc5ec1ca0 con 0x560fc5e850b0 2024-05-27T09:27:39.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.612+0000 7fa2d3400640 1 --2- 172.21.15.114:0/117667230 >> [v2:172.21.15.114:6836/1448950017,v1:172.21.15.114:6837/1448950017] conn(0x560fc5e850b0 0x560fc5f59580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:39.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.613+0000 7fa2d3400640 1 --2- 172.21.15.114:0/117667230 >> [v2:172.21.15.114:6836/1448950017,v1:172.21.15.114:6837/1448950017] conn(0x560fc5e850b0 0x560fc5f59580 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e8785f74cbb1312 server_cookie=b6f9ec6bc83c6419 in_seq=0 out_seq=0 2024-05-27T09:27:39.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.613+0000 7fa2bcc00640 10 client.4563 ms_handle_connect on v2:172.21.15.114:6836/1448950017 2024-05-27T09:27:39.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.613+0000 7fa2bcc00640 1 -- 172.21.15.114:0/117667230 <== mds.0 v2:172.21.15.114:6836/1448950017 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x560fc5ec1ca0 con 0x560fc5e850b0 2024-05-27T09:27:39.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.613+0000 7fa2bcc00640 10 client.4563 handle_command_reply: tid=0 2024-05-27T09:27:39.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.613+0000 7fa2e11fd740 1 client.4563 shutdown 2024-05-27T09:27:39.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.613+0000 7fa2e11fd740 1 -- 172.21.15.114:0/117667230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5ae5270 msgr2=0x560fc5f4d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:39.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.613+0000 7fa2e11fd740 1 --2- 172.21.15.114:0/117667230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5ae5270 0x560fc5f4d190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc017f60 tx=0x7fa2cc017f90 comp rx=0 tx=0).stop 2024-05-27T09:27:39.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.613+0000 7fa2e11fd740 1 -- 172.21.15.114:0/117667230 shutdown_connections 2024-05-27T09:27:39.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.613+0000 7fa2e11fd740 1 --2- 172.21.15.114:0/117667230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560fc5ae5270 0x560fc5f4d190 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.613+0000 7fa2e11fd740 1 --2- 172.21.15.114:0/117667230 >> [v2:172.21.15.114:6836/1448950017,v1:172.21.15.114:6837/1448950017] conn(0x560fc5e850b0 0x560fc5f59580 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.613+0000 7fa2e11fd740 1 --2- 172.21.15.114:0/117667230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560fc5ac4590 0x560fc5eaf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.613+0000 7fa2e11fd740 1 --2- 172.21.15.114:0/117667230 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560fc5abc3b0 0x560fc5e81f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.613+0000 7fa2bcc00640 0 client.4563 ms_handle_reset on v2:172.21.15.114:6836/1448950017 2024-05-27T09:27:39.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.613+0000 7fa2e11fd740 1 -- 172.21.15.114:0/117667230 >> 172.21.15.114:0/117667230 conn(0x560fc5ad93c0 msgr2=0x560fc5ec2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:39.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.613+0000 7fa2e11fd740 1 -- 172.21.15.114:0/117667230 shutdown_connections 2024-05-27T09:27:39.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.613+0000 7fa2e11fd740 1 -- 172.21.15.114:0/117667230 wait complete. 2024-05-27T09:27:39.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.614+0000 7fa2e11fd740 20 client.4563 trim_cache size 0 max 16384 2024-05-27T09:27:39.618 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-05-27T09:27:39.618 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48", 2024-05-27T09:27:39.618 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 0, 2024-05-27T09:27:39.618 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 4300, 2024-05-27T09:27:39.619 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-05-27T09:27:39.619 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-05-27T09:27:39.619 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-05-27T09:27:39.619 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 21.081779206, 2024-05-27T09:27:39.619 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 12, 2024-05-27T09:27:39.619 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 21, 2024-05-27T09:27:39.619 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 21, 2024-05-27T09:27:39.619 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 27.501021934000001 2024-05-27T09:27:39.619 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-05-27T09:27:39.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.614+0000 7fa2da600640 1 -- 172.21.15.114:0/3041948855 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa2b40762d0 msgr2=0x7fa2b40787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:39.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.614+0000 7fa2da600640 1 --2- 172.21.15.114:0/3041948855 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa2b40762d0 0x7fa2b40787e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4149730 tx=0x7fa2c0009190 comp rx=0 tx=0).stop 2024-05-27T09:27:39.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.614+0000 7fa2da600640 1 -- 172.21.15.114:0/3041948855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2d4073260 msgr2=0x7fa2d41557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:39.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.614+0000 7fa2da600640 1 --2- 172.21.15.114:0/3041948855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2d4073260 0x7fa2d41557b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa2c402a2b0 tx=0x7fa2c40046c0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.614+0000 7fa2da600640 1 -- 172.21.15.114:0/3041948855 shutdown_connections 2024-05-27T09:27:39.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.614+0000 7fa2da600640 1 --2- 172.21.15.114:0/3041948855 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa2b40762d0 0x7fa2b40787e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.614+0000 7fa2da600640 1 --2- 172.21.15.114:0/3041948855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2d40ca2d0 0x7fa2d4148d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.614+0000 7fa2da600640 1 --2- 172.21.15.114:0/3041948855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2d4074a60 0x7fa2d41f42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.614+0000 7fa2da600640 1 --2- 172.21.15.114:0/3041948855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa2d4073260 0x7fa2d41557b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:39.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.614+0000 7fa2da600640 1 -- 172.21.15.114:0/3041948855 >> 172.21.15.114:0/3041948855 conn(0x7fa2d406f6e0 msgr2=0x7fa2d41525b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:39.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.614+0000 7fa2da600640 1 -- 172.21.15.114:0/3041948855 shutdown_connections 2024-05-27T09:27:39.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:39.615+0000 7fa2da600640 1 -- 172.21.15.114:0/3041948855 wait complete. 2024-05-27T09:27:39.643 INFO:teuthology.run_tasks:Running task sequential... 2024-05-27T09:27:39.653 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-27T09:27:39.653 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-27T09:27:39.653 INFO:teuthology.task.print:Enabling mgr modules 2024-05-27T09:27:39.653 INFO:teuthology.run_tasks:Running task kclient... 2024-05-27T09:27:39.677 INFO:tasks.kclient:Mounting kernel clients... 2024-05-27T09:27:39.678 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-27T09:27:39.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:27:39.678 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2024-05-27T09:27:39.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:27:39.732 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link delete ceph-brx 2024-05-27T09:27:39.817 INFO:teuthology.orchestra.run.smithi114.stderr:Cannot find device "ceph-brx" 2024-05-27T09:27:39.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:27:39.821 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-05-27T09:27:39.821 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-27T09:27:39.821 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-27T09:27:39.821 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-27T09:27:40.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.093+0000 7f50ae000640 1 Processor -- start 2024-05-27T09:27:40.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.093+0000 7f50ae000640 1 -- start start 2024-05-27T09:27:40.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.094+0000 7f50ae000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50a8152d90 0x7f50a80c9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:40.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.095+0000 7f50ae000640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50a8074780 con 0x7f50a8159540 2024-05-27T09:27:40.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.095+0000 7f50ae000640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50a8074a60 con 0x7f50a80b34a0 2024-05-27T09:27:40.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.095+0000 7f50ae000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f50a8074530 con 0x7f50a8152d90 2024-05-27T09:27:40.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.095+0000 7f50a7e00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f50a8159540 0x7f50a80c85f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:46922/0 (socket says 172.21.15.114:46922) 2024-05-27T09:27:40.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.095+0000 7f50a7e00640 1 -- 172.21.15.114:0/3516740136 learned_addr learned my addr 172.21.15.114:0/3516740136 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:40.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.095+0000 7f50acc00640 1 --2- 172.21.15.114:0/3516740136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50a8152d90 0x7f50a80c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:40.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.095+0000 7f50a7400640 1 -- 172.21.15.114:0/3516740136 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4171703857 0 0) 0x7f50a8074a60 con 0x7f50a80b34a0 2024-05-27T09:27:40.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.095+0000 7f50a7400640 1 -- 172.21.15.114:0/3516740136 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f508c003440 con 0x7f50a80b34a0 2024-05-27T09:27:40.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.095+0000 7f50a7400640 1 -- 172.21.15.114:0/3516740136 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3186999254 0 0) 0x7f50a8074780 con 0x7f50a8159540 2024-05-27T09:27:40.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.095+0000 7f50acc00640 1 -- 172.21.15.114:0/3516740136 >> v1:172.21.15.163:6789/0 conn(0x7f50a80b34a0 legacy=0x7f50a806f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:40.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.095+0000 7f50acc00640 1 -- 172.21.15.114:0/3516740136 >> v1:172.21.15.114:6789/0 conn(0x7f50a8159540 legacy=0x7f50a80c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:40.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.096+0000 7f50acc00640 1 -- 172.21.15.114:0/3516740136 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a8158a60 con 0x7f50a8152d90 2024-05-27T09:27:40.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.097+0000 7f50acc00640 1 --2- 172.21.15.114:0/3516740136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50a8152d90 0x7f50a80c9130 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5098009dc0 tx=0x7f509802a760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22b2371cb9b82aa0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:40.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.097+0000 7f50a7400640 1 -- 172.21.15.114:0/3516740136 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5098007740 con 0x7f50a8152d90 2024-05-27T09:27:40.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.097+0000 7f50a7400640 1 -- 172.21.15.114:0/3516740136 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50980078e0 con 0x7f50a8152d90 2024-05-27T09:27:40.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.097+0000 7f50a7400640 1 -- 172.21.15.114:0/3516740136 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5098007cb0 con 0x7f50a8152d90 2024-05-27T09:27:40.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.098+0000 7f50ae000640 1 -- 172.21.15.114:0/3516740136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50a8152d90 msgr2=0x7f50a80c9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:40.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.098+0000 7f50ae000640 1 --2- 172.21.15.114:0/3516740136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50a8152d90 0x7f50a80c9130 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5098009dc0 tx=0x7f509802a760 comp rx=0 tx=0).stop 2024-05-27T09:27:40.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.098+0000 7f50ae000640 1 -- 172.21.15.114:0/3516740136 shutdown_connections 2024-05-27T09:27:40.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.098+0000 7f50ae000640 1 --2- 172.21.15.114:0/3516740136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50a8152d90 0x7f50a80c9130 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:40.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.098+0000 7f50ae000640 1 -- 172.21.15.114:0/3516740136 >> 172.21.15.114:0/3516740136 conn(0x7f50a80ca320 msgr2=0x7f50a80ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:40.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.098+0000 7f50ae000640 1 -- 172.21.15.114:0/3516740136 shutdown_connections 2024-05-27T09:27:40.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.098+0000 7f50ae000640 1 -- 172.21.15.114:0/3516740136 wait complete. 2024-05-27T09:27:40.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.100+0000 7f50ae000640 1 Processor -- start 2024-05-27T09:27:40.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.100+0000 7f50ae000640 1 -- start start 2024-05-27T09:27:40.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.102+0000 7f50ae000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a80b34a0 0x7f50a81567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:40.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.102+0000 7f50ae000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50a8152d90 0x7f50a81573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:40.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.102+0000 7f50ae000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a8159540 0x7f50a80c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:40.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.102+0000 7f50ae000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f50a80c9670 con 0x7f50a8159540 2024-05-27T09:27:40.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.102+0000 7f50ae000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81590b0 con 0x7f50a80b34a0 2024-05-27T09:27:40.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.102+0000 7f50ae000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f50a80c3270 con 0x7f50a8152d90 2024-05-27T09:27:40.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.102+0000 7f50ad600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a8159540 0x7f50a80c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:40.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.102+0000 7f50a7e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50a8152d90 0x7f50a81573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:40.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.102+0000 7f50acc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a80b34a0 0x7f50a81567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:40.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.102+0000 7f50ad600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a8159540 0x7f50a80c62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47942/0 (socket says 172.21.15.114:47942) 2024-05-27T09:27:40.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.102+0000 7f50acc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a80b34a0 0x7f50a81567d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:44660/0 (socket says 172.21.15.114:44660) 2024-05-27T09:27:40.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.102+0000 7f50ad600640 1 -- 172.21.15.114:0/1011939624 learned_addr learned my addr 172.21.15.114:0/1011939624 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:40.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.103+0000 7f50acc00640 1 -- 172.21.15.114:0/1011939624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50a8152d90 msgr2=0x7f50a81573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:40.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.103+0000 7f50acc00640 1 --2- 172.21.15.114:0/1011939624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50a8152d90 0x7f50a81573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:40.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.103+0000 7f50acc00640 1 -- 172.21.15.114:0/1011939624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a8159540 msgr2=0x7f50a80c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:40.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.103+0000 7f50acc00640 1 --2- 172.21.15.114:0/1011939624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a8159540 0x7f50a80c62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:40.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.103+0000 7f50acc00640 1 -- 172.21.15.114:0/1011939624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a8156d10 con 0x7f50a80b34a0 2024-05-27T09:27:40.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.103+0000 7f50a7e00640 1 --2- 172.21.15.114:0/1011939624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50a8152d90 0x7f50a81573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:40.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.103+0000 7f50acc00640 1 --2- 172.21.15.114:0/1011939624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a80b34a0 0x7f50a81567d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5098031000 tx=0x7f5098004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:40.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.103+0000 7f50a4c00640 1 -- 172.21.15.114:0/1011939624 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509803f070 con 0x7f50a80b34a0 2024-05-27T09:27:40.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.103+0000 7f50a4c00640 1 -- 172.21.15.114:0/1011939624 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5098031c40 con 0x7f50a80b34a0 2024-05-27T09:27:40.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.103+0000 7f50a4c00640 1 -- 172.21.15.114:0/1011939624 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5098009450 con 0x7f50a80b34a0 2024-05-27T09:27:40.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.103+0000 7f50ae000640 1 -- 172.21.15.114:0/1011939624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a814c990 con 0x7f50a80b34a0 2024-05-27T09:27:40.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.103+0000 7f50ae000640 1 -- 172.21.15.114:0/1011939624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a8157f50 con 0x7f50a80b34a0 2024-05-27T09:27:40.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.105+0000 7f50ae000640 1 -- 172.21.15.114:0/1011939624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a806a380 con 0x7f50a80b34a0 2024-05-27T09:27:40.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.106+0000 7f50a4c00640 1 -- 172.21.15.114:0/1011939624 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f509803b410 con 0x7f50a80b34a0 2024-05-27T09:27:40.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.107+0000 7f50a4c00640 1 --2- 172.21.15.114:0/1011939624 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5088076300 0x7f5088078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:40.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.107+0000 7f50a4c00640 1 -- 172.21.15.114:0/1011939624 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f50980b85d0 con 0x7f50a80b34a0 2024-05-27T09:27:40.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.107+0000 7f50a7e00640 1 --2- 172.21.15.114:0/1011939624 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5088076300 0x7f5088078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:40.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.108+0000 7f50a7e00640 1 --2- 172.21.15.114:0/1011939624 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5088076300 0x7f5088078810 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f50a8141490 tx=0x7f5094007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:40.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.112+0000 7f50a4c00640 1 -- 172.21.15.114:0/1011939624 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f509804b070 con 0x7f50a80b34a0 2024-05-27T09:27:40.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.361+0000 7f50ae000640 1 -- 172.21.15.114:0/1011939624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f50a8156f20 con 0x7f50a80b34a0 2024-05-27T09:27:40.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.362+0000 7f50a4c00640 1 -- 172.21.15.114:0/1011939624 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v12) ==== 53+0+73 (secure 0 0 0) 0x7f5098081840 con 0x7f50a80b34a0 2024-05-27T09:27:40.363 INFO:teuthology.orchestra.run.smithi114.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-27T09:27:40.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.364+0000 7f50ae000640 1 -- 172.21.15.114:0/1011939624 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5088076300 msgr2=0x7f5088078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:40.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.364+0000 7f50ae000640 1 --2- 172.21.15.114:0/1011939624 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5088076300 0x7f5088078810 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f50a8141490 tx=0x7f5094007de0 comp rx=0 tx=0).stop 2024-05-27T09:27:40.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.364+0000 7f50ae000640 1 -- 172.21.15.114:0/1011939624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a80b34a0 msgr2=0x7f50a81567d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:40.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.364+0000 7f50ae000640 1 --2- 172.21.15.114:0/1011939624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a80b34a0 0x7f50a81567d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5098031000 tx=0x7f5098004a40 comp rx=0 tx=0).stop 2024-05-27T09:27:40.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.365+0000 7f50ae000640 1 -- 172.21.15.114:0/1011939624 shutdown_connections 2024-05-27T09:27:40.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.365+0000 7f50ae000640 1 --2- 172.21.15.114:0/1011939624 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5088076300 0x7f5088078810 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:40.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.365+0000 7f50ae000640 1 --2- 172.21.15.114:0/1011939624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50a8159540 0x7f50a80c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:40.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.365+0000 7f50ae000640 1 --2- 172.21.15.114:0/1011939624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50a8152d90 0x7f50a81573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:40.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.365+0000 7f50ae000640 1 --2- 172.21.15.114:0/1011939624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a80b34a0 0x7f50a81567d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:40.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.365+0000 7f50ae000640 1 -- 172.21.15.114:0/1011939624 >> 172.21.15.114:0/1011939624 conn(0x7f50a80ca320 msgr2=0x7f50a8073060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:40.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.365+0000 7f50ae000640 1 -- 172.21.15.114:0/1011939624 shutdown_connections 2024-05-27T09:27:40.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40.365+0000 7f50ae000640 1 -- 172.21.15.114:0/1011939624 wait complete. 2024-05-27T09:27:40.391 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-27T09:27:40.391 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-27T09:27:40.391 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi114.front.sepia.ceph.com 2024-05-27T09:27:40.391 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-27T09:27:40.391 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-27T09:27:40.391 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-27T09:27:40.391 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-27T09:27:40.391 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-27T09:27:40.391 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-27T09:27:40.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:27:40.392 DEBUG:teuthology.orchestra.run.smithi114:> ip addr 2024-05-27T09:27:40.440 INFO:teuthology.orchestra.run.smithi114.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T09:27:40.440 INFO:teuthology.orchestra.run.smithi114.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T09:27:40.440 INFO:teuthology.orchestra.run.smithi114.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T09:27:40.440 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-05-27T09:27:40.440 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 ::1/128 scope host 2024-05-27T09:27:40.440 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-05-27T09:27:40.440 INFO:teuthology.orchestra.run.smithi114.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T09:27:40.440 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:92 brd ff:ff:ff:ff:ff:ff 2024-05-27T09:27:40.441 INFO:teuthology.orchestra.run.smithi114.stdout: altname enp7s0f0 2024-05-27T09:27:40.441 INFO:teuthology.orchestra.run.smithi114.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T09:27:40.441 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:93 brd ff:ff:ff:ff:ff:ff 2024-05-27T09:27:40.441 INFO:teuthology.orchestra.run.smithi114.stdout: altname enp7s0f1 2024-05-27T09:27:40.441 INFO:teuthology.orchestra.run.smithi114.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T09:27:40.441 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:62 brd ff:ff:ff:ff:ff:ff 2024-05-27T09:27:40.441 INFO:teuthology.orchestra.run.smithi114.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-27T09:27:40.441 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:63 brd ff:ff:ff:ff:ff:ff 2024-05-27T09:27:40.441 INFO:teuthology.orchestra.run.smithi114.stdout: inet 172.21.15.114/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-27T09:27:40.441 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft 42593sec preferred_lft 42593sec 2024-05-27T09:27:40.441 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::ec4:7aff:fe8f:cb63/64 scope link 2024-05-27T09:27:40.442 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-05-27T09:27:40.443 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-27T09:27:40.443 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T09:27:40.443 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-05-27T09:27:40.443 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link add name ceph-brx type bridge 2024-05-27T09:27:40.443 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip addr flush dev ceph-brx 2024-05-27T09:27:40.443 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set ceph-brx up 2024-05-27T09:27:40.443 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-27T09:27:40.443 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-05-27T09:27:40.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T09:27:40.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:27:40.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:27:40.880 DEBUG:teuthology.orchestra.run.smithi114:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-27T09:27:40.938 INFO:teuthology.orchestra.run.smithi114.stdout:1 2024-05-27T09:27:40.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:27:40.942 DEBUG:teuthology.orchestra.run.smithi114:> route 2024-05-27T09:27:41.007 INFO:teuthology.orchestra.run.smithi114.stdout:Kernel IP routing table 2024-05-27T09:27:41.008 INFO:teuthology.orchestra.run.smithi114.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-27T09:27:41.008 INFO:teuthology.orchestra.run.smithi114.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-27T09:27:41.008 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-27T09:27:41.008 INFO:teuthology.orchestra.run.smithi114.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-27T09:27:41.008 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T09:27:41.008 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-05-27T09:27:41.009 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-27T09:27:41.009 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-27T09:27:41.009 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-27T09:27:41.009 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-05-27T09:27:41.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T09:27:41.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:27:41.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:27:41.732 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2024-05-27T09:27:41.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:27:41.785 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list-id 2024-05-27T09:27:41.837 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T09:27:41.837 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-05-27T09:27:41.837 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T09:27:41.837 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-27T09:27:41.838 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-05-27T09:27:41.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T09:27:42.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:27:42.026 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-27T09:27:42.027 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T09:27:42.027 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-05-27T09:27:42.027 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-27T09:27:42.027 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-27T09:27:42.027 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-27T09:27:42.027 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-27T09:27:42.027 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-27T09:27:42.027 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-05-27T09:27:42.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T09:27:42.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:27:42.323 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T09:27:42.323 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-05-27T09:27:42.323 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set brx.0 up 2024-05-27T09:27:42.323 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set dev brx.0 master ceph-brx 2024-05-27T09:27:42.324 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-05-27T09:27:42.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T09:27:42.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:27:42.518 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T09:27:42.518 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-27T09:27:42.560 INFO:teuthology.orchestra.run.smithi114.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-27T09:27:42.561 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T09:27:42.561 DEBUG:teuthology.orchestra.run.smithi114:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-27T09:27:42.654 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-27T09:27:42.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:27:42.655 DEBUG:teuthology.orchestra.run.smithi114:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-27T09:27:42.870 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-27T09:27:42.870 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-27T09:27:42.870 INFO:teuthology.orchestra.run.smithi114.stdout:invalid new device string format 2024-05-27T09:27:43.529 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-27T09:27:43.529 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-27T09:27:43.529 INFO:teuthology.orchestra.run.smithi114.stdout:invalid new device string format 2024-05-27T09:27:43.529 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: resolved to: "172.21.15.114:3300,172.21.15.163:3300,172.21.15.163:3301" 2024-05-27T09:27:43.529 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-27T09:27:43.575 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-27T09:27:43.575 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-27T09:27:43.575 INFO:teuthology.orchestra.run.smithi114.stdout:invalid new device string format 2024-05-27T09:27:43.576 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: resolved to: "172.21.15.114:3300,172.21.15.163:3300,172.21.15.163:3301" 2024-05-27T09:27:43.576 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-27T09:27:43.576 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: resolved to: "172.21.15.114:6789,172.21.15.163:6789,172.21.15.163:6790" 2024-05-27T09:27:43.576 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.114:6789,172.21.15.163:6789,172.21.15.163:6790:/ 2024-05-27T09:27:43.576 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48" will pass to kernel 2024-05-27T09:27:43.578 INFO:teuthology.orchestra.run.smithi114.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-27T09:27:43.579 INFO:teuthology.orchestra.run.smithi114.stdout: You just mounted a file system that supports labels which does not 2024-05-27T09:27:43.579 INFO:teuthology.orchestra.run.smithi114.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-27T09:27:43.579 INFO:teuthology.orchestra.run.smithi114.stdout: applications will generate AVC messages and not be allowed access to 2024-05-27T09:27:43.579 INFO:teuthology.orchestra.run.smithi114.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-27T09:27:43.583 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-27T09:27:43.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:27:43.583 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-27T09:27:43.673 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T09:27:43.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T09:27:43.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:27:43.803 INFO:teuthology.orchestra.run.smithi114.stdout:client4550 2024-05-27T09:27:43.804 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:27:43.804 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T09:27:43.857 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T09:27:43.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T09:27:43.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:27:43.987 INFO:teuthology.orchestra.run.smithi114.stdout:client4550 2024-05-27T09:27:43.988 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T09:27:44.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T09:27:44.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:27:44.125 INFO:teuthology.orchestra.run.smithi114.stdout:client4550 2024-05-27T09:27:44.125 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48.client4550/status') 2024-05-27T09:27:44.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T09:27:44.264 INFO:teuthology.orchestra.run.smithi114.stdout:instance: client.4550 (3)172.21.15.114:0/3024664810 2024-05-27T09:27:44.265 INFO:teuthology.orchestra.run.smithi114.stdout:blocklisted: false 2024-05-27T09:27:44.265 INFO:teuthology.orchestra.run.smithi114.stderr:0+1 records in 2024-05-27T09:27:44.265 INFO:teuthology.orchestra.run.smithi114.stderr:0+1 records out 2024-05-27T09:27:44.265 INFO:teuthology.orchestra.run.smithi114.stderr:71 bytes copied, 7.4706e-05 s, 950 kB/s 2024-05-27T09:27:44.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:27:44.279 INFO:teuthology.run_tasks:Running task mds_thrash... 2024-05-27T09:27:44.306 INFO:tasks.mds_thrash:mds thrasher using random seed: 1716802064 2024-05-27T09:27:44.306 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T09:27:44.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.583+0000 7f42c3e00640 1 Processor -- start 2024-05-27T09:27:44.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.583+0000 7f42c3e00640 1 -- start start 2024-05-27T09:27:44.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.585+0000 7f42c3e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f42c40c2a00 0x7f42c4159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:44.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.585+0000 7f42c3e00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42c40c96d0 con 0x7f42c4159e40 2024-05-27T09:27:44.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.585+0000 7f42c3e00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42c40c8930 con 0x7f42c41534c0 2024-05-27T09:27:44.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.585+0000 7f42c3e00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f42c4159190 con 0x7f42c40c2a00 2024-05-27T09:27:44.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.586+0000 7f42bbe00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f42c40c2a00 0x7f42c4159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:44.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.586+0000 7f42c2a00640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f42c41534c0 0x7f42c40c8220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:59152/0 (socket says 172.21.15.114:59152) 2024-05-27T09:27:44.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.586+0000 7f42c2a00640 1 -- 172.21.15.114:0/964572872 learned_addr learned my addr 172.21.15.114:0/964572872 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:44.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.586+0000 7f42bbe00640 1 -- 172.21.15.114:0/964572872 >> v1:172.21.15.163:6789/0 conn(0x7f42c41534c0 legacy=0x7f42c40c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:44.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.586+0000 7f42c2000640 1 -- 172.21.15.114:0/964572872 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 188926022 0 0) 0x7f42c40c8930 con 0x7f42c41534c0 2024-05-27T09:27:44.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.586+0000 7f42bbe00640 1 -- 172.21.15.114:0/964572872 >> v1:172.21.15.114:6789/0 conn(0x7f42c4159e40 legacy=0x7f42c4074140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:44.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.586+0000 7f42bbe00640 1 -- 172.21.15.114:0/964572872 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c40c9970 con 0x7f42c40c2a00 2024-05-27T09:27:44.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.587+0000 7f42bbe00640 1 --2- 172.21.15.114:0/964572872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42c40c2a00 0x7f42c4159900 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f42ac0097a0 tx=0x7f42ac02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7614dd9ecf7583d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:44.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.588+0000 7f42c2000640 1 -- 172.21.15.114:0/964572872 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac037070 con 0x7f42c40c2a00 2024-05-27T09:27:44.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.588+0000 7f42c2000640 1 -- 172.21.15.114:0/964572872 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42ac02adc0 con 0x7f42c40c2a00 2024-05-27T09:27:44.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.588+0000 7f42c2000640 1 -- 172.21.15.114:0/964572872 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac032660 con 0x7f42c40c2a00 2024-05-27T09:27:44.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.588+0000 7f42c3e00640 1 -- 172.21.15.114:0/964572872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42c40c2a00 msgr2=0x7f42c4159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:44.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.588+0000 7f42c3e00640 1 --2- 172.21.15.114:0/964572872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42c40c2a00 0x7f42c4159900 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f42ac0097a0 tx=0x7f42ac02a130 comp rx=0 tx=0).stop 2024-05-27T09:27:44.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.589+0000 7f42c3e00640 1 -- 172.21.15.114:0/964572872 shutdown_connections 2024-05-27T09:27:44.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.589+0000 7f42c3e00640 1 --2- 172.21.15.114:0/964572872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42c40c2a00 0x7f42c4159900 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:44.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.589+0000 7f42c3e00640 1 -- 172.21.15.114:0/964572872 >> 172.21.15.114:0/964572872 conn(0x7f42c40ca320 msgr2=0x7f42c40ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:44.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.589+0000 7f42c3e00640 1 -- 172.21.15.114:0/964572872 shutdown_connections 2024-05-27T09:27:44.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.589+0000 7f42c3e00640 1 -- 172.21.15.114:0/964572872 wait complete. 2024-05-27T09:27:44.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.591+0000 7f42c3e00640 1 Processor -- start 2024-05-27T09:27:44.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.591+0000 7f42c3e00640 1 -- start start 2024-05-27T09:27:44.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.593+0000 7f42c3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42c40c2a00 0x7f42c4156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:44.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.593+0000 7f42c3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42c41534c0 0x7f42c40cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:44.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.593+0000 7f42c3e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42c4159e40 0x7f42c40cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:44.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.593+0000 7f42c3e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f42c40c9f30 con 0x7f42c40c2a00 2024-05-27T09:27:44.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.593+0000 7f42c3e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f42c41538d0 con 0x7f42c41534c0 2024-05-27T09:27:44.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.593+0000 7f42c3e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f42c41593e0 con 0x7f42c4159e40 2024-05-27T09:27:44.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.593+0000 7f42c2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42c40c2a00 0x7f42c4156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:44.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.593+0000 7f42c2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42c40c2a00 0x7f42c4156760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47958/0 (socket says 172.21.15.114:47958) 2024-05-27T09:27:44.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.593+0000 7f42c2a00640 1 -- 172.21.15.114:0/2613354241 learned_addr learned my addr 172.21.15.114:0/2613354241 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:44.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.593+0000 7f42bbe00640 1 --2- 172.21.15.114:0/2613354241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42c41534c0 0x7f42c40cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:44.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.593+0000 7f42c3400640 1 --2- 172.21.15.114:0/2613354241 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42c4159e40 0x7f42c40cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:44.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.593+0000 7f42c2a00640 1 -- 172.21.15.114:0/2613354241 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42c4159e40 msgr2=0x7f42c40cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:44.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.593+0000 7f42c2a00640 1 --2- 172.21.15.114:0/2613354241 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42c4159e40 0x7f42c40cc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:44.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.594+0000 7f42c2a00640 1 -- 172.21.15.114:0/2613354241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42c41534c0 msgr2=0x7f42c40cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:44.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.594+0000 7f42c2a00640 1 --2- 172.21.15.114:0/2613354241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42c41534c0 0x7f42c40cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:44.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.594+0000 7f42c2a00640 1 -- 172.21.15.114:0/2613354241 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c4156040 con 0x7f42c40c2a00 2024-05-27T09:27:44.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.594+0000 7f42c2a00640 1 --2- 172.21.15.114:0/2613354241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42c40c2a00 0x7f42c4156760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f42b400c3a0 tx=0x7f42b400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:44.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.594+0000 7f42bb400640 1 -- 172.21.15.114:0/2613354241 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b40074e0 con 0x7f42c40c2a00 2024-05-27T09:27:44.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.594+0000 7f42bb400640 1 -- 172.21.15.114:0/2613354241 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42b40047e0 con 0x7f42c40c2a00 2024-05-27T09:27:44.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.594+0000 7f42bb400640 1 -- 172.21.15.114:0/2613354241 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4004c50 con 0x7f42c40c2a00 2024-05-27T09:27:44.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.594+0000 7f42c3e00640 1 -- 172.21.15.114:0/2613354241 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c40cd5b0 con 0x7f42c40c2a00 2024-05-27T09:27:44.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.594+0000 7f42c3e00640 1 -- 172.21.15.114:0/2613354241 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c4154a10 con 0x7f42c40c2a00 2024-05-27T09:27:44.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.595+0000 7f42c3e00640 1 -- 172.21.15.114:0/2613354241 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42c4152d70 con 0x7f42c40c2a00 2024-05-27T09:27:44.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.597+0000 7f42bb400640 1 -- 172.21.15.114:0/2613354241 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f42b4011070 con 0x7f42c40c2a00 2024-05-27T09:27:44.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.598+0000 7f42bb400640 1 --2- 172.21.15.114:0/2613354241 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f42a0076300 0x7f42a0078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:44.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.598+0000 7f42bb400640 1 -- 172.21.15.114:0/2613354241 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f42b40970e0 con 0x7f42c40c2a00 2024-05-27T09:27:44.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.598+0000 7f42bbe00640 1 --2- 172.21.15.114:0/2613354241 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f42a0076300 0x7f42a0078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:44.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.599+0000 7f42bbe00640 1 --2- 172.21.15.114:0/2613354241 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f42a0076300 0x7f42a0078810 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f42ac034000 tx=0x7f42ac02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:44.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.603+0000 7f42bb400640 1 -- 172.21.15.114:0/2613354241 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f42c4152d70 con 0x7f42c40c2a00 2024-05-27T09:27:44.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.811+0000 7f42c3e00640 1 -- 172.21.15.114:0/2613354241 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f42c40c7f40 con 0x7f42c40c2a00 2024-05-27T09:27:44.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.814+0000 7f42bb400640 1 -- 172.21.15.114:0/2613354241 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16645 (secure 0 0 0) 0x7f42c40c7f40 con 0x7f42c40c2a00 2024-05-27T09:27:44.816 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:44.816 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"fsid":"2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48","created":"2024-05-27T09:27:00.307278+0000","modified":"2024-05-27T09:27:19.530054+0000","last_up_change":"2024-05-27T09:27:11.463226+0000","last_in_change":"2024-05-27T09:27:02.972880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T09:27:10.796859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T09:27:16.063199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T09:27:17.064857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3561c750-1d63-42f6-a6ae-0ee63721e15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6817","nonce":1960550710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6819","nonce":1960550710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6824","nonce":1960550710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6821","nonce":1960550710}]},"public_addr":"172.21.15.114:6817/1960550710","cluster_addr":"172.21.15.114:6819/1960550710","heartbeat_back_addr":"172.21.15.114:6824/1960550710","heartbeat_front_addr":"172.21.15.114:6821/1960550710","state":["exists","up"]},{"osd":1,"uuid":"9a407d04-38c0-4676-928f-d04e8116bd19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6825","nonce":326580057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6827","nonce":326580057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6831","nonce":326580057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6829","nonce":326580057}]},"public_addr":"172.21.15.114:6825/326580057","cluster_addr":"172.21.15.114:6827/326580057","heartbeat_back_addr":"172.21.15.114:6831/326580057","heartbeat_front_addr":"172.21.15.114:6829/326580057","state":["exists","up"]},{"osd":2,"uuid":"255a1cc6-9357-43f7-9db5-d760f4806ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6809","nonce":865568619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6811","nonce":865568619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6815","nonce":865568619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6813","nonce":865568619}]},"public_addr":"172.21.15.114:6809/865568619","cluster_addr":"172.21.15.114:6811/865568619","heartbeat_back_addr":"172.21.15.114:6815/865568619","heartbeat_front_addr":"172.21.15.114:6813/865568619","state":["exists","up"]},{"osd":3,"uuid":"cb4d7692-9a17-41b9-927e-d8eac3e217c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6801","nonce":1601792164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6803","nonce":1601792164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6807","nonce":1601792164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6805","nonce":1601792164}]},"public_addr":"172.21.15.114:6801/1601792164","cluster_addr":"172.21.15.114:6803/1601792164","heartbeat_back_addr":"172.21.15.114:6807/1601792164","heartbeat_front_addr":"172.21.15.114:6805/1601792164","state":["exists","up"]},{"osd":4,"uuid":"cdef1940-3453-4fbd-b0f1-459f5f5c7142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6817","nonce":1982246015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6819","nonce":1982246015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6823","nonce":1982246015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6821","nonce":1982246015}]},"public_addr":"172.21.15.163:6817/1982246015","cluster_addr":"172.21.15.163:6819/1982246015","heartbeat_back_addr":"172.21.15.163:6823/1982246015","heartbeat_front_addr":"172.21.15.163:6821/1982246015","state":["exists","up"]},{"osd":5,"uuid":"220a7b32-abcc-492f-bca8-4615cc2ea111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6801","nonce":1695294004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6803","nonce":1695294004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6807","nonce":1695294004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6805","nonce":1695294004}]},"public_addr":"172.21.15.163:6801/1695294004","cluster_addr":"172.21.15.163:6803/1695294004","heartbeat_back_addr":"172.21.15.163:6807/1695294004","heartbeat_front_addr":"172.21.15.163:6805/1695294004","state":["exists","up"]},{"osd":6,"uuid":"43a74d13-d03f-493a-a6c0-e73b0a056c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6809","nonce":3649718505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6811","nonce":3649718505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6815","nonce":3649718505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6813","nonce":3649718505}]},"public_addr":"172.21.15.163:6809/3649718505","cluster_addr":"172.21.15.163:6811/3649718505","heartbeat_back_addr":"172.21.15.163:6815/3649718505","heartbeat_front_addr":"172.21.15.163:6813/3649718505","state":["exists","up"]},{"osd":7,"uuid":"134629c2-70af-4b84-83a6-7ddf53681baa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6825","nonce":2539451311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6827","nonce":2539451311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6831","nonce":2539451311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6829","nonce":2539451311}]},"public_addr":"172.21.15.163:6825/2539451311","cluster_addr":"172.21.15.163:6827/2539451311","heartbeat_back_addr":"172.21.15.163:6831/2539451311","heartbeat_front_addr":"172.21.15.163:6829/2539451311","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.388942+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.515675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.646761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.400212+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.220072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.079531+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.180419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.175008+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T09:27:44.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.817+0000 7f42c3e00640 1 -- 172.21.15.114:0/2613354241 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f42a0076300 msgr2=0x7f42a0078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:44.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.817+0000 7f42c3e00640 1 --2- 172.21.15.114:0/2613354241 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f42a0076300 0x7f42a0078810 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f42ac034000 tx=0x7f42ac02e000 comp rx=0 tx=0).stop 2024-05-27T09:27:44.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.817+0000 7f42c3e00640 1 -- 172.21.15.114:0/2613354241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42c40c2a00 msgr2=0x7f42c4156760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:44.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.817+0000 7f42c3e00640 1 --2- 172.21.15.114:0/2613354241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42c40c2a00 0x7f42c4156760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f42b400c3a0 tx=0x7f42b400c860 comp rx=0 tx=0).stop 2024-05-27T09:27:44.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.817+0000 7f42c3e00640 1 -- 172.21.15.114:0/2613354241 shutdown_connections 2024-05-27T09:27:44.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.817+0000 7f42c3e00640 1 --2- 172.21.15.114:0/2613354241 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f42a0076300 0x7f42a0078810 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:44.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.817+0000 7f42c3e00640 1 --2- 172.21.15.114:0/2613354241 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42c4159e40 0x7f42c40cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:44.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.817+0000 7f42c3e00640 1 --2- 172.21.15.114:0/2613354241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42c41534c0 0x7f42c40cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:44.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.817+0000 7f42c3e00640 1 --2- 172.21.15.114:0/2613354241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42c40c2a00 0x7f42c4156760 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:44.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.817+0000 7f42c3e00640 1 -- 172.21.15.114:0/2613354241 >> 172.21.15.114:0/2613354241 conn(0x7f42c40ca320 msgr2=0x7f42c4073940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:44.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.817+0000 7f42c3e00640 1 -- 172.21.15.114:0/2613354241 shutdown_connections 2024-05-27T09:27:44.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:44.817+0000 7f42c3e00640 1 -- 172.21.15.114:0/2613354241 wait complete. 2024-05-27T09:27:44.846 INFO:tasks.mds_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T09:27:10.796859+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T09:27:16.063199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T09:27:17.064857+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T09:27:44.847 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T09:27:45.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.127+0000 7feab3400640 1 Processor -- start 2024-05-27T09:27:45.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.127+0000 7feab3400640 1 -- start start 2024-05-27T09:27:45.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.131+0000 7feab3400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7feaac152fe0 0x7feaac164620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:45.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.131+0000 7feab3400640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaac15c990 con 0x7feaac0c83e0 2024-05-27T09:27:45.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.131+0000 7feab3400640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaac074070 con 0x7feaac0c2a00 2024-05-27T09:27:45.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.131+0000 7feab3400640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7feaac1533f0 con 0x7feaac152fe0 2024-05-27T09:27:45.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.131+0000 7feab1400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7feaac152fe0 0x7feaac164620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:45.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.131+0000 7feab0a00640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7feaac0c2a00 0x7feaac15c280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:41192/0 (socket says 172.21.15.114:41192) 2024-05-27T09:27:45.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.131+0000 7feab0a00640 1 -- 172.21.15.114:0/1965725633 learned_addr learned my addr 172.21.15.114:0/1965725633 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:45.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.131+0000 7feaabe00640 1 -- 172.21.15.114:0/1965725633 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1456260963 0 0) 0x7feaac074070 con 0x7feaac0c2a00 2024-05-27T09:27:45.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.131+0000 7feaabe00640 1 -- 172.21.15.114:0/1965725633 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea90003520 con 0x7feaac0c2a00 2024-05-27T09:27:45.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.131+0000 7feab1400640 1 -- 172.21.15.114:0/1965725633 >> v1:172.21.15.163:6789/0 conn(0x7feaac0c2a00 legacy=0x7feaac15c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:45.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.131+0000 7feab1400640 1 -- 172.21.15.114:0/1965725633 >> v1:172.21.15.114:6789/0 conn(0x7feaac0c83e0 legacy=0x7feaac0ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:45.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.131+0000 7feaabe00640 1 -- 172.21.15.114:0/1965725633 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 338976274 0 0) 0x7feaac15c990 con 0x7feaac0c83e0 2024-05-27T09:27:45.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.131+0000 7feab1400640 1 -- 172.21.15.114:0/1965725633 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaac0c9330 con 0x7feaac152fe0 2024-05-27T09:27:45.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.132+0000 7feab1400640 1 --2- 172.21.15.114:0/1965725633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaac152fe0 0x7feaac164620 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009d90 tx=0x7feaa402b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63d6bc6fbeab43c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:45.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.132+0000 7feaabe00640 1 -- 172.21.15.114:0/1965725633 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa4031790 con 0x7feaac152fe0 2024-05-27T09:27:45.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.132+0000 7feaabe00640 1 -- 172.21.15.114:0/1965725633 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaa4037070 con 0x7feaac152fe0 2024-05-27T09:27:45.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.132+0000 7feaabe00640 1 -- 172.21.15.114:0/1965725633 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa4031b50 con 0x7feaac152fe0 2024-05-27T09:27:45.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.133+0000 7feab3400640 1 -- 172.21.15.114:0/1965725633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaac152fe0 msgr2=0x7feaac164620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:45.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.133+0000 7feab3400640 1 --2- 172.21.15.114:0/1965725633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaac152fe0 0x7feaac164620 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009d90 tx=0x7feaa402b890 comp rx=0 tx=0).stop 2024-05-27T09:27:45.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.133+0000 7feab3400640 1 -- 172.21.15.114:0/1965725633 shutdown_connections 2024-05-27T09:27:45.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.133+0000 7feab3400640 1 --2- 172.21.15.114:0/1965725633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaac152fe0 0x7feaac164620 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:45.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.133+0000 7feab3400640 1 -- 172.21.15.114:0/1965725633 >> 172.21.15.114:0/1965725633 conn(0x7feaac06f6e0 msgr2=0x7feaac070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:45.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.133+0000 7feab3400640 1 -- 172.21.15.114:0/1965725633 shutdown_connections 2024-05-27T09:27:45.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.133+0000 7feab3400640 1 -- 172.21.15.114:0/1965725633 wait complete. 2024-05-27T09:27:45.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.135+0000 7feab3400640 1 Processor -- start 2024-05-27T09:27:45.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.136+0000 7feab3400640 1 -- start start 2024-05-27T09:27:45.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.138+0000 7feab3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaac0c2a00 0x7feaac1f60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:45.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.138+0000 7feab3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaac0c83e0 0x7feaac1450b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:45.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.138+0000 7feab3400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feaac152fe0 0x7feaac148270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:45.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.138+0000 7feab3400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7feaac0c98f0 con 0x7feaac0c83e0 2024-05-27T09:27:45.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.138+0000 7feab3400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7feaac073e20 con 0x7feaac0c2a00 2024-05-27T09:27:45.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.138+0000 7feab3400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7feaac0c9bd0 con 0x7feaac152fe0 2024-05-27T09:27:45.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.138+0000 7feaa3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaac0c83e0 0x7feaac1450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:45.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.138+0000 7feaa3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaac0c83e0 0x7feaac1450b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37984/0 (socket says 172.21.15.114:37984) 2024-05-27T09:27:45.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.138+0000 7feaa3e00640 1 -- 172.21.15.114:0/1581339765 learned_addr learned my addr 172.21.15.114:0/1581339765 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:45.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.138+0000 7feab0a00640 1 --2- 172.21.15.114:0/1581339765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaac0c2a00 0x7feaac1f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:45.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.138+0000 7feab1400640 1 --2- 172.21.15.114:0/1581339765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feaac152fe0 0x7feaac148270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:45.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.138+0000 7feaa3e00640 1 -- 172.21.15.114:0/1581339765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feaac152fe0 msgr2=0x7feaac148270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:45.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.138+0000 7feaa3e00640 1 --2- 172.21.15.114:0/1581339765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feaac152fe0 0x7feaac148270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:45.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.138+0000 7feaa3e00640 1 -- 172.21.15.114:0/1581339765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaac0c2a00 msgr2=0x7feaac1f60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:45.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.138+0000 7feaa3e00640 1 --2- 172.21.15.114:0/1581339765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaac0c2a00 0x7feaac1f60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:45.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.138+0000 7feaa3e00640 1 -- 172.21.15.114:0/1581339765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaac1f65e0 con 0x7feaac0c83e0 2024-05-27T09:27:45.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.138+0000 7feaa3e00640 1 --2- 172.21.15.114:0/1581339765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaac0c83e0 0x7feaac1450b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fea9400c3f0 tx=0x7fea9400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:45.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.138+0000 7feaa9600640 1 -- 172.21.15.114:0/1581339765 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea94017070 con 0x7feaac0c83e0 2024-05-27T09:27:45.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.138+0000 7feaa9600640 1 -- 172.21.15.114:0/1581339765 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea940035f0 con 0x7feaac0c83e0 2024-05-27T09:27:45.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.138+0000 7feaa9600640 1 -- 172.21.15.114:0/1581339765 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea940039e0 con 0x7feaac0c83e0 2024-05-27T09:27:45.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.138+0000 7feab3400640 1 -- 172.21.15.114:0/1581339765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaac1e3d10 con 0x7feaac0c83e0 2024-05-27T09:27:45.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.139+0000 7feab3400640 1 -- 172.21.15.114:0/1581339765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaac1e42d0 con 0x7feaac0c83e0 2024-05-27T09:27:45.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.140+0000 7feab3400640 1 -- 172.21.15.114:0/1581339765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaac06a380 con 0x7feaac0c83e0 2024-05-27T09:27:45.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.141+0000 7feaa9600640 1 -- 172.21.15.114:0/1581339765 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fea9401a070 con 0x7feaac0c83e0 2024-05-27T09:27:45.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.142+0000 7feaa9600640 1 --2- 172.21.15.114:0/1581339765 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fea88076300 0x7fea88078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:45.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.142+0000 7feaa9600640 1 -- 172.21.15.114:0/1581339765 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fea940969e0 con 0x7feaac0c83e0 2024-05-27T09:27:45.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.142+0000 7feab0a00640 1 --2- 172.21.15.114:0/1581339765 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fea88076300 0x7fea88078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:45.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.143+0000 7feab0a00640 1 --2- 172.21.15.114:0/1581339765 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fea88076300 0x7fea88078810 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fea9c005880 tx=0x7fea9c006aa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:45.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.147+0000 7feaa9600640 1 -- 172.21.15.114:0/1581339765 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fea9405fbc0 con 0x7feaac0c83e0 2024-05-27T09:27:45.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.352+0000 7feab3400640 1 -- 172.21.15.114:0/1581339765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feaac0c8f00 con 0x7feaac0c83e0 2024-05-27T09:27:45.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.353+0000 7feaa9600640 1 -- 172.21.15.114:0/1581339765 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fea94063ec0 con 0x7feaac0c83e0 2024-05-27T09:27:45.354 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-05-27T09:27:45.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.356+0000 7feab3400640 1 -- 172.21.15.114:0/1581339765 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fea88076300 msgr2=0x7fea88078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:45.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.356+0000 7feab3400640 1 --2- 172.21.15.114:0/1581339765 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fea88076300 0x7fea88078810 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fea9c005880 tx=0x7fea9c006aa0 comp rx=0 tx=0).stop 2024-05-27T09:27:45.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.356+0000 7feab3400640 1 -- 172.21.15.114:0/1581339765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaac0c83e0 msgr2=0x7feaac1450b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:45.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.356+0000 7feab3400640 1 --2- 172.21.15.114:0/1581339765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaac0c83e0 0x7feaac1450b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fea9400c3f0 tx=0x7fea9400c8b0 comp rx=0 tx=0).stop 2024-05-27T09:27:45.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.356+0000 7feab3400640 1 -- 172.21.15.114:0/1581339765 shutdown_connections 2024-05-27T09:27:45.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.356+0000 7feab3400640 1 --2- 172.21.15.114:0/1581339765 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fea88076300 0x7fea88078810 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:45.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.356+0000 7feab3400640 1 --2- 172.21.15.114:0/1581339765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feaac152fe0 0x7feaac148270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:45.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.356+0000 7feab3400640 1 --2- 172.21.15.114:0/1581339765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaac0c83e0 0x7feaac1450b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:45.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.356+0000 7feab3400640 1 --2- 172.21.15.114:0/1581339765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaac0c2a00 0x7feaac1f60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:45.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.356+0000 7feab3400640 1 -- 172.21.15.114:0/1581339765 >> 172.21.15.114:0/1581339765 conn(0x7feaac06f6e0 msgr2=0x7feaac151210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:45.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.356+0000 7feab3400640 1 -- 172.21.15.114:0/1581339765 shutdown_connections 2024-05-27T09:27:45.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.356+0000 7feab3400640 1 -- 172.21.15.114:0/1581339765 wait complete. 2024-05-27T09:27:45.383 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T09:27:45.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.661+0000 7fa3a7200640 1 Processor -- start 2024-05-27T09:27:45.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.662+0000 7fa3a7200640 1 -- start start 2024-05-27T09:27:45.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.664+0000 7fa3a7200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3a00c53f0 0x7fa3a0071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:45.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.664+0000 7fa3a7200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa3a00c9d10 0x7fa3a0071f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:45.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.664+0000 7fa3a7200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3a00c8180 con 0x7fa3a0068990 2024-05-27T09:27:45.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.664+0000 7fa3a7200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fa3a006dc90 con 0x7fa3a00c9d10 2024-05-27T09:27:45.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.664+0000 7fa3a7200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa3a00ca6e0 con 0x7fa3a00c53f0 2024-05-27T09:27:45.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.664+0000 7fa3a5400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa3a00c9d10 0x7fa3a0071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:45.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.664+0000 7fa3a6800640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fa3a0068990 0x7fa3a01593d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:45382/0 (socket says 172.21.15.114:45382) 2024-05-27T09:27:45.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.664+0000 7fa3a6800640 1 -- 172.21.15.114:0/1635098219 learned_addr learned my addr 172.21.15.114:0/1635098219 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:45.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.664+0000 7fa3a5e00640 1 --2- 172.21.15.114:0/1635098219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3a00c53f0 0x7fa3a0071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:45.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.665+0000 7fa3a5400640 1 -- 172.21.15.114:0/1635098219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3a00c53f0 msgr2=0x7fa3a0071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:45.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.665+0000 7fa3a5400640 1 --2- 172.21.15.114:0/1635098219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3a00c53f0 0x7fa3a0071a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:45.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.665+0000 7fa3a5400640 1 -- 172.21.15.114:0/1635098219 >> v1:172.21.15.114:6789/0 conn(0x7fa3a0068990 legacy=0x7fa3a01593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:45.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.665+0000 7fa3a5400640 1 -- 172.21.15.114:0/1635098219 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a006d9b0 con 0x7fa3a00c9d10 2024-05-27T09:27:45.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.666+0000 7fa3a5400640 1 --2- 172.21.15.114:0/1635098219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3a00c9d10 0x7fa3a0071f90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa39000ba40 tx=0x7fa39002c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de718695e4b31e05 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:45.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.666+0000 7fa3a4a00640 1 -- 172.21.15.114:0/1635098219 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa390032750 con 0x7fa3a00c9d10 2024-05-27T09:27:45.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.666+0000 7fa3a4a00640 1 -- 172.21.15.114:0/1635098219 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa390030050 con 0x7fa3a00c9d10 2024-05-27T09:27:45.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.666+0000 7fa3a4a00640 1 -- 172.21.15.114:0/1635098219 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa390032b10 con 0x7fa3a00c9d10 2024-05-27T09:27:45.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.667+0000 7fa3a7200640 1 -- 172.21.15.114:0/1635098219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3a00c9d10 msgr2=0x7fa3a0071f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:45.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.667+0000 7fa3a7200640 1 --2- 172.21.15.114:0/1635098219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3a00c9d10 0x7fa3a0071f90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa39000ba40 tx=0x7fa39002c850 comp rx=0 tx=0).stop 2024-05-27T09:27:45.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.667+0000 7fa3a7200640 1 -- 172.21.15.114:0/1635098219 shutdown_connections 2024-05-27T09:27:45.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.667+0000 7fa3a7200640 1 --2- 172.21.15.114:0/1635098219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3a00c9d10 0x7fa3a0071f90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:45.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.667+0000 7fa3a7200640 1 --2- 172.21.15.114:0/1635098219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3a00c53f0 0x7fa3a0071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:45.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.667+0000 7fa3a7200640 1 -- 172.21.15.114:0/1635098219 >> 172.21.15.114:0/1635098219 conn(0x7fa3a0070f30 msgr2=0x7fa3a0071340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:45.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.667+0000 7fa3a7200640 1 -- 172.21.15.114:0/1635098219 shutdown_connections 2024-05-27T09:27:45.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.667+0000 7fa3a7200640 1 -- 172.21.15.114:0/1635098219 wait complete. 2024-05-27T09:27:45.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.669+0000 7fa3a7200640 1 Processor -- start 2024-05-27T09:27:45.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.669+0000 7fa3a7200640 1 -- start start 2024-05-27T09:27:45.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.671+0000 7fa3a7200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3a0068990 0x7fa3a01f3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:45.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.671+0000 7fa3a7200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3a00c53f0 0x7fa3a015b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:45.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.671+0000 7fa3a7200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3a00c9d10 0x7fa3a0157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:45.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.671+0000 7fa3a7200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01536b0 con 0x7fa3a0068990 2024-05-27T09:27:45.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.671+0000 7fa3a7200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a00c6a10 con 0x7fa3a00c9d10 2024-05-27T09:27:45.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.671+0000 7fa3a7200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa3a0153830 con 0x7fa3a00c53f0 2024-05-27T09:27:45.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.671+0000 7fa3a5e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3a0068990 0x7fa3a01f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:45.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.671+0000 7fa3a5e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3a0068990 0x7fa3a01f3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37986/0 (socket says 172.21.15.114:37986) 2024-05-27T09:27:45.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.671+0000 7fa3a5e00640 1 -- 172.21.15.114:0/164946846 learned_addr learned my addr 172.21.15.114:0/164946846 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:45.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.671+0000 7fa3a5400640 1 --2- 172.21.15.114:0/164946846 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3a00c53f0 0x7fa3a015b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:45.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.671+0000 7fa3a6800640 1 --2- 172.21.15.114:0/164946846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3a00c9d10 0x7fa3a0157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:45.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.672+0000 7fa3a5e00640 1 -- 172.21.15.114:0/164946846 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3a00c53f0 msgr2=0x7fa3a015b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:45.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.672+0000 7fa3a5e00640 1 --2- 172.21.15.114:0/164946846 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3a00c53f0 0x7fa3a015b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:45.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.672+0000 7fa3a5e00640 1 -- 172.21.15.114:0/164946846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3a00c9d10 msgr2=0x7fa3a0157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:45.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.672+0000 7fa3a5e00640 1 --2- 172.21.15.114:0/164946846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3a00c9d10 0x7fa3a0157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:45.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.672+0000 7fa3a5e00640 1 -- 172.21.15.114:0/164946846 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a0150c30 con 0x7fa3a0068990 2024-05-27T09:27:45.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.672+0000 7fa3a5e00640 1 --2- 172.21.15.114:0/164946846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3a0068990 0x7fa3a01f3340 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa398009d10 tx=0x7fa39800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:45.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.672+0000 7fa396000640 1 -- 172.21.15.114:0/164946846 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398018020 con 0x7fa3a0068990 2024-05-27T09:27:45.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.672+0000 7fa396000640 1 -- 172.21.15.114:0/164946846 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa398014480 con 0x7fa3a0068990 2024-05-27T09:27:45.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.672+0000 7fa3a7200640 1 -- 172.21.15.114:0/164946846 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a01f60d0 con 0x7fa3a0068990 2024-05-27T09:27:45.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.672+0000 7fa396000640 1 -- 172.21.15.114:0/164946846 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398014830 con 0x7fa3a0068990 2024-05-27T09:27:45.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.672+0000 7fa3a7200640 1 -- 172.21.15.114:0/164946846 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a01f6690 con 0x7fa3a0068990 2024-05-27T09:27:45.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.674+0000 7fa3a7200640 1 -- 172.21.15.114:0/164946846 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3a01f5ec0 con 0x7fa3a0068990 2024-05-27T09:27:45.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.675+0000 7fa396000640 1 -- 172.21.15.114:0/164946846 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa39801b020 con 0x7fa3a0068990 2024-05-27T09:27:45.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.676+0000 7fa396000640 1 --2- 172.21.15.114:0/164946846 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa3840762d0 0x7fa3840787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:45.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.676+0000 7fa396000640 1 -- 172.21.15.114:0/164946846 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa398098450 con 0x7fa3a0068990 2024-05-27T09:27:45.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.676+0000 7fa3a5400640 1 --2- 172.21.15.114:0/164946846 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa3840762d0 0x7fa3840787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:45.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.677+0000 7fa3a5400640 1 --2- 172.21.15.114:0/164946846 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa3840762d0 0x7fa3840787e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa390037000 tx=0x7fa390008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:45.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.681+0000 7fa396000640 1 -- 172.21.15.114:0/164946846 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa398022020 con 0x7fa3a0068990 2024-05-27T09:27:45.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.888+0000 7fa3a7200640 1 -- 172.21.15.114:0/164946846 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa3a0050860 con 0x7fa3a0068990 2024-05-27T09:27:45.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.889+0000 7fa396000640 1 -- 172.21.15.114:0/164946846 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fa398010040 con 0x7fa3a0068990 2024-05-27T09:27:45.890 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-05-27T09:27:45.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.892+0000 7fa3a7200640 1 -- 172.21.15.114:0/164946846 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa3840762d0 msgr2=0x7fa3840787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:45.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.892+0000 7fa3a7200640 1 --2- 172.21.15.114:0/164946846 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa3840762d0 0x7fa3840787e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa390037000 tx=0x7fa390008040 comp rx=0 tx=0).stop 2024-05-27T09:27:45.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.892+0000 7fa3a7200640 1 -- 172.21.15.114:0/164946846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3a0068990 msgr2=0x7fa3a01f3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:45.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.892+0000 7fa3a7200640 1 --2- 172.21.15.114:0/164946846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3a0068990 0x7fa3a01f3340 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa398009d10 tx=0x7fa39800c490 comp rx=0 tx=0).stop 2024-05-27T09:27:45.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.892+0000 7fa3a7200640 1 -- 172.21.15.114:0/164946846 shutdown_connections 2024-05-27T09:27:45.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.892+0000 7fa3a7200640 1 --2- 172.21.15.114:0/164946846 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa3840762d0 0x7fa3840787e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:45.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.892+0000 7fa3a7200640 1 --2- 172.21.15.114:0/164946846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3a00c9d10 0x7fa3a0157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:45.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.892+0000 7fa3a7200640 1 --2- 172.21.15.114:0/164946846 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3a00c53f0 0x7fa3a015b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:45.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.892+0000 7fa3a7200640 1 --2- 172.21.15.114:0/164946846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3a0068990 0x7fa3a01f3340 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:45.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.892+0000 7fa3a7200640 1 -- 172.21.15.114:0/164946846 >> 172.21.15.114:0/164946846 conn(0x7fa3a0070f30 msgr2=0x7fa3a0074250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:45.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.892+0000 7fa3a7200640 1 -- 172.21.15.114:0/164946846 shutdown_connections 2024-05-27T09:27:45.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:45.892+0000 7fa3a7200640 1 -- 172.21.15.114:0/164946846 wait complete. 2024-05-27T09:27:45.917 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T09:27:46.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.174+0000 7f357e600640 1 Processor -- start 2024-05-27T09:27:46.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.175+0000 7f357e600640 1 -- start start 2024-05-27T09:27:46.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.177+0000 7f357e600640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f357815c4b0 0x7f35780c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:46.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.177+0000 7f357e600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35781568f0 0x7f3578166e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:46.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.177+0000 7f357e600640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35780c8960 con 0x7f35780ce440 2024-05-27T09:27:46.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.177+0000 7f357e600640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f35780c2c30 con 0x7f357815c4b0 2024-05-27T09:27:46.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.177+0000 7f357e600640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f35780c2950 con 0x7f35781568f0 2024-05-27T09:27:46.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.177+0000 7f357dc00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35781568f0 0x7f3578166e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:46.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.177+0000 7f357d200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f357815c4b0 0x7f35780c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:46.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.177+0000 7f3577e00640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f35780ce440 0x7f3578070940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:41198/0 (socket says 172.21.15.114:41198) 2024-05-27T09:27:46.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.177+0000 7f3577e00640 1 -- 172.21.15.114:0/1641606231 learned_addr learned my addr 172.21.15.114:0/1641606231 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:46.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.178+0000 7f357dc00640 1 -- 172.21.15.114:0/1641606231 >> v2:172.21.15.163:3300/0 conn(0x7f357815c4b0 msgr2=0x7f35780c9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:46.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.178+0000 7f357dc00640 1 --2- 172.21.15.114:0/1641606231 >> v2:172.21.15.163:3300/0 conn(0x7f357815c4b0 0x7f35780c9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:46.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.178+0000 7f357dc00640 1 -- 172.21.15.114:0/1641606231 >> v1:172.21.15.163:6789/0 conn(0x7f35780ce440 legacy=0x7f3578070940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:46.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.178+0000 7f357dc00640 1 -- 172.21.15.114:0/1641606231 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35780c83a0 con 0x7f35781568f0 2024-05-27T09:27:46.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.179+0000 7f357dc00640 1 --2- 172.21.15.114:0/1641606231 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35781568f0 0x7f3578166e30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3570009d90 tx=0x7f357002a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a36056ba063cf48a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:46.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.179+0000 7f3577400640 1 -- 172.21.15.114:0/1641606231 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570007740 con 0x7f35781568f0 2024-05-27T09:27:46.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.179+0000 7f3577400640 1 -- 172.21.15.114:0/1641606231 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35700078e0 con 0x7f35781568f0 2024-05-27T09:27:46.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.179+0000 7f3577400640 1 -- 172.21.15.114:0/1641606231 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570007cb0 con 0x7f35781568f0 2024-05-27T09:27:46.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.180+0000 7f357e600640 1 -- 172.21.15.114:0/1641606231 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35781568f0 msgr2=0x7f3578166e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:46.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.180+0000 7f357e600640 1 --2- 172.21.15.114:0/1641606231 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35781568f0 0x7f3578166e30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3570009d90 tx=0x7f357002a6f0 comp rx=0 tx=0).stop 2024-05-27T09:27:46.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.180+0000 7f357e600640 1 -- 172.21.15.114:0/1641606231 shutdown_connections 2024-05-27T09:27:46.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.180+0000 7f357e600640 1 --2- 172.21.15.114:0/1641606231 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35781568f0 0x7f3578166e30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:46.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.180+0000 7f357e600640 1 --2- 172.21.15.114:0/1641606231 >> v2:172.21.15.163:3300/0 conn(0x7f357815c4b0 0x7f35780c9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:46.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.180+0000 7f357e600640 1 -- 172.21.15.114:0/1641606231 >> 172.21.15.114:0/1641606231 conn(0x7f35780cd890 msgr2=0x7f35780cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:46.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.180+0000 7f357e600640 1 -- 172.21.15.114:0/1641606231 shutdown_connections 2024-05-27T09:27:46.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.180+0000 7f357e600640 1 -- 172.21.15.114:0/1641606231 wait complete. 2024-05-27T09:27:46.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.183+0000 7f357e600640 1 Processor -- start 2024-05-27T09:27:46.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.183+0000 7f357e600640 1 -- start start 2024-05-27T09:27:46.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.185+0000 7f357e600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35780ce440 0x7f35780cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:46.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.185+0000 7f357e600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35781568f0 0x7f3578155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:46.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.185+0000 7f357e600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f357815c4b0 0x7f35781e8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:46.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.185+0000 7f357e600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f35780c9ff0 con 0x7f35781568f0 2024-05-27T09:27:46.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.185+0000 7f357e600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f35780c9590 con 0x7f357815c4b0 2024-05-27T09:27:46.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.185+0000 7f357e600640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f35780745a0 con 0x7f35780ce440 2024-05-27T09:27:46.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.186+0000 7f357d200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35780ce440 0x7f35780cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:46.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.186+0000 7f3577e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35781568f0 0x7f3578155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:46.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.186+0000 7f357dc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f357815c4b0 0x7f35781e8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:46.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.186+0000 7f357d200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35780ce440 0x7f35780cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:40650/0 (socket says 172.21.15.114:40650) 2024-05-27T09:27:46.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.186+0000 7f357d200640 1 -- 172.21.15.114:0/674530057 learned_addr learned my addr 172.21.15.114:0/674530057 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:46.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.186+0000 7f357d200640 1 -- 172.21.15.114:0/674530057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f357815c4b0 msgr2=0x7f35781e8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:46.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.186+0000 7f357d200640 1 --2- 172.21.15.114:0/674530057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f357815c4b0 0x7f35781e8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:46.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.186+0000 7f357d200640 1 -- 172.21.15.114:0/674530057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35781568f0 msgr2=0x7f3578155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:46.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.186+0000 7f357d200640 1 --2- 172.21.15.114:0/674530057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35781568f0 0x7f3578155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:46.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.186+0000 7f357d200640 1 -- 172.21.15.114:0/674530057 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357814e6c0 con 0x7f35780ce440 2024-05-27T09:27:46.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.186+0000 7f357d200640 1 --2- 172.21.15.114:0/674530057 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35780ce440 0x7f35780cc530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f356800e870 tx=0x7f356800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:46.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.187+0000 7f3574c00640 1 -- 172.21.15.114:0/674530057 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3568017070 con 0x7f35780ce440 2024-05-27T09:27:46.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.187+0000 7f3574c00640 1 -- 172.21.15.114:0/674530057 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f356800c940 con 0x7f35780ce440 2024-05-27T09:27:46.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.187+0000 7f3574c00640 1 -- 172.21.15.114:0/674530057 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356800cd30 con 0x7f35780ce440 2024-05-27T09:27:46.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.187+0000 7f357e600640 1 -- 172.21.15.114:0/674530057 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35781517c0 con 0x7f35780ce440 2024-05-27T09:27:46.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.187+0000 7f357e600640 1 -- 172.21.15.114:0/674530057 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f357814e980 con 0x7f35780ce440 2024-05-27T09:27:46.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.188+0000 7f357e600640 1 -- 172.21.15.114:0/674530057 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3578050860 con 0x7f35780ce440 2024-05-27T09:27:46.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.189+0000 7f3574c00640 1 -- 172.21.15.114:0/674530057 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f356801a070 con 0x7f35780ce440 2024-05-27T09:27:46.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.190+0000 7f3574c00640 1 --2- 172.21.15.114:0/674530057 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f35580762d0 0x7f35580787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:46.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.190+0000 7f3574c00640 1 -- 172.21.15.114:0/674530057 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f35680973c0 con 0x7f35780ce440 2024-05-27T09:27:46.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.191+0000 7f3577e00640 1 --2- 172.21.15.114:0/674530057 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f35580762d0 0x7f35580787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:46.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.192+0000 7f3577e00640 1 --2- 172.21.15.114:0/674530057 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f35580762d0 0x7f35580787e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3564000f60 tx=0x7f3564009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:46.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.196+0000 7f3574c00640 1 -- 172.21.15.114:0/674530057 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f35680606c0 con 0x7f35780ce440 2024-05-27T09:27:46.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.399+0000 7f357e600640 1 -- 172.21.15.114:0/674530057 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3578074920 con 0x7f35780ce440 2024-05-27T09:27:46.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.400+0000 7f3574c00640 1 -- 172.21.15.114:0/674530057 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f35680649c0 con 0x7f35780ce440 2024-05-27T09:27:46.401 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2024-05-27T09:27:46.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.403+0000 7f357e600640 1 -- 172.21.15.114:0/674530057 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f35580762d0 msgr2=0x7f35580787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:46.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.403+0000 7f357e600640 1 --2- 172.21.15.114:0/674530057 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f35580762d0 0x7f35580787e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3564000f60 tx=0x7f3564009040 comp rx=0 tx=0).stop 2024-05-27T09:27:46.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.403+0000 7f357e600640 1 -- 172.21.15.114:0/674530057 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35780ce440 msgr2=0x7f35780cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:46.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.403+0000 7f357e600640 1 --2- 172.21.15.114:0/674530057 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35780ce440 0x7f35780cc530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f356800e870 tx=0x7f356800ed30 comp rx=0 tx=0).stop 2024-05-27T09:27:46.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.404+0000 7f357e600640 1 -- 172.21.15.114:0/674530057 shutdown_connections 2024-05-27T09:27:46.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.404+0000 7f357e600640 1 --2- 172.21.15.114:0/674530057 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f35580762d0 0x7f35580787e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:46.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.404+0000 7f357e600640 1 --2- 172.21.15.114:0/674530057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f357815c4b0 0x7f35781e8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:46.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.404+0000 7f357e600640 1 --2- 172.21.15.114:0/674530057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35781568f0 0x7f3578155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:46.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.404+0000 7f357e600640 1 --2- 172.21.15.114:0/674530057 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35780ce440 0x7f35780cc530 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:46.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.404+0000 7f357e600640 1 -- 172.21.15.114:0/674530057 >> 172.21.15.114:0/674530057 conn(0x7f35780cd890 msgr2=0x7f3578151a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:46.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.404+0000 7f357e600640 1 -- 172.21.15.114:0/674530057 shutdown_connections 2024-05-27T09:27:46.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.404+0000 7f357e600640 1 -- 172.21.15.114:0/674530057 wait complete. 2024-05-27T09:27:46.428 INFO:tasks.mds_thrash:Wait for all MDSs to reach steady state... 2024-05-27T09:27:46.428 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:27:46.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.697+0000 7f8965200640 1 Processor -- start 2024-05-27T09:27:46.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.697+0000 7f8965200640 1 -- start start 2024-05-27T09:27:46.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.699+0000 7f8965200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89600741f0 0x7f89600c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:46.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.699+0000 7f8965200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f896006fe50 con 0x7f8960074840 2024-05-27T09:27:46.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.699+0000 7f8965200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89600700f0 con 0x7f89601547d0 2024-05-27T09:27:46.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.699+0000 7f8965200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f89600703d0 con 0x7f89600741f0 2024-05-27T09:27:46.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.700+0000 7f895fe00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f8960074840 0x7f8960164b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:45390/0 (socket says 172.21.15.114:45390) 2024-05-27T09:27:46.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.700+0000 7f895fe00640 1 -- 172.21.15.114:0/551063999 learned_addr learned my addr 172.21.15.114:0/551063999 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:46.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.700+0000 7f895f400640 1 --2- 172.21.15.114:0/551063999 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89600741f0 0x7f89600c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:46.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.700+0000 7f895e000640 1 -- 172.21.15.114:0/551063999 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4130263107 0 0) 0x7f896006fe50 con 0x7f8960074840 2024-05-27T09:27:46.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.700+0000 7f895e000640 1 -- 172.21.15.114:0/551063999 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8944003520 con 0x7f8960074840 2024-05-27T09:27:46.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.700+0000 7f895e000640 1 -- 172.21.15.114:0/551063999 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3174871349 0 0) 0x7f89600700f0 con 0x7f89601547d0 2024-05-27T09:27:46.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.700+0000 7f895f400640 1 -- 172.21.15.114:0/551063999 >> v1:172.21.15.163:6789/0 conn(0x7f89601547d0 legacy=0x7f896006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:46.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.700+0000 7f895f400640 1 -- 172.21.15.114:0/551063999 >> v1:172.21.15.114:6789/0 conn(0x7f8960074840 legacy=0x7f8960164b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:46.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.701+0000 7f895f400640 1 -- 172.21.15.114:0/551063999 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89600731d0 con 0x7f89600741f0 2024-05-27T09:27:46.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.702+0000 7f895f400640 1 --2- 172.21.15.114:0/551063999 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89600741f0 0x7f89600c63a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f89500097d0 tx=0x7f895002a2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ff02cce425b53aa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:46.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.702+0000 7f895e000640 1 -- 172.21.15.114:0/551063999 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8950037070 con 0x7f89600741f0 2024-05-27T09:27:46.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.702+0000 7f895e000640 1 -- 172.21.15.114:0/551063999 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8950032640 con 0x7f89600741f0 2024-05-27T09:27:46.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.702+0000 7f895e000640 1 -- 172.21.15.114:0/551063999 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8950032a10 con 0x7f89600741f0 2024-05-27T09:27:46.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.703+0000 7f8965200640 1 -- 172.21.15.114:0/551063999 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89600741f0 msgr2=0x7f89600c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:46.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.703+0000 7f8965200640 1 --2- 172.21.15.114:0/551063999 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89600741f0 0x7f89600c63a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f89500097d0 tx=0x7f895002a2c0 comp rx=0 tx=0).stop 2024-05-27T09:27:46.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.703+0000 7f8965200640 1 -- 172.21.15.114:0/551063999 shutdown_connections 2024-05-27T09:27:46.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.703+0000 7f8965200640 1 --2- 172.21.15.114:0/551063999 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89600741f0 0x7f89600c63a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:46.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.703+0000 7f8965200640 1 -- 172.21.15.114:0/551063999 >> 172.21.15.114:0/551063999 conn(0x7f89600ca320 msgr2=0x7f89600ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:46.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.703+0000 7f8965200640 1 -- 172.21.15.114:0/551063999 shutdown_connections 2024-05-27T09:27:46.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.703+0000 7f8965200640 1 -- 172.21.15.114:0/551063999 wait complete. 2024-05-27T09:27:46.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.705+0000 7f8965200640 1 Processor -- start 2024-05-27T09:27:46.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.706+0000 7f8965200640 1 -- start start 2024-05-27T09:27:46.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.707+0000 7f8965200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89600741f0 0x7f89600cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:46.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.707+0000 7f8965200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8960074840 0x7f8960149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:46.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.707+0000 7f8965200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89601547d0 0x7f8960148cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:46.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.707+0000 7f8965200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f896006f500 con 0x7f8960074840 2024-05-27T09:27:46.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.707+0000 7f8965200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8960073790 con 0x7f89601547d0 2024-05-27T09:27:46.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.707+0000 7f8965200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f896006f680 con 0x7f89600741f0 2024-05-27T09:27:46.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.707+0000 7f895ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8960074840 0x7f8960149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:46.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.707+0000 7f895fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89601547d0 0x7f8960148cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:46.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.707+0000 7f895ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8960074840 0x7f8960149be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38004/0 (socket says 172.21.15.114:38004) 2024-05-27T09:27:46.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.707+0000 7f895ea00640 1 -- 172.21.15.114:0/122105951 learned_addr learned my addr 172.21.15.114:0/122105951 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:46.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.707+0000 7f895f400640 1 --2- 172.21.15.114:0/122105951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89600741f0 0x7f89600cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:46.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.708+0000 7f895ea00640 1 -- 172.21.15.114:0/122105951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89600741f0 msgr2=0x7f89600cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:46.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.708+0000 7f895ea00640 1 --2- 172.21.15.114:0/122105951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89600741f0 0x7f89600cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:46.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.708+0000 7f895ea00640 1 -- 172.21.15.114:0/122105951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89601547d0 msgr2=0x7f8960148cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:46.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.708+0000 7f895ea00640 1 --2- 172.21.15.114:0/122105951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89601547d0 0x7f8960148cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:46.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.708+0000 7f895ea00640 1 -- 172.21.15.114:0/122105951 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89600ccb90 con 0x7f8960074840 2024-05-27T09:27:46.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.708+0000 7f895f400640 1 --2- 172.21.15.114:0/122105951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89600741f0 0x7f89600cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:46.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.708+0000 7f895fe00640 1 --2- 172.21.15.114:0/122105951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89601547d0 0x7f8960148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:46.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.708+0000 7f895ea00640 1 --2- 172.21.15.114:0/122105951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8960074840 0x7f8960149be0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f894c00c340 tx=0x7f894c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:46.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.709+0000 7f8957400640 1 -- 172.21.15.114:0/122105951 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c017070 con 0x7f8960074840 2024-05-27T09:27:46.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.709+0000 7f8957400640 1 -- 172.21.15.114:0/122105951 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f894c0035f0 con 0x7f8960074840 2024-05-27T09:27:46.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.709+0000 7f8957400640 1 -- 172.21.15.114:0/122105951 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c0039e0 con 0x7f8960074840 2024-05-27T09:27:46.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.709+0000 7f8965200640 1 -- 172.21.15.114:0/122105951 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89600c35f0 con 0x7f8960074840 2024-05-27T09:27:46.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.709+0000 7f8965200640 1 -- 172.21.15.114:0/122105951 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89600c3bb0 con 0x7f8960074840 2024-05-27T09:27:46.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.710+0000 7f8965200640 1 -- 172.21.15.114:0/122105951 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89600c4d20 con 0x7f8960074840 2024-05-27T09:27:46.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.711+0000 7f8957400640 1 -- 172.21.15.114:0/122105951 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f894c01a070 con 0x7f8960074840 2024-05-27T09:27:46.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.712+0000 7f8957400640 1 --2- 172.21.15.114:0/122105951 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f89400762d0 0x7f89400787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:46.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.713+0000 7f8957400640 1 -- 172.21.15.114:0/122105951 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f894c0968b0 con 0x7f8960074840 2024-05-27T09:27:46.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.713+0000 7f895f400640 1 --2- 172.21.15.114:0/122105951 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f89400762d0 0x7f89400787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:46.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.714+0000 7f895f400640 1 --2- 172.21.15.114:0/122105951 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f89400762d0 0x7f89400787e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8950034000 tx=0x7f895002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:46.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.717+0000 7f8957400640 1 -- 172.21.15.114:0/122105951 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f89600c4d20 con 0x7f8960074840 2024-05-27T09:27:46.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.962+0000 7f8965200640 1 -- 172.21.15.114:0/122105951 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8960157cb0 con 0x7f8960074840 2024-05-27T09:27:46.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.962+0000 7f8957400640 1 -- 172.21.15.114:0/122105951 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+6877 (secure 0 0 0) 0x7f894c05fb20 con 0x7f8960074840 2024-05-27T09:27:46.967 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:46.967 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:27:33.841352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"f","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6839/444766663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":444766663},{"type":"v1","addr":"172.21.15.163:6839","nonce":444766663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4279,4297,4300]},"id":1}]} 2024-05-27T09:27:46.968 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 12 2024-05-27T09:27:46.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.965+0000 7f8965200640 1 -- 172.21.15.114:0/122105951 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f89400762d0 msgr2=0x7f89400787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:46.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.965+0000 7f8965200640 1 --2- 172.21.15.114:0/122105951 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f89400762d0 0x7f89400787e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8950034000 tx=0x7f895002e000 comp rx=0 tx=0).stop 2024-05-27T09:27:46.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.965+0000 7f8965200640 1 -- 172.21.15.114:0/122105951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8960074840 msgr2=0x7f8960149be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:46.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.965+0000 7f8965200640 1 --2- 172.21.15.114:0/122105951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8960074840 0x7f8960149be0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f894c00c340 tx=0x7f894c00c800 comp rx=0 tx=0).stop 2024-05-27T09:27:46.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.965+0000 7f8965200640 1 -- 172.21.15.114:0/122105951 shutdown_connections 2024-05-27T09:27:46.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.965+0000 7f8965200640 1 --2- 172.21.15.114:0/122105951 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f89400762d0 0x7f89400787e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:46.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.965+0000 7f8965200640 1 --2- 172.21.15.114:0/122105951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89601547d0 0x7f8960148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:46.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.965+0000 7f8965200640 1 --2- 172.21.15.114:0/122105951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8960074840 0x7f8960149be0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:46.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.965+0000 7f8965200640 1 --2- 172.21.15.114:0/122105951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89600741f0 0x7f89600cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:46.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.965+0000 7f8965200640 1 -- 172.21.15.114:0/122105951 >> 172.21.15.114:0/122105951 conn(0x7f89600ca320 msgr2=0x7f896006edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:46.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.965+0000 7f8965200640 1 -- 172.21.15.114:0/122105951 shutdown_connections 2024-05-27T09:27:46.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:46.965+0000 7f8965200640 1 -- 172.21.15.114:0/122105951 wait complete. 2024-05-27T09:27:46.991 INFO:tasks.mds_thrash:Ready to start thrashing 2024-05-27T09:27:46.991 INFO:tasks.mds_thrash.ceph_manager:waiting for clean 2024-05-27T09:27:46.991 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T09:27:47.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.249+0000 7f4d12000640 1 Processor -- start 2024-05-27T09:27:47.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.250+0000 7f4d12000640 1 -- start start 2024-05-27T09:27:47.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.252+0000 7f4d12000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d0c0b75b0 0x7f4d0c0d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:47.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.252+0000 7f4d12000640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d0c0cd1e0 con 0x7f4d0c0b46e0 2024-05-27T09:27:47.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.252+0000 7f4d12000640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d0c0cd4c0 con 0x7f4d0c0c8780 2024-05-27T09:27:47.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.252+0000 7f4d12000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f4d0c0c7f80 con 0x7f4d0c0b75b0 2024-05-27T09:27:47.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.253+0000 7f4d10c00640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f4d0c0c8780 0x7f4d0c15fc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:41226/0 (socket says 172.21.15.114:41226) 2024-05-27T09:27:47.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.253+0000 7f4d10c00640 1 -- 172.21.15.114:0/1527972568 learned_addr learned my addr 172.21.15.114:0/1527972568 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:47.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.253+0000 7f4d0be00640 1 --2- 172.21.15.114:0/1527972568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d0c0b75b0 0x7f4d0c0d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:47.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.253+0000 7f4d0b400640 1 -- 172.21.15.114:0/1527972568 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3897834700 0 0) 0x7f4d0c0cd4c0 con 0x7f4d0c0c8780 2024-05-27T09:27:47.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.253+0000 7f4d0b400640 1 -- 172.21.15.114:0/1527972568 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4cf0003520 con 0x7f4d0c0c8780 2024-05-27T09:27:47.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.253+0000 7f4d0b400640 1 -- 172.21.15.114:0/1527972568 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1137598615 0 0) 0x7f4d0c0cd1e0 con 0x7f4d0c0b46e0 2024-05-27T09:27:47.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.253+0000 7f4d0be00640 1 -- 172.21.15.114:0/1527972568 >> v1:172.21.15.163:6789/0 conn(0x7f4d0c0c8780 legacy=0x7f4d0c15fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:47.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.253+0000 7f4d0be00640 1 -- 172.21.15.114:0/1527972568 >> v1:172.21.15.114:6789/0 conn(0x7f4d0c0b46e0 legacy=0x7f4d0c166fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:47.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.253+0000 7f4d0be00640 1 -- 172.21.15.114:0/1527972568 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0c0c6bb0 con 0x7f4d0c0b75b0 2024-05-27T09:27:47.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.254+0000 7f4d0be00640 1 --2- 172.21.15.114:0/1527972568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d0c0b75b0 0x7f4d0c0d1e50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4cf40097d0 tx=0x7f4cf402a2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6641bb6f32276c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:47.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.255+0000 7f4d0b400640 1 -- 172.21.15.114:0/1527972568 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4037070 con 0x7f4d0c0b75b0 2024-05-27T09:27:47.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.255+0000 7f4d0b400640 1 -- 172.21.15.114:0/1527972568 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cf4032640 con 0x7f4d0c0b75b0 2024-05-27T09:27:47.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.255+0000 7f4d0b400640 1 -- 172.21.15.114:0/1527972568 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf40329f0 con 0x7f4d0c0b75b0 2024-05-27T09:27:47.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.256+0000 7f4d12000640 1 -- 172.21.15.114:0/1527972568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d0c0b75b0 msgr2=0x7f4d0c0d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:47.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.256+0000 7f4d12000640 1 --2- 172.21.15.114:0/1527972568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d0c0b75b0 0x7f4d0c0d1e50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4cf40097d0 tx=0x7f4cf402a2c0 comp rx=0 tx=0).stop 2024-05-27T09:27:47.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.256+0000 7f4d12000640 1 -- 172.21.15.114:0/1527972568 shutdown_connections 2024-05-27T09:27:47.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.256+0000 7f4d12000640 1 --2- 172.21.15.114:0/1527972568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d0c0b75b0 0x7f4d0c0d1e50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:47.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.256+0000 7f4d12000640 1 -- 172.21.15.114:0/1527972568 >> 172.21.15.114:0/1527972568 conn(0x7f4d0c06e040 msgr2=0x7f4d0c06e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:47.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.256+0000 7f4d12000640 1 -- 172.21.15.114:0/1527972568 shutdown_connections 2024-05-27T09:27:47.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.256+0000 7f4d12000640 1 -- 172.21.15.114:0/1527972568 wait complete. 2024-05-27T09:27:47.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.258+0000 7f4d12000640 1 Processor -- start 2024-05-27T09:27:47.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.258+0000 7f4d12000640 1 -- start start 2024-05-27T09:27:47.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.259+0000 7f4d12000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d0c0b46e0 0x7f4d0c1f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:47.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.260+0000 7f4d12000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d0c0b75b0 0x7f4d0c159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:47.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.260+0000 7f4d12000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d0c0c8780 0x7f4d0c14e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:47.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.260+0000 7f4d12000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0c0ce0b0 con 0x7f4d0c0c8780 2024-05-27T09:27:47.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.260+0000 7f4d12000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0c0cdf30 con 0x7f4d0c0b46e0 2024-05-27T09:27:47.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.260+0000 7f4d12000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f4d0c156c40 con 0x7f4d0c0b75b0 2024-05-27T09:27:47.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.260+0000 7f4d11600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d0c0c8780 0x7f4d0c14e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:47.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.260+0000 7f4d11600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d0c0c8780 0x7f4d0c14e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38018/0 (socket says 172.21.15.114:38018) 2024-05-27T09:27:47.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.260+0000 7f4d11600640 1 -- 172.21.15.114:0/509283859 learned_addr learned my addr 172.21.15.114:0/509283859 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:47.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.260+0000 7f4d10c00640 1 --2- 172.21.15.114:0/509283859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d0c0b46e0 0x7f4d0c1f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:47.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.260+0000 7f4d0be00640 1 --2- 172.21.15.114:0/509283859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d0c0b75b0 0x7f4d0c159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:47.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.260+0000 7f4d11600640 1 -- 172.21.15.114:0/509283859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d0c0b75b0 msgr2=0x7f4d0c159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:47.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.260+0000 7f4d11600640 1 --2- 172.21.15.114:0/509283859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d0c0b75b0 0x7f4d0c159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:47.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.260+0000 7f4d11600640 1 -- 172.21.15.114:0/509283859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d0c0b46e0 msgr2=0x7f4d0c1f59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:47.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.260+0000 7f4d11600640 1 --2- 172.21.15.114:0/509283859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d0c0b46e0 0x7f4d0c1f59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:47.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.260+0000 7f4d11600640 1 -- 172.21.15.114:0/509283859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0c1f5ee0 con 0x7f4d0c0c8780 2024-05-27T09:27:47.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.261+0000 7f4d11600640 1 --2- 172.21.15.114:0/509283859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d0c0c8780 0x7f4d0c14e1b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400c760 tx=0x7f4d0400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:47.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.261+0000 7f4d08c00640 1 -- 172.21.15.114:0/509283859 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d04003b90 con 0x7f4d0c0c8780 2024-05-27T09:27:47.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.261+0000 7f4d08c00640 1 -- 172.21.15.114:0/509283859 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d04003d50 con 0x7f4d0c0c8780 2024-05-27T09:27:47.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.261+0000 7f4d12000640 1 -- 172.21.15.114:0/509283859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d0c15f970 con 0x7f4d0c0c8780 2024-05-27T09:27:47.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.261+0000 7f4d08c00640 1 -- 172.21.15.114:0/509283859 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d04005310 con 0x7f4d0c0c8780 2024-05-27T09:27:47.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.261+0000 7f4d12000640 1 -- 172.21.15.114:0/509283859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d0c15df10 con 0x7f4d0c0c8780 2024-05-27T09:27:47.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.262+0000 7f4d12000640 1 -- 172.21.15.114:0/509283859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d0c050860 con 0x7f4d0c0c8780 2024-05-27T09:27:47.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.264+0000 7f4d08c00640 1 -- 172.21.15.114:0/509283859 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4d040054b0 con 0x7f4d0c0c8780 2024-05-27T09:27:47.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.264+0000 7f4d08c00640 1 --2- 172.21.15.114:0/509283859 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4ce8076300 0x7f4ce8078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:47.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.265+0000 7f4d08c00640 1 -- 172.21.15.114:0/509283859 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4d040973b0 con 0x7f4d0c0c8780 2024-05-27T09:27:47.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.265+0000 7f4d10c00640 1 --2- 172.21.15.114:0/509283859 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4ce8076300 0x7f4ce8078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:47.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.266+0000 7f4d10c00640 1 --2- 172.21.15.114:0/509283859 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4ce8076300 0x7f4ce8078810 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc005880 tx=0x7f4cfc006aa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:47.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.269+0000 7f4d08c00640 1 -- 172.21.15.114:0/509283859 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4d04060630 con 0x7f4d0c0c8780 2024-05-27T09:27:47.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.461+0000 7f4d12000640 1 -- 172.21.15.114:0/509283859 --> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4d0c15f720 con 0x7f4ce8076300 2024-05-27T09:27:47.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.469+0000 7f4d08c00640 1 -- 172.21.15.114:0/509283859 <== mgr.4108 v2:172.21.15.163:6832/1122018204 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251982 (secure 0 0 0) 0x7f4d0c15f720 con 0x7f4ce8076300 2024-05-27T09:27:47.472 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:47.474 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-05-27T09:27:47.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.475+0000 7f4d12000640 1 -- 172.21.15.114:0/509283859 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4ce8076300 msgr2=0x7f4ce8078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:47.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.475+0000 7f4d12000640 1 --2- 172.21.15.114:0/509283859 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4ce8076300 0x7f4ce8078810 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc005880 tx=0x7f4cfc006aa0 comp rx=0 tx=0).stop 2024-05-27T09:27:47.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.475+0000 7f4d12000640 1 -- 172.21.15.114:0/509283859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d0c0c8780 msgr2=0x7f4d0c14e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:47.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.475+0000 7f4d12000640 1 --2- 172.21.15.114:0/509283859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d0c0c8780 0x7f4d0c14e1b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400c760 tx=0x7f4d0400cc20 comp rx=0 tx=0).stop 2024-05-27T09:27:47.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.475+0000 7f4d12000640 1 -- 172.21.15.114:0/509283859 shutdown_connections 2024-05-27T09:27:47.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.475+0000 7f4d12000640 1 --2- 172.21.15.114:0/509283859 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4ce8076300 0x7f4ce8078810 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:47.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.475+0000 7f4d12000640 1 --2- 172.21.15.114:0/509283859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d0c0c8780 0x7f4d0c14e1b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:47.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.475+0000 7f4d12000640 1 --2- 172.21.15.114:0/509283859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d0c0b75b0 0x7f4d0c159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:47.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.475+0000 7f4d12000640 1 --2- 172.21.15.114:0/509283859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d0c0b46e0 0x7f4d0c1f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:47.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.475+0000 7f4d12000640 1 -- 172.21.15.114:0/509283859 >> 172.21.15.114:0/509283859 conn(0x7f4d0c06e040 msgr2=0x7f4d0c0b66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:47.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.475+0000 7f4d12000640 1 -- 172.21.15.114:0/509283859 shutdown_connections 2024-05-27T09:27:47.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.475+0000 7f4d12000640 1 -- 172.21.15.114:0/509283859 wait complete. 2024-05-27T09:27:47.500 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-05-27T09:27:46.769776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595844,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":227,"num_write_kb":1667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":182,"ondisk_log_size":182,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218048,"kb_used_data":1920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756672,"statfs":{"total":773094113280,"available":772870832128,"internally_reserved":0,"allocated":1966080,"data_stored":1773342,"data_compressed":65078,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1660,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003132"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544950+0000","last_change":"2024-05-27T09:27:18.585655+0000","last_active":"2024-05-27T09:27:19.544950+0000","last_peered":"2024-05-27T09:27:19.544950+0000","last_clean":"2024-05-27T09:27:19.544950+0000","last_became_active":"2024-05-27T09:27:18.585169+0000","last_became_peered":"2024-05-27T09:27:18.585169+0000","last_unstale":"2024-05-27T09:27:19.544950+0000","last_undegraded":"2024-05-27T09:27:19.544950+0000","last_fullsized":"2024-05-27T09:27:19.544950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:39:59.034873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835983+0000","last_change":"2024-05-27T09:27:18.566621+0000","last_active":"2024-05-27T09:27:20.835983+0000","last_peered":"2024-05-27T09:27:20.835983+0000","last_clean":"2024-05-27T09:27:20.835983+0000","last_became_active":"2024-05-27T09:27:18.566269+0000","last_became_peered":"2024-05-27T09:27:18.566269+0000","last_unstale":"2024-05-27T09:27:20.835983+0000","last_undegraded":"2024-05-27T09:27:20.835983+0000","last_fullsized":"2024-05-27T09:27:20.835983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:24:24.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835531+0000","last_change":"2024-05-27T09:27:18.560958+0000","last_active":"2024-05-27T09:27:20.835531+0000","last_peered":"2024-05-27T09:27:20.835531+0000","last_clean":"2024-05-27T09:27:20.835531+0000","last_became_active":"2024-05-27T09:27:18.560620+0000","last_became_peered":"2024-05-27T09:27:18.560620+0000","last_unstale":"2024-05-27T09:27:20.835531+0000","last_undegraded":"2024-05-27T09:27:20.835531+0000","last_fullsized":"2024-05-27T09:27:20.835531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:42:38.351682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542664+0000","last_change":"2024-05-27T09:27:18.569804+0000","last_active":"2024-05-27T09:27:19.542664+0000","last_peered":"2024-05-27T09:27:19.542664+0000","last_clean":"2024-05-27T09:27:19.542664+0000","last_became_active":"2024-05-27T09:27:18.569481+0000","last_became_peered":"2024-05-27T09:27:18.569481+0000","last_unstale":"2024-05-27T09:27:19.542664+0000","last_undegraded":"2024-05-27T09:27:19.542664+0000","last_fullsized":"2024-05-27T09:27:19.542664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:55:03.583659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543808+0000","last_change":"2024-05-27T09:27:18.568138+0000","last_active":"2024-05-27T09:27:19.543808+0000","last_peered":"2024-05-27T09:27:19.543808+0000","last_clean":"2024-05-27T09:27:19.543808+0000","last_became_active":"2024-05-27T09:27:18.567795+0000","last_became_peered":"2024-05-27T09:27:18.567795+0000","last_unstale":"2024-05-27T09:27:19.543808+0000","last_undegraded":"2024-05-27T09:27:19.543808+0000","last_fullsized":"2024-05-27T09:27:19.543808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:17:42.005593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.540719+0000","last_change":"2024-05-27T09:27:18.573984+0000","last_active":"2024-05-27T09:27:19.540719+0000","last_peered":"2024-05-27T09:27:19.540719+0000","last_clean":"2024-05-27T09:27:19.540719+0000","last_became_active":"2024-05-27T09:27:18.573576+0000","last_became_peered":"2024-05-27T09:27:18.573576+0000","last_unstale":"2024-05-27T09:27:19.540719+0000","last_undegraded":"2024-05-27T09:27:19.540719+0000","last_fullsized":"2024-05-27T09:27:19.540719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:06.224637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543010+0000","last_change":"2024-05-27T09:27:18.543122+0000","last_active":"2024-05-27T09:27:19.543010+0000","last_peered":"2024-05-27T09:27:19.543010+0000","last_clean":"2024-05-27T09:27:19.543010+0000","last_became_active":"2024-05-27T09:27:18.542593+0000","last_became_peered":"2024-05-27T09:27:18.542593+0000","last_unstale":"2024-05-27T09:27:19.543010+0000","last_undegraded":"2024-05-27T09:27:19.543010+0000","last_fullsized":"2024-05-27T09:27:19.543010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:28:19.579432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835946+0000","last_change":"2024-05-27T09:27:18.562292+0000","last_active":"2024-05-27T09:27:20.835946+0000","last_peered":"2024-05-27T09:27:20.835946+0000","last_clean":"2024-05-27T09:27:20.835946+0000","last_became_active":"2024-05-27T09:27:18.561788+0000","last_became_peered":"2024-05-27T09:27:18.561788+0000","last_unstale":"2024-05-27T09:27:20.835946+0000","last_undegraded":"2024-05-27T09:27:20.835946+0000","last_fullsized":"2024-05-27T09:27:20.835946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:32:39.043268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542996+0000","last_change":"2024-05-27T09:27:18.583770+0000","last_active":"2024-05-27T09:27:19.542996+0000","last_peered":"2024-05-27T09:27:19.542996+0000","last_clean":"2024-05-27T09:27:19.542996+0000","last_became_active":"2024-05-27T09:27:18.582041+0000","last_became_peered":"2024-05-27T09:27:18.582041+0000","last_unstale":"2024-05-27T09:27:19.542996+0000","last_undegraded":"2024-05-27T09:27:19.542996+0000","last_fullsized":"2024-05-27T09:27:19.542996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:19.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.545125+0000","last_change":"2024-05-27T09:27:18.574658+0000","last_active":"2024-05-27T09:27:19.545125+0000","last_peered":"2024-05-27T09:27:19.545125+0000","last_clean":"2024-05-27T09:27:19.545125+0000","last_became_active":"2024-05-27T09:27:18.574409+0000","last_became_peered":"2024-05-27T09:27:18.574409+0000","last_unstale":"2024-05-27T09:27:19.545125+0000","last_undegraded":"2024-05-27T09:27:19.545125+0000","last_fullsized":"2024-05-27T09:27:19.545125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:19.131343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544558+0000","last_change":"2024-05-27T09:27:18.579141+0000","last_active":"2024-05-27T09:27:19.544558+0000","last_peered":"2024-05-27T09:27:19.544558+0000","last_clean":"2024-05-27T09:27:19.544558+0000","last_became_active":"2024-05-27T09:27:18.578679+0000","last_became_peered":"2024-05-27T09:27:18.578679+0000","last_unstale":"2024-05-27T09:27:19.544558+0000","last_undegraded":"2024-05-27T09:27:19.544558+0000","last_fullsized":"2024-05-27T09:27:19.544558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:44.336453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544110+0000","last_change":"2024-05-27T09:27:18.547375+0000","last_active":"2024-05-27T09:27:19.544110+0000","last_peered":"2024-05-27T09:27:19.544110+0000","last_clean":"2024-05-27T09:27:19.544110+0000","last_became_active":"2024-05-27T09:27:18.544328+0000","last_became_peered":"2024-05-27T09:27:18.544328+0000","last_unstale":"2024-05-27T09:27:19.544110+0000","last_undegraded":"2024-05-27T09:27:19.544110+0000","last_fullsized":"2024-05-27T09:27:19.544110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:59:50.499894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836590+0000","last_change":"2024-05-27T09:27:18.561903+0000","last_active":"2024-05-27T09:27:20.836590+0000","last_peered":"2024-05-27T09:27:20.836590+0000","last_clean":"2024-05-27T09:27:20.836590+0000","last_became_active":"2024-05-27T09:27:18.561493+0000","last_became_peered":"2024-05-27T09:27:18.561493+0000","last_unstale":"2024-05-27T09:27:20.836590+0000","last_undegraded":"2024-05-27T09:27:20.836590+0000","last_fullsized":"2024-05-27T09:27:20.836590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:28:31.470269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836378+0000","last_change":"2024-05-27T09:27:18.563168+0000","last_active":"2024-05-27T09:27:20.836378+0000","last_peered":"2024-05-27T09:27:20.836378+0000","last_clean":"2024-05-27T09:27:20.836378+0000","last_became_active":"2024-05-27T09:27:18.562843+0000","last_became_peered":"2024-05-27T09:27:18.562843+0000","last_unstale":"2024-05-27T09:27:20.836378+0000","last_undegraded":"2024-05-27T09:27:20.836378+0000","last_fullsized":"2024-05-27T09:27:20.836378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:33:36.671764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835457+0000","last_change":"2024-05-27T09:27:18.566502+0000","last_active":"2024-05-27T09:27:20.835457+0000","last_peered":"2024-05-27T09:27:20.835457+0000","last_clean":"2024-05-27T09:27:20.835457+0000","last_became_active":"2024-05-27T09:27:18.566108+0000","last_became_peered":"2024-05-27T09:27:18.566108+0000","last_unstale":"2024-05-27T09:27:20.835457+0000","last_undegraded":"2024-05-27T09:27:20.835457+0000","last_fullsized":"2024-05-27T09:27:20.835457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:44:47.447245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836471+0000","last_change":"2024-05-27T09:27:18.582595+0000","last_active":"2024-05-27T09:27:20.836471+0000","last_peered":"2024-05-27T09:27:20.836471+0000","last_clean":"2024-05-27T09:27:20.836471+0000","last_became_active":"2024-05-27T09:27:18.581719+0000","last_became_peered":"2024-05-27T09:27:18.581719+0000","last_unstale":"2024-05-27T09:27:20.836471+0000","last_undegraded":"2024-05-27T09:27:20.836471+0000","last_fullsized":"2024-05-27T09:27:20.836471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:48:19.573654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837234+0000","last_change":"2024-05-27T09:27:18.577565+0000","last_active":"2024-05-27T09:27:20.837234+0000","last_peered":"2024-05-27T09:27:20.837234+0000","last_clean":"2024-05-27T09:27:20.837234+0000","last_became_active":"2024-05-27T09:27:18.577193+0000","last_became_peered":"2024-05-27T09:27:18.577193+0000","last_unstale":"2024-05-27T09:27:20.837234+0000","last_undegraded":"2024-05-27T09:27:20.837234+0000","last_fullsized":"2024-05-27T09:27:20.837234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:43:20.065873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835530+0000","last_change":"2024-05-27T09:27:18.571089+0000","last_active":"2024-05-27T09:27:20.835530+0000","last_peered":"2024-05-27T09:27:20.835530+0000","last_clean":"2024-05-27T09:27:20.835530+0000","last_became_active":"2024-05-27T09:27:18.570712+0000","last_became_peered":"2024-05-27T09:27:18.570712+0000","last_unstale":"2024-05-27T09:27:20.835530+0000","last_undegraded":"2024-05-27T09:27:20.835530+0000","last_fullsized":"2024-05-27T09:27:20.835530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:40:19.310581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837956+0000","last_change":"2024-05-27T09:27:18.572675+0000","last_active":"2024-05-27T09:27:20.837956+0000","last_peered":"2024-05-27T09:27:20.837956+0000","last_clean":"2024-05-27T09:27:20.837956+0000","last_became_active":"2024-05-27T09:27:18.564643+0000","last_became_peered":"2024-05-27T09:27:18.564643+0000","last_unstale":"2024-05-27T09:27:20.837956+0000","last_undegraded":"2024-05-27T09:27:20.837956+0000","last_fullsized":"2024-05-27T09:27:20.837956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:36:15.602241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544179+0000","last_change":"2024-05-27T09:27:18.544554+0000","last_active":"2024-05-27T09:27:19.544179+0000","last_peered":"2024-05-27T09:27:19.544179+0000","last_clean":"2024-05-27T09:27:19.544179+0000","last_became_active":"2024-05-27T09:27:18.543777+0000","last_became_peered":"2024-05-27T09:27:18.543777+0000","last_unstale":"2024-05-27T09:27:19.544179+0000","last_undegraded":"2024-05-27T09:27:19.544179+0000","last_fullsized":"2024-05-27T09:27:19.544179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:35:43.354270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835033+0000","last_change":"2024-05-27T09:27:18.571321+0000","last_active":"2024-05-27T09:27:20.835033+0000","last_peered":"2024-05-27T09:27:20.835033+0000","last_clean":"2024-05-27T09:27:20.835033+0000","last_became_active":"2024-05-27T09:27:18.570948+0000","last_became_peered":"2024-05-27T09:27:18.570948+0000","last_unstale":"2024-05-27T09:27:20.835033+0000","last_undegraded":"2024-05-27T09:27:20.835033+0000","last_fullsized":"2024-05-27T09:27:20.835033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:28:31.470269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539490+0000","last_change":"2024-05-27T09:27:18.544282+0000","last_active":"2024-05-27T09:27:19.539490+0000","last_peered":"2024-05-27T09:27:19.539490+0000","last_clean":"2024-05-27T09:27:19.539490+0000","last_became_active":"2024-05-27T09:27:18.543921+0000","last_became_peered":"2024-05-27T09:27:18.543921+0000","last_unstale":"2024-05-27T09:27:19.539490+0000","last_undegraded":"2024-05-27T09:27:19.539490+0000","last_fullsized":"2024-05-27T09:27:19.539490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:27.225150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835732+0000","last_change":"2024-05-27T09:27:18.575415+0000","last_active":"2024-05-27T09:27:20.835732+0000","last_peered":"2024-05-27T09:27:20.835732+0000","last_clean":"2024-05-27T09:27:20.835732+0000","last_became_active":"2024-05-27T09:27:18.575115+0000","last_became_peered":"2024-05-27T09:27:18.575115+0000","last_unstale":"2024-05-27T09:27:20.835732+0000","last_undegraded":"2024-05-27T09:27:20.835732+0000","last_fullsized":"2024-05-27T09:27:20.835732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:50:48.091125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836496+0000","last_change":"2024-05-27T09:27:18.564422+0000","last_active":"2024-05-27T09:27:20.836496+0000","last_peered":"2024-05-27T09:27:20.836496+0000","last_clean":"2024-05-27T09:27:20.836496+0000","last_became_active":"2024-05-27T09:27:18.563991+0000","last_became_peered":"2024-05-27T09:27:18.563991+0000","last_unstale":"2024-05-27T09:27:20.836496+0000","last_undegraded":"2024-05-27T09:27:20.836496+0000","last_fullsized":"2024-05-27T09:27:20.836496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:19.131343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836032+0000","last_change":"2024-05-27T09:27:18.563557+0000","last_active":"2024-05-27T09:27:20.836032+0000","last_peered":"2024-05-27T09:27:20.836032+0000","last_clean":"2024-05-27T09:27:20.836032+0000","last_became_active":"2024-05-27T09:27:18.562803+0000","last_became_peered":"2024-05-27T09:27:18.562803+0000","last_unstale":"2024-05-27T09:27:20.836032+0000","last_undegraded":"2024-05-27T09:27:20.836032+0000","last_fullsized":"2024-05-27T09:27:20.836032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:47:54.271208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542223+0000","last_change":"2024-05-27T09:27:18.590430+0000","last_active":"2024-05-27T09:27:19.542223+0000","last_peered":"2024-05-27T09:27:19.542223+0000","last_clean":"2024-05-27T09:27:19.542223+0000","last_became_active":"2024-05-27T09:27:18.590083+0000","last_became_peered":"2024-05-27T09:27:18.590083+0000","last_unstale":"2024-05-27T09:27:19.542223+0000","last_undegraded":"2024-05-27T09:27:19.542223+0000","last_fullsized":"2024-05-27T09:27:19.542223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:03.762818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837167+0000","last_change":"2024-05-27T09:27:18.558248+0000","last_active":"2024-05-27T09:27:20.837167+0000","last_peered":"2024-05-27T09:27:20.837167+0000","last_clean":"2024-05-27T09:27:20.837167+0000","last_became_active":"2024-05-27T09:27:18.557953+0000","last_became_peered":"2024-05-27T09:27:18.557953+0000","last_unstale":"2024-05-27T09:27:20.837167+0000","last_undegraded":"2024-05-27T09:27:20.837167+0000","last_fullsized":"2024-05-27T09:27:20.837167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:44:47.447245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836514+0000","last_change":"2024-05-27T09:27:18.563213+0000","last_active":"2024-05-27T09:27:20.836514+0000","last_peered":"2024-05-27T09:27:20.836514+0000","last_clean":"2024-05-27T09:27:20.836514+0000","last_became_active":"2024-05-27T09:27:18.562839+0000","last_became_peered":"2024-05-27T09:27:18.562839+0000","last_unstale":"2024-05-27T09:27:20.836514+0000","last_undegraded":"2024-05-27T09:27:20.836514+0000","last_fullsized":"2024-05-27T09:27:20.836514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:43.851698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836053+0000","last_change":"2024-05-27T09:27:18.562237+0000","last_active":"2024-05-27T09:27:20.836053+0000","last_peered":"2024-05-27T09:27:20.836053+0000","last_clean":"2024-05-27T09:27:20.836053+0000","last_became_active":"2024-05-27T09:27:18.561475+0000","last_became_peered":"2024-05-27T09:27:18.561475+0000","last_unstale":"2024-05-27T09:27:20.836053+0000","last_undegraded":"2024-05-27T09:27:20.836053+0000","last_fullsized":"2024-05-27T09:27:20.836053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:39:58.110527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837611+0000","last_change":"2024-05-27T09:27:18.563469+0000","last_active":"2024-05-27T09:27:20.837611+0000","last_peered":"2024-05-27T09:27:20.837611+0000","last_clean":"2024-05-27T09:27:20.837611+0000","last_became_active":"2024-05-27T09:27:18.563103+0000","last_became_peered":"2024-05-27T09:27:18.563103+0000","last_unstale":"2024-05-27T09:27:20.837611+0000","last_undegraded":"2024-05-27T09:27:20.837611+0000","last_fullsized":"2024-05-27T09:27:20.837611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:53:57.281169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.834984+0000","last_change":"2024-05-27T09:27:18.560551+0000","last_active":"2024-05-27T09:27:20.834984+0000","last_peered":"2024-05-27T09:27:20.834984+0000","last_clean":"2024-05-27T09:27:20.834984+0000","last_became_active":"2024-05-27T09:27:18.560244+0000","last_became_peered":"2024-05-27T09:27:18.560244+0000","last_unstale":"2024-05-27T09:27:20.834984+0000","last_undegraded":"2024-05-27T09:27:20.834984+0000","last_fullsized":"2024-05-27T09:27:20.834984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:00.131180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.538536+0000","last_change":"2024-05-27T09:27:18.568200+0000","last_active":"2024-05-27T09:27:19.538536+0000","last_peered":"2024-05-27T09:27:19.538536+0000","last_clean":"2024-05-27T09:27:19.538536+0000","last_became_active":"2024-05-27T09:27:18.567845+0000","last_became_peered":"2024-05-27T09:27:18.567845+0000","last_unstale":"2024-05-27T09:27:19.538536+0000","last_undegraded":"2024-05-27T09:27:19.538536+0000","last_fullsized":"2024-05-27T09:27:19.538536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:54.726617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.540019+0000","last_change":"2024-05-27T09:27:18.560521+0000","last_active":"2024-05-27T09:27:19.540019+0000","last_peered":"2024-05-27T09:27:19.540019+0000","last_clean":"2024-05-27T09:27:19.540019+0000","last_became_active":"2024-05-27T09:27:18.556156+0000","last_became_peered":"2024-05-27T09:27:18.556156+0000","last_unstale":"2024-05-27T09:27:19.540019+0000","last_undegraded":"2024-05-27T09:27:19.540019+0000","last_fullsized":"2024-05-27T09:27:19.540019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:03.762818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.631378+0000","last_change":"2024-05-27T09:27:17.548038+0000","last_active":"2024-05-27T09:27:28.631378+0000","last_peered":"2024-05-27T09:27:28.631378+0000","last_clean":"2024-05-27T09:27:28.631378+0000","last_became_active":"2024-05-27T09:27:17.547653+0000","last_became_peered":"2024-05-27T09:27:17.547653+0000","last_unstale":"2024-05-27T09:27:28.631378+0000","last_undegraded":"2024-05-27T09:27:28.631378+0000","last_fullsized":"2024-05-27T09:27:28.631378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":876,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835265+0000","last_change":"2024-05-27T09:27:18.543048+0000","last_active":"2024-05-27T09:27:20.835265+0000","last_peered":"2024-05-27T09:27:20.835265+0000","last_clean":"2024-05-27T09:27:20.835265+0000","last_became_active":"2024-05-27T09:27:18.542715+0000","last_became_peered":"2024-05-27T09:27:18.542715+0000","last_unstale":"2024-05-27T09:27:20.835265+0000","last_undegraded":"2024-05-27T09:27:20.835265+0000","last_fullsized":"2024-05-27T09:27:20.835265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:46:49.444709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.738235+0000","last_change":"2024-05-27T09:27:17.570014+0000","last_active":"2024-05-27T09:27:33.738235+0000","last_peered":"2024-05-27T09:27:33.738235+0000","last_clean":"2024-05-27T09:27:33.738235+0000","last_became_active":"2024-05-27T09:27:17.569673+0000","last_became_peered":"2024-05-27T09:27:17.569673+0000","last_unstale":"2024-05-27T09:27:33.738235+0000","last_undegraded":"2024-05-27T09:27:33.738235+0000","last_fullsized":"2024-05-27T09:27:33.738235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:26.210041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.738367+0000","last_change":"2024-05-27T09:27:17.565489+0000","last_active":"2024-05-27T09:27:33.738367+0000","last_peered":"2024-05-27T09:27:33.738367+0000","last_clean":"2024-05-27T09:27:33.738367+0000","last_became_active":"2024-05-27T09:27:17.565137+0000","last_became_peered":"2024-05-27T09:27:17.565137+0000","last_unstale":"2024-05-27T09:27:33.738367+0000","last_undegraded":"2024-05-27T09:27:33.738367+0000","last_fullsized":"2024-05-27T09:27:33.738367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836835+0000","last_change":"2024-05-27T09:27:18.565294+0000","last_active":"2024-05-27T09:27:20.836835+0000","last_peered":"2024-05-27T09:27:20.836835+0000","last_clean":"2024-05-27T09:27:20.836835+0000","last_became_active":"2024-05-27T09:27:18.564115+0000","last_became_peered":"2024-05-27T09:27:18.564115+0000","last_unstale":"2024-05-27T09:27:20.836835+0000","last_undegraded":"2024-05-27T09:27:20.836835+0000","last_fullsized":"2024-05-27T09:27:20.836835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:57:20.488849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836816+0000","last_change":"2024-05-27T09:27:17.561204+0000","last_active":"2024-05-27T09:27:20.836816+0000","last_peered":"2024-05-27T09:27:20.836816+0000","last_clean":"2024-05-27T09:27:20.836816+0000","last_became_active":"2024-05-27T09:27:17.560149+0000","last_became_peered":"2024-05-27T09:27:17.560149+0000","last_unstale":"2024-05-27T09:27:20.836816+0000","last_undegraded":"2024-05-27T09:27:20.836816+0000","last_fullsized":"2024-05-27T09:27:20.836816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.284270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836343+0000","last_change":"2024-05-27T09:27:18.542204+0000","last_active":"2024-05-27T09:27:20.836343+0000","last_peered":"2024-05-27T09:27:20.836343+0000","last_clean":"2024-05-27T09:27:20.836343+0000","last_became_active":"2024-05-27T09:27:18.541946+0000","last_became_peered":"2024-05-27T09:27:18.541946+0000","last_unstale":"2024-05-27T09:27:20.836343+0000","last_undegraded":"2024-05-27T09:27:20.836343+0000","last_fullsized":"2024-05-27T09:27:20.836343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:50:48.091125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.639056+0000","last_change":"2024-05-27T09:27:17.561714+0000","last_active":"2024-05-27T09:27:28.639056+0000","last_peered":"2024-05-27T09:27:28.639056+0000","last_clean":"2024-05-27T09:27:28.639056+0000","last_became_active":"2024-05-27T09:27:17.561355+0000","last_became_peered":"2024-05-27T09:27:17.561355+0000","last_unstale":"2024-05-27T09:27:28.639056+0000","last_undegraded":"2024-05-27T09:27:28.639056+0000","last_fullsized":"2024-05-27T09:27:28.639056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:07:02.481451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542128+0000","last_change":"2024-05-27T09:27:18.588199+0000","last_active":"2024-05-27T09:27:19.542128+0000","last_peered":"2024-05-27T09:27:19.542128+0000","last_clean":"2024-05-27T09:27:19.542128+0000","last_became_active":"2024-05-27T09:27:18.587921+0000","last_became_peered":"2024-05-27T09:27:18.587921+0000","last_unstale":"2024-05-27T09:27:19.542128+0000","last_undegraded":"2024-05-27T09:27:19.542128+0000","last_fullsized":"2024-05-27T09:27:19.542128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:17.895211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.620737+0000","last_change":"2024-05-27T09:27:17.565854+0000","last_active":"2024-05-27T09:27:28.620737+0000","last_peered":"2024-05-27T09:27:28.620737+0000","last_clean":"2024-05-27T09:27:28.620737+0000","last_became_active":"2024-05-27T09:27:17.565488+0000","last_became_peered":"2024-05-27T09:27:17.565488+0000","last_unstale":"2024-05-27T09:27:28.620737+0000","last_undegraded":"2024-05-27T09:27:28.620737+0000","last_fullsized":"2024-05-27T09:27:28.620737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:53.711508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836966+0000","last_change":"2024-05-27T09:27:18.585592+0000","last_active":"2024-05-27T09:27:20.836966+0000","last_peered":"2024-05-27T09:27:20.836966+0000","last_clean":"2024-05-27T09:27:20.836966+0000","last_became_active":"2024-05-27T09:27:18.585243+0000","last_became_peered":"2024-05-27T09:27:18.585243+0000","last_unstale":"2024-05-27T09:27:20.836966+0000","last_undegraded":"2024-05-27T09:27:20.836966+0000","last_fullsized":"2024-05-27T09:27:20.836966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:28.872653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.548249+0000","last_change":"2024-05-27T09:27:17.548651+0000","last_active":"2024-05-27T09:27:23.548249+0000","last_peered":"2024-05-27T09:27:23.548249+0000","last_clean":"2024-05-27T09:27:23.548249+0000","last_became_active":"2024-05-27T09:27:17.548307+0000","last_became_peered":"2024-05-27T09:27:17.548307+0000","last_unstale":"2024-05-27T09:27:23.548249+0000","last_undegraded":"2024-05-27T09:27:23.548249+0000","last_fullsized":"2024-05-27T09:27:23.548249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:48.905422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837273+0000","last_change":"2024-05-27T09:27:18.582895+0000","last_active":"2024-05-27T09:27:20.837273+0000","last_peered":"2024-05-27T09:27:20.837273+0000","last_clean":"2024-05-27T09:27:20.837273+0000","last_became_active":"2024-05-27T09:27:18.581742+0000","last_became_peered":"2024-05-27T09:27:18.581742+0000","last_unstale":"2024-05-27T09:27:20.837273+0000","last_undegraded":"2024-05-27T09:27:20.837273+0000","last_fullsized":"2024-05-27T09:27:20.837273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:23:26.888905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.625723+0000","last_change":"2024-05-27T09:27:17.556419+0000","last_active":"2024-05-27T09:27:28.625723+0000","last_peered":"2024-05-27T09:27:28.625723+0000","last_clean":"2024-05-27T09:27:28.625723+0000","last_became_active":"2024-05-27T09:27:17.556036+0000","last_became_peered":"2024-05-27T09:27:17.556036+0000","last_unstale":"2024-05-27T09:27:28.625723+0000","last_undegraded":"2024-05-27T09:27:28.625723+0000","last_fullsized":"2024-05-27T09:27:28.625723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:48:20.718085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544281+0000","last_change":"2024-05-27T09:27:18.568187+0000","last_active":"2024-05-27T09:27:19.544281+0000","last_peered":"2024-05-27T09:27:19.544281+0000","last_clean":"2024-05-27T09:27:19.544281+0000","last_became_active":"2024-05-27T09:27:18.565500+0000","last_became_peered":"2024-05-27T09:27:18.565500+0000","last_unstale":"2024-05-27T09:27:19.544281+0000","last_undegraded":"2024-05-27T09:27:19.544281+0000","last_fullsized":"2024-05-27T09:27:19.544281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:16:08.688048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.734325+0000","last_change":"2024-05-27T09:27:17.572168+0000","last_active":"2024-05-27T09:27:33.734325+0000","last_peered":"2024-05-27T09:27:33.734325+0000","last_clean":"2024-05-27T09:27:33.734325+0000","last_became_active":"2024-05-27T09:27:17.571709+0000","last_became_peered":"2024-05-27T09:27:17.571709+0000","last_unstale":"2024-05-27T09:27:33.734325+0000","last_undegraded":"2024-05-27T09:27:33.734325+0000","last_fullsized":"2024-05-27T09:27:33.734325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836574+0000","last_change":"2024-05-27T09:27:18.564930+0000","last_active":"2024-05-27T09:27:20.836574+0000","last_peered":"2024-05-27T09:27:20.836574+0000","last_clean":"2024-05-27T09:27:20.836574+0000","last_became_active":"2024-05-27T09:27:18.564589+0000","last_became_peered":"2024-05-27T09:27:18.564589+0000","last_unstale":"2024-05-27T09:27:20.836574+0000","last_undegraded":"2024-05-27T09:27:20.836574+0000","last_fullsized":"2024-05-27T09:27:20.836574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:21:00.644551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.650158+0000","last_change":"2024-05-27T09:27:17.561344+0000","last_active":"2024-05-27T09:27:28.650158+0000","last_peered":"2024-05-27T09:27:28.650158+0000","last_clean":"2024-05-27T09:27:28.650158+0000","last_became_active":"2024-05-27T09:27:17.554860+0000","last_became_peered":"2024-05-27T09:27:17.554860+0000","last_unstale":"2024-05-27T09:27:28.650158+0000","last_undegraded":"2024-05-27T09:27:28.650158+0000","last_fullsized":"2024-05-27T09:27:28.650158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:16.880101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837236+0000","last_change":"2024-05-27T09:27:18.575317+0000","last_active":"2024-05-27T09:27:20.837236+0000","last_peered":"2024-05-27T09:27:20.837236+0000","last_clean":"2024-05-27T09:27:20.837236+0000","last_became_active":"2024-05-27T09:27:18.572024+0000","last_became_peered":"2024-05-27T09:27:18.572024+0000","last_unstale":"2024-05-27T09:27:20.837236+0000","last_undegraded":"2024-05-27T09:27:20.837236+0000","last_fullsized":"2024-05-27T09:27:20.837236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:23:35.418697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.739197+0000","last_change":"2024-05-27T09:27:17.575859+0000","last_active":"2024-05-27T09:27:33.739197+0000","last_peered":"2024-05-27T09:27:33.739197+0000","last_clean":"2024-05-27T09:27:33.739197+0000","last_became_active":"2024-05-27T09:27:17.574695+0000","last_became_peered":"2024-05-27T09:27:17.574695+0000","last_unstale":"2024-05-27T09:27:33.739197+0000","last_undegraded":"2024-05-27T09:27:33.739197+0000","last_fullsized":"2024-05-27T09:27:33.739197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:24:19.379819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539060+0000","last_change":"2024-05-27T09:27:18.555137+0000","last_active":"2024-05-27T09:27:19.539060+0000","last_peered":"2024-05-27T09:27:19.539060+0000","last_clean":"2024-05-27T09:27:19.539060+0000","last_became_active":"2024-05-27T09:27:18.554932+0000","last_became_peered":"2024-05-27T09:27:18.554932+0000","last_unstale":"2024-05-27T09:27:19.539060+0000","last_undegraded":"2024-05-27T09:27:19.539060+0000","last_fullsized":"2024-05-27T09:27:19.539060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:06.224637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'15","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.739565+0000","last_change":"2024-05-27T09:27:17.547895+0000","last_active":"2024-05-27T09:27:33.739565+0000","last_peered":"2024-05-27T09:27:33.739565+0000","last_clean":"2024-05-27T09:27:33.739565+0000","last_became_active":"2024-05-27T09:27:17.547523+0000","last_became_peered":"2024-05-27T09:27:17.547523+0000","last_unstale":"2024-05-27T09:27:33.739565+0000","last_undegraded":"2024-05-27T09:27:33.739565+0000","last_fullsized":"2024-05-27T09:27:33.739565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836706+0000","last_change":"2024-05-27T09:27:18.584284+0000","last_active":"2024-05-27T09:27:20.836706+0000","last_peered":"2024-05-27T09:27:20.836706+0000","last_clean":"2024-05-27T09:27:20.836706+0000","last_became_active":"2024-05-27T09:27:18.582360+0000","last_became_peered":"2024-05-27T09:27:18.582360+0000","last_unstale":"2024-05-27T09:27:20.836706+0000","last_undegraded":"2024-05-27T09:27:20.836706+0000","last_fullsized":"2024-05-27T09:27:20.836706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:32.535070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.550902+0000","last_change":"2024-05-27T09:27:17.571481+0000","last_active":"2024-05-27T09:27:23.550902+0000","last_peered":"2024-05-27T09:27:23.550902+0000","last_clean":"2024-05-27T09:27:23.550902+0000","last_became_active":"2024-05-27T09:27:17.571146+0000","last_became_peered":"2024-05-27T09:27:17.571146+0000","last_unstale":"2024-05-27T09:27:23.550902+0000","last_undegraded":"2024-05-27T09:27:23.550902+0000","last_fullsized":"2024-05-27T09:27:23.550902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:18.038221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542481+0000","last_change":"2024-05-27T09:27:18.580425+0000","last_active":"2024-05-27T09:27:19.542481+0000","last_peered":"2024-05-27T09:27:19.542481+0000","last_clean":"2024-05-27T09:27:19.542481+0000","last_became_active":"2024-05-27T09:27:18.579440+0000","last_became_peered":"2024-05-27T09:27:18.579440+0000","last_unstale":"2024-05-27T09:27:19.542481+0000","last_undegraded":"2024-05-27T09:27:19.542481+0000","last_fullsized":"2024-05-27T09:27:19.542481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:19.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.554242+0000","last_change":"2024-05-27T09:27:17.551295+0000","last_active":"2024-05-27T09:27:23.554242+0000","last_peered":"2024-05-27T09:27:23.554242+0000","last_clean":"2024-05-27T09:27:23.554242+0000","last_became_active":"2024-05-27T09:27:17.550861+0000","last_became_peered":"2024-05-27T09:27:17.550861+0000","last_unstale":"2024-05-27T09:27:23.554242+0000","last_undegraded":"2024-05-27T09:27:23.554242+0000","last_fullsized":"2024-05-27T09:27:23.554242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:13.955191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.545103+0000","last_change":"2024-05-27T09:27:18.547199+0000","last_active":"2024-05-27T09:27:19.545103+0000","last_peered":"2024-05-27T09:27:19.545103+0000","last_clean":"2024-05-27T09:27:19.545103+0000","last_became_active":"2024-05-27T09:27:18.543683+0000","last_became_peered":"2024-05-27T09:27:18.543683+0000","last_unstale":"2024-05-27T09:27:19.545103+0000","last_undegraded":"2024-05-27T09:27:19.545103+0000","last_fullsized":"2024-05-27T09:27:19.545103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:48:19.573654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:21.370295+0000","last_change":"2024-05-27T09:27:17.559255+0000","last_active":"2024-05-27T09:27:21.370295+0000","last_peered":"2024-05-27T09:27:21.370295+0000","last_clean":"2024-05-27T09:27:21.370295+0000","last_became_active":"2024-05-27T09:27:17.558973+0000","last_became_peered":"2024-05-27T09:27:17.558973+0000","last_unstale":"2024-05-27T09:27:21.370295+0000","last_undegraded":"2024-05-27T09:27:21.370295+0000","last_fullsized":"2024-05-27T09:27:21.370295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544602+0000","last_change":"2024-05-27T09:27:18.557423+0000","last_active":"2024-05-27T09:27:19.544602+0000","last_peered":"2024-05-27T09:27:19.544602+0000","last_clean":"2024-05-27T09:27:19.544602+0000","last_became_active":"2024-05-27T09:27:18.557076+0000","last_became_peered":"2024-05-27T09:27:18.557076+0000","last_unstale":"2024-05-27T09:27:19.544602+0000","last_undegraded":"2024-05-27T09:27:19.544602+0000","last_fullsized":"2024-05-27T09:27:19.544602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:11:04.612363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.738040+0000","last_change":"2024-05-27T09:27:17.573700+0000","last_active":"2024-05-27T09:27:33.738040+0000","last_peered":"2024-05-27T09:27:33.738040+0000","last_clean":"2024-05-27T09:27:33.738040+0000","last_became_active":"2024-05-27T09:27:17.573484+0000","last_became_peered":"2024-05-27T09:27:17.573484+0000","last_unstale":"2024-05-27T09:27:33.738040+0000","last_undegraded":"2024-05-27T09:27:33.738040+0000","last_fullsized":"2024-05-27T09:27:33.738040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:48.905422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837609+0000","last_change":"2024-05-27T09:27:18.582777+0000","last_active":"2024-05-27T09:27:20.837609+0000","last_peered":"2024-05-27T09:27:20.837609+0000","last_clean":"2024-05-27T09:27:20.837609+0000","last_became_active":"2024-05-27T09:27:18.574959+0000","last_became_peered":"2024-05-27T09:27:18.574959+0000","last_unstale":"2024-05-27T09:27:20.837609+0000","last_undegraded":"2024-05-27T09:27:20.837609+0000","last_fullsized":"2024-05-27T09:27:20.837609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:40:41.206755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":110,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539210+0000","last_change":"2024-05-27T09:27:12.693423+0000","last_active":"2024-05-27T09:27:19.539210+0000","last_peered":"2024-05-27T09:27:19.539210+0000","last_clean":"2024-05-27T09:27:19.539210+0000","last_became_active":"2024-05-27T09:27:12.690301+0000","last_became_peered":"2024-05-27T09:27:12.690301+0000","last_unstale":"2024-05-27T09:27:19.539210+0000","last_undegraded":"2024-05-27T09:27:19.539210+0000","last_fullsized":"2024-05-27T09:27:19.539210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:11.463226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:11.463226+0000","last_clean_scrub_stamp":"2024-05-27T09:27:11.463226+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:01.810071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.650695+0000","last_change":"2024-05-27T09:27:17.567670+0000","last_active":"2024-05-27T09:27:28.650695+0000","last_peered":"2024-05-27T09:27:28.650695+0000","last_clean":"2024-05-27T09:27:28.650695+0000","last_became_active":"2024-05-27T09:27:17.566292+0000","last_became_peered":"2024-05-27T09:27:17.566292+0000","last_unstale":"2024-05-27T09:27:28.650695+0000","last_undegraded":"2024-05-27T09:27:28.650695+0000","last_fullsized":"2024-05-27T09:27:28.650695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:18.038221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836742+0000","last_change":"2024-05-27T09:27:18.571124+0000","last_active":"2024-05-27T09:27:20.836742+0000","last_peered":"2024-05-27T09:27:20.836742+0000","last_clean":"2024-05-27T09:27:20.836742+0000","last_became_active":"2024-05-27T09:27:18.570782+0000","last_became_peered":"2024-05-27T09:27:18.570782+0000","last_unstale":"2024-05-27T09:27:20.836742+0000","last_undegraded":"2024-05-27T09:27:20.836742+0000","last_fullsized":"2024-05-27T09:27:20.836742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:39:58.110527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.734194+0000","last_change":"2024-05-27T09:27:17.563259+0000","last_active":"2024-05-27T09:27:33.734194+0000","last_peered":"2024-05-27T09:27:33.734194+0000","last_clean":"2024-05-27T09:27:33.734194+0000","last_became_active":"2024-05-27T09:27:17.563010+0000","last_became_peered":"2024-05-27T09:27:17.563010+0000","last_unstale":"2024-05-27T09:27:33.734194+0000","last_undegraded":"2024-05-27T09:27:33.734194+0000","last_fullsized":"2024-05-27T09:27:33.734194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:53.711508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836371+0000","last_change":"2024-05-27T09:27:18.574466+0000","last_active":"2024-05-27T09:27:20.836371+0000","last_peered":"2024-05-27T09:27:20.836371+0000","last_clean":"2024-05-27T09:27:20.836371+0000","last_became_active":"2024-05-27T09:27:18.574090+0000","last_became_peered":"2024-05-27T09:27:18.574090+0000","last_unstale":"2024-05-27T09:27:20.836371+0000","last_undegraded":"2024-05-27T09:27:20.836371+0000","last_fullsized":"2024-05-27T09:27:20.836371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:41:22.515484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.737344+0000","last_change":"2024-05-27T09:27:17.551310+0000","last_active":"2024-05-27T09:27:33.737344+0000","last_peered":"2024-05-27T09:27:33.737344+0000","last_clean":"2024-05-27T09:27:33.737344+0000","last_became_active":"2024-05-27T09:27:17.550982+0000","last_became_peered":"2024-05-27T09:27:17.550982+0000","last_unstale":"2024-05-27T09:27:33.737344+0000","last_undegraded":"2024-05-27T09:27:33.737344+0000","last_fullsized":"2024-05-27T09:27:33.737344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836574+0000","last_change":"2024-05-27T09:27:18.559416+0000","last_active":"2024-05-27T09:27:20.836574+0000","last_peered":"2024-05-27T09:27:20.836574+0000","last_clean":"2024-05-27T09:27:20.836574+0000","last_became_active":"2024-05-27T09:27:18.559017+0000","last_became_peered":"2024-05-27T09:27:18.559017+0000","last_unstale":"2024-05-27T09:27:20.836574+0000","last_undegraded":"2024-05-27T09:27:20.836574+0000","last_fullsized":"2024-05-27T09:27:20.836574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:33:48.400306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837736+0000","last_change":"2024-05-27T09:27:17.563947+0000","last_active":"2024-05-27T09:27:20.837736+0000","last_peered":"2024-05-27T09:27:20.837736+0000","last_clean":"2024-05-27T09:27:20.837736+0000","last_became_active":"2024-05-27T09:27:17.563640+0000","last_became_peered":"2024-05-27T09:27:17.563640+0000","last_unstale":"2024-05-27T09:27:20.837736+0000","last_undegraded":"2024-05-27T09:27:20.837736+0000","last_fullsized":"2024-05-27T09:27:20.837736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:02.747708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543615+0000","last_change":"2024-05-27T09:27:18.575835+0000","last_active":"2024-05-27T09:27:19.543615+0000","last_peered":"2024-05-27T09:27:19.543615+0000","last_clean":"2024-05-27T09:27:19.543615+0000","last_became_active":"2024-05-27T09:27:18.574710+0000","last_became_peered":"2024-05-27T09:27:18.574710+0000","last_unstale":"2024-05-27T09:27:19.543615+0000","last_undegraded":"2024-05-27T09:27:19.543615+0000","last_fullsized":"2024-05-27T09:27:19.543615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:16:00.934068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835781+0000","last_change":"2024-05-27T09:27:17.550749+0000","last_active":"2024-05-27T09:27:20.835781+0000","last_peered":"2024-05-27T09:27:20.835781+0000","last_clean":"2024-05-27T09:27:20.835781+0000","last_became_active":"2024-05-27T09:27:17.550378+0000","last_became_peered":"2024-05-27T09:27:17.550378+0000","last_unstale":"2024-05-27T09:27:20.835781+0000","last_undegraded":"2024-05-27T09:27:20.835781+0000","last_fullsized":"2024-05-27T09:27:20.835781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:20.167629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.541045+0000","last_change":"2024-05-27T09:27:18.567233+0000","last_active":"2024-05-27T09:27:19.541045+0000","last_peered":"2024-05-27T09:27:19.541045+0000","last_clean":"2024-05-27T09:27:19.541045+0000","last_became_active":"2024-05-27T09:27:18.566326+0000","last_became_peered":"2024-05-27T09:27:18.566326+0000","last_unstale":"2024-05-27T09:27:19.541045+0000","last_undegraded":"2024-05-27T09:27:19.541045+0000","last_fullsized":"2024-05-27T09:27:19.541045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:27.225150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837888+0000","last_change":"2024-05-27T09:27:17.557912+0000","last_active":"2024-05-27T09:27:20.837888+0000","last_peered":"2024-05-27T09:27:20.837888+0000","last_clean":"2024-05-27T09:27:20.837888+0000","last_became_active":"2024-05-27T09:27:17.557216+0000","last_became_peered":"2024-05-27T09:27:17.557216+0000","last_unstale":"2024-05-27T09:27:20.837888+0000","last_undegraded":"2024-05-27T09:27:20.837888+0000","last_fullsized":"2024-05-27T09:27:20.837888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:21:22.020724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836847+0000","last_change":"2024-05-27T09:27:18.563649+0000","last_active":"2024-05-27T09:27:20.836847+0000","last_peered":"2024-05-27T09:27:20.836847+0000","last_clean":"2024-05-27T09:27:20.836847+0000","last_became_active":"2024-05-27T09:27:18.562565+0000","last_became_peered":"2024-05-27T09:27:18.562565+0000","last_unstale":"2024-05-27T09:27:20.836847+0000","last_undegraded":"2024-05-27T09:27:20.836847+0000","last_fullsized":"2024-05-27T09:27:20.836847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:46:49.444709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.734132+0000","last_change":"2024-05-27T09:27:17.571911+0000","last_active":"2024-05-27T09:27:33.734132+0000","last_peered":"2024-05-27T09:27:33.734132+0000","last_clean":"2024-05-27T09:27:33.734132+0000","last_became_active":"2024-05-27T09:27:17.571551+0000","last_became_peered":"2024-05-27T09:27:17.571551+0000","last_unstale":"2024-05-27T09:27:33.734132+0000","last_undegraded":"2024-05-27T09:27:33.734132+0000","last_fullsized":"2024-05-27T09:27:33.734132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:20.167629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835899+0000","last_change":"2024-05-27T09:27:18.558780+0000","last_active":"2024-05-27T09:27:20.835899+0000","last_peered":"2024-05-27T09:27:20.835899+0000","last_clean":"2024-05-27T09:27:20.835899+0000","last_became_active":"2024-05-27T09:27:18.558463+0000","last_became_peered":"2024-05-27T09:27:18.558463+0000","last_unstale":"2024-05-27T09:27:20.835899+0000","last_undegraded":"2024-05-27T09:27:20.835899+0000","last_fullsized":"2024-05-27T09:27:20.835899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:42:38.351682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.646698+0000","last_change":"2024-05-27T09:27:17.567973+0000","last_active":"2024-05-27T09:27:28.646698+0000","last_peered":"2024-05-27T09:27:28.646698+0000","last_clean":"2024-05-27T09:27:28.646698+0000","last_became_active":"2024-05-27T09:27:17.567650+0000","last_became_peered":"2024-05-27T09:27:17.567650+0000","last_unstale":"2024-05-27T09:27:28.646698+0000","last_undegraded":"2024-05-27T09:27:28.646698+0000","last_fullsized":"2024-05-27T09:27:28.646698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:25:52.924185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835408+0000","last_change":"2024-05-27T09:27:18.554845+0000","last_active":"2024-05-27T09:27:20.835408+0000","last_peered":"2024-05-27T09:27:20.835408+0000","last_clean":"2024-05-27T09:27:20.835408+0000","last_became_active":"2024-05-27T09:27:18.554462+0000","last_became_peered":"2024-05-27T09:27:18.554462+0000","last_unstale":"2024-05-27T09:27:20.835408+0000","last_undegraded":"2024-05-27T09:27:20.835408+0000","last_fullsized":"2024-05-27T09:27:20.835408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:44:47.447245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.625211+0000","last_change":"2024-05-27T09:27:17.570300+0000","last_active":"2024-05-27T09:27:28.625211+0000","last_peered":"2024-05-27T09:27:28.625211+0000","last_clean":"2024-05-27T09:27:28.625211+0000","last_became_active":"2024-05-27T09:27:17.569685+0000","last_became_peered":"2024-05-27T09:27:17.569685+0000","last_unstale":"2024-05-27T09:27:28.625211+0000","last_undegraded":"2024-05-27T09:27:28.625211+0000","last_fullsized":"2024-05-27T09:27:28.625211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:26.210041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836430+0000","last_change":"2024-05-27T09:27:18.562774+0000","last_active":"2024-05-27T09:27:20.836430+0000","last_peered":"2024-05-27T09:27:20.836430+0000","last_clean":"2024-05-27T09:27:20.836430+0000","last_became_active":"2024-05-27T09:27:18.562455+0000","last_became_peered":"2024-05-27T09:27:18.562455+0000","last_unstale":"2024-05-27T09:27:20.836430+0000","last_undegraded":"2024-05-27T09:27:20.836430+0000","last_fullsized":"2024-05-27T09:27:20.836430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:39:59.034873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.627233+0000","last_change":"2024-05-27T09:27:17.563172+0000","last_active":"2024-05-27T09:27:28.627233+0000","last_peered":"2024-05-27T09:27:28.627233+0000","last_clean":"2024-05-27T09:27:28.627233+0000","last_became_active":"2024-05-27T09:27:17.562941+0000","last_became_peered":"2024-05-27T09:27:17.562941+0000","last_unstale":"2024-05-27T09:27:28.627233+0000","last_undegraded":"2024-05-27T09:27:28.627233+0000","last_fullsized":"2024-05-27T09:27:28.627233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:51.742078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836477+0000","last_change":"2024-05-27T09:27:18.555617+0000","last_active":"2024-05-27T09:27:20.836477+0000","last_peered":"2024-05-27T09:27:20.836477+0000","last_clean":"2024-05-27T09:27:20.836477+0000","last_became_active":"2024-05-27T09:27:18.555280+0000","last_became_peered":"2024-05-27T09:27:18.555280+0000","last_unstale":"2024-05-27T09:27:20.836477+0000","last_undegraded":"2024-05-27T09:27:20.836477+0000","last_fullsized":"2024-05-27T09:27:20.836477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:32:39.043268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:21.367453+0000","last_change":"2024-05-27T09:27:17.548266+0000","last_active":"2024-05-27T09:27:21.367453+0000","last_peered":"2024-05-27T09:27:21.367453+0000","last_clean":"2024-05-27T09:27:21.367453+0000","last_became_active":"2024-05-27T09:27:17.547938+0000","last_became_peered":"2024-05-27T09:27:17.547938+0000","last_unstale":"2024-05-27T09:27:21.367453+0000","last_undegraded":"2024-05-27T09:27:21.367453+0000","last_fullsized":"2024-05-27T09:27:21.367453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835677+0000","last_change":"2024-05-27T09:27:18.563586+0000","last_active":"2024-05-27T09:27:20.835677+0000","last_peered":"2024-05-27T09:27:20.835677+0000","last_clean":"2024-05-27T09:27:20.835677+0000","last_became_active":"2024-05-27T09:27:18.563227+0000","last_became_peered":"2024-05-27T09:27:18.563227+0000","last_unstale":"2024-05-27T09:27:20.835677+0000","last_undegraded":"2024-05-27T09:27:20.835677+0000","last_fullsized":"2024-05-27T09:27:20.835677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:53.222040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.647667+0000","last_change":"2024-05-27T09:27:17.562200+0000","last_active":"2024-05-27T09:27:28.647667+0000","last_peered":"2024-05-27T09:27:28.647667+0000","last_clean":"2024-05-27T09:27:28.647667+0000","last_became_active":"2024-05-27T09:27:17.561857+0000","last_became_peered":"2024-05-27T09:27:17.561857+0000","last_unstale":"2024-05-27T09:27:28.647667+0000","last_undegraded":"2024-05-27T09:27:28.647667+0000","last_fullsized":"2024-05-27T09:27:28.647667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:05.209528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836936+0000","last_change":"2024-05-27T09:27:18.572676+0000","last_active":"2024-05-27T09:27:20.836936+0000","last_peered":"2024-05-27T09:27:20.836936+0000","last_clean":"2024-05-27T09:27:20.836936+0000","last_became_active":"2024-05-27T09:27:18.572339+0000","last_became_peered":"2024-05-27T09:27:18.572339+0000","last_unstale":"2024-05-27T09:27:20.836936+0000","last_undegraded":"2024-05-27T09:27:20.836936+0000","last_fullsized":"2024-05-27T09:27:20.836936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:09:23.795528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837085+0000","last_change":"2024-05-27T09:27:18.575394+0000","last_active":"2024-05-27T09:27:20.837085+0000","last_peered":"2024-05-27T09:27:20.837085+0000","last_clean":"2024-05-27T09:27:20.837085+0000","last_became_active":"2024-05-27T09:27:18.572307+0000","last_became_peered":"2024-05-27T09:27:18.572307+0000","last_unstale":"2024-05-27T09:27:20.837085+0000","last_undegraded":"2024-05-27T09:27:20.837085+0000","last_fullsized":"2024-05-27T09:27:20.837085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:29:05.116582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.555603+0000","last_change":"2024-05-27T09:27:17.570634+0000","last_active":"2024-05-27T09:27:23.555603+0000","last_peered":"2024-05-27T09:27:23.555603+0000","last_clean":"2024-05-27T09:27:23.555603+0000","last_became_active":"2024-05-27T09:27:17.569866+0000","last_became_peered":"2024-05-27T09:27:17.569866+0000","last_unstale":"2024-05-27T09:27:23.555603+0000","last_undegraded":"2024-05-27T09:27:23.555603+0000","last_fullsized":"2024-05-27T09:27:23.555603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:21:38.268535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539980+0000","last_change":"2024-05-27T09:27:18.549339+0000","last_active":"2024-05-27T09:27:19.539980+0000","last_peered":"2024-05-27T09:27:19.539980+0000","last_clean":"2024-05-27T09:27:19.539980+0000","last_became_active":"2024-05-27T09:27:18.549035+0000","last_became_peered":"2024-05-27T09:27:18.549035+0000","last_unstale":"2024-05-27T09:27:19.539980+0000","last_undegraded":"2024-05-27T09:27:19.539980+0000","last_fullsized":"2024-05-27T09:27:19.539980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:17.895211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.737950+0000","last_change":"2024-05-27T09:27:17.568143+0000","last_active":"2024-05-27T09:27:33.737950+0000","last_peered":"2024-05-27T09:27:33.737950+0000","last_clean":"2024-05-27T09:27:33.737950+0000","last_became_active":"2024-05-27T09:27:17.567235+0000","last_became_peered":"2024-05-27T09:27:17.567235+0000","last_unstale":"2024-05-27T09:27:33.737950+0000","last_undegraded":"2024-05-27T09:27:33.737950+0000","last_fullsized":"2024-05-27T09:27:33.737950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.284270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836871+0000","last_change":"2024-05-27T09:27:18.563270+0000","last_active":"2024-05-27T09:27:20.836871+0000","last_peered":"2024-05-27T09:27:20.836871+0000","last_clean":"2024-05-27T09:27:20.836871+0000","last_became_active":"2024-05-27T09:27:18.562919+0000","last_became_peered":"2024-05-27T09:27:18.562919+0000","last_unstale":"2024-05-27T09:27:20.836871+0000","last_undegraded":"2024-05-27T09:27:20.836871+0000","last_fullsized":"2024-05-27T09:27:20.836871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:55:47.799529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.554180+0000","last_change":"2024-05-27T09:27:17.551193+0000","last_active":"2024-05-27T09:27:23.554180+0000","last_peered":"2024-05-27T09:27:23.554180+0000","last_clean":"2024-05-27T09:27:23.554180+0000","last_became_active":"2024-05-27T09:27:17.550849+0000","last_became_peered":"2024-05-27T09:27:17.550849+0000","last_unstale":"2024-05-27T09:27:23.554180+0000","last_undegraded":"2024-05-27T09:27:23.554180+0000","last_fullsized":"2024-05-27T09:27:23.554180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:51.742078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836183+0000","last_change":"2024-05-27T09:27:18.579803+0000","last_active":"2024-05-27T09:27:20.836183+0000","last_peered":"2024-05-27T09:27:20.836183+0000","last_clean":"2024-05-27T09:27:20.836183+0000","last_became_active":"2024-05-27T09:27:18.579574+0000","last_became_peered":"2024-05-27T09:27:18.579574+0000","last_unstale":"2024-05-27T09:27:20.836183+0000","last_undegraded":"2024-05-27T09:27:20.836183+0000","last_fullsized":"2024-05-27T09:27:20.836183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:11:22.320778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'9","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.737701+0000","last_change":"2024-05-27T09:27:17.549711+0000","last_active":"2024-05-27T09:27:33.737701+0000","last_peered":"2024-05-27T09:27:33.737701+0000","last_clean":"2024-05-27T09:27:33.737701+0000","last_became_active":"2024-05-27T09:27:17.549291+0000","last_became_peered":"2024-05-27T09:27:17.549291+0000","last_unstale":"2024-05-27T09:27:33.737701+0000","last_undegraded":"2024-05-27T09:27:33.737701+0000","last_fullsized":"2024-05-27T09:27:33.737701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:16:22.057803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5476,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":11206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65078,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":225280,"data_stored":73901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":225280,"data_stored":74321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":663203,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":73991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542154,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":73583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":74133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542154,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":664101,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":76109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T09:27:47.504 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T09:27:47.527 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:27:47.554 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:27:47.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.797+0000 7ff8ccc00640 1 Processor -- start 2024-05-27T09:27:47.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.797+0000 7ff8ccc00640 1 -- start start 2024-05-27T09:27:47.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.799+0000 7ff8ccc00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff8c81593b0 0x7ff8c8142720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:47.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.799+0000 7ff8ccc00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8c80c2990 0x7ff8c80c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:47.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.799+0000 7ff8ccc00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8c80739a0 con 0x7ff8c8073260 2024-05-27T09:27:47.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.799+0000 7ff8ccc00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7ff8c8074d20 con 0x7ff8c81593b0 2024-05-27T09:27:47.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.799+0000 7ff8ccc00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7ff8c8073c80 con 0x7ff8c80c2990 2024-05-27T09:27:47.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.799+0000 7ff8c5e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff8c81593b0 0x7ff8c8142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:47.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.799+0000 7ff8c5e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff8c81593b0 0x7ff8c8142720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38020/0 (socket says 172.21.15.114:38020) 2024-05-27T09:27:47.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.799+0000 7ff8c5e00640 1 -- 172.21.15.114:0/3592199015 learned_addr learned my addr 172.21.15.114:0/3592199015 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:47.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.800+0000 7ff8c6800640 1 --2- 172.21.15.114:0/3592199015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8c80c2990 0x7ff8c80c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:47.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.800+0000 7ff8c5e00640 1 -- 172.21.15.114:0/3592199015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8c80c2990 msgr2=0x7ff8c80c2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:47.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.800+0000 7ff8c5e00640 1 --2- 172.21.15.114:0/3592199015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8c80c2990 0x7ff8c80c2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:47.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.800+0000 7ff8c4a00640 1 -- 172.21.15.114:0/3592199015 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3511369330 0 0) 0x7ff8c80739a0 con 0x7ff8c8073260 2024-05-27T09:27:47.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.800+0000 7ff8c5e00640 1 -- 172.21.15.114:0/3592199015 >> v1:172.21.15.163:6789/0 conn(0x7ff8c8073260 legacy=0x7ff8c815a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:47.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.800+0000 7ff8c5e00640 1 -- 172.21.15.114:0/3592199015 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c8073f60 con 0x7ff8c81593b0 2024-05-27T09:27:47.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.801+0000 7ff8c5e00640 1 --2- 172.21.15.114:0/3592199015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8c81593b0 0x7ff8c8142720 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009870 tx=0x7ff8b802c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598fc765f69a37f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:47.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.801+0000 7ff8c4a00640 1 -- 172.21.15.114:0/3592199015 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8038070 con 0x7ff8c81593b0 2024-05-27T09:27:47.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.801+0000 7ff8c4a00640 1 -- 172.21.15.114:0/3592199015 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b8031710 con 0x7ff8c81593b0 2024-05-27T09:27:47.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.801+0000 7ff8c4a00640 1 -- 172.21.15.114:0/3592199015 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8031ae0 con 0x7ff8c81593b0 2024-05-27T09:27:47.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.802+0000 7ff8ccc00640 1 -- 172.21.15.114:0/3592199015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8c81593b0 msgr2=0x7ff8c8142720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:47.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.802+0000 7ff8ccc00640 1 --2- 172.21.15.114:0/3592199015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8c81593b0 0x7ff8c8142720 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009870 tx=0x7ff8b802c510 comp rx=0 tx=0).stop 2024-05-27T09:27:47.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.802+0000 7ff8ccc00640 1 -- 172.21.15.114:0/3592199015 shutdown_connections 2024-05-27T09:27:47.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.802+0000 7ff8ccc00640 1 --2- 172.21.15.114:0/3592199015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8c80c2990 0x7ff8c80c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:47.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.802+0000 7ff8ccc00640 1 --2- 172.21.15.114:0/3592199015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8c81593b0 0x7ff8c8142720 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:47.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.802+0000 7ff8ccc00640 1 -- 172.21.15.114:0/3592199015 >> 172.21.15.114:0/3592199015 conn(0x7ff8c806f6e0 msgr2=0x7ff8c8070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:47.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.802+0000 7ff8ccc00640 1 -- 172.21.15.114:0/3592199015 shutdown_connections 2024-05-27T09:27:47.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.802+0000 7ff8ccc00640 1 -- 172.21.15.114:0/3592199015 wait complete. 2024-05-27T09:27:47.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.805+0000 7ff8ccc00640 1 Processor -- start 2024-05-27T09:27:47.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.805+0000 7ff8ccc00640 1 -- start start 2024-05-27T09:27:47.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.806+0000 7ff8ccc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8c8073260 0x7ff8c81f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:47.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.806+0000 7ff8ccc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8c80c2990 0x7ff8c81413f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:47.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.806+0000 7ff8ccc00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8c81593b0 0x7ff8c813d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:47.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.806+0000 7ff8ccc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c8074800 con 0x7ff8c8073260 2024-05-27T09:27:47.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.806+0000 7ff8ccc00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c8074520 con 0x7ff8c80c2990 2024-05-27T09:27:47.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.806+0000 7ff8ccc00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7ff8c8073670 con 0x7ff8c81593b0 2024-05-27T09:27:47.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.807+0000 7ff8c5e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8c8073260 0x7ff8c81f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:47.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.807+0000 7ff8c6800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8c81593b0 0x7ff8c813d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:47.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.807+0000 7ff8c5e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8c8073260 0x7ff8c81f2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38032/0 (socket says 172.21.15.114:38032) 2024-05-27T09:27:47.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.807+0000 7ff8c5400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8c80c2990 0x7ff8c81413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:47.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.807+0000 7ff8c5e00640 1 -- 172.21.15.114:0/2207765326 learned_addr learned my addr 172.21.15.114:0/2207765326 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:47.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.807+0000 7ff8c5e00640 1 -- 172.21.15.114:0/2207765326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8c81593b0 msgr2=0x7ff8c813d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:47.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.807+0000 7ff8c5e00640 1 --2- 172.21.15.114:0/2207765326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8c81593b0 0x7ff8c813d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:47.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.807+0000 7ff8c5e00640 1 -- 172.21.15.114:0/2207765326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8c80c2990 msgr2=0x7ff8c81413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:47.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.807+0000 7ff8c5e00640 1 --2- 172.21.15.114:0/2207765326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8c80c2990 0x7ff8c81413f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:47.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.807+0000 7ff8c5e00640 1 -- 172.21.15.114:0/2207765326 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c81f2fd0 con 0x7ff8c8073260 2024-05-27T09:27:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.807+0000 7ff8c5e00640 1 --2- 172.21.15.114:0/2207765326 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8c8073260 0x7ff8c81f2a90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009870 tx=0x7ff8b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.808+0000 7ff8be000640 1 -- 172.21.15.114:0/2207765326 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8038040 con 0x7ff8c8073260 2024-05-27T09:27:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.808+0000 7ff8be000640 1 -- 172.21.15.114:0/2207765326 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b8043070 con 0x7ff8c8073260 2024-05-27T09:27:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.808+0000 7ff8be000640 1 -- 172.21.15.114:0/2207765326 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8033a50 con 0x7ff8c8073260 2024-05-27T09:27:47.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.808+0000 7ff8ccc00640 1 -- 172.21.15.114:0/2207765326 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c81ecbe0 con 0x7ff8c8073260 2024-05-27T09:27:47.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.808+0000 7ff8ccc00640 1 -- 172.21.15.114:0/2207765326 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c815abd0 con 0x7ff8c8073260 2024-05-27T09:27:47.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.809+0000 7ff8ccc00640 1 -- 172.21.15.114:0/2207765326 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8c806a380 con 0x7ff8c8073260 2024-05-27T09:27:47.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.811+0000 7ff8be000640 1 -- 172.21.15.114:0/2207765326 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff8b8033bf0 con 0x7ff8c8073260 2024-05-27T09:27:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.813+0000 7ff8be000640 1 --2- 172.21.15.114:0/2207765326 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff8a80762d0 0x7ff8a80787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.813+0000 7ff8be000640 1 -- 172.21.15.114:0/2207765326 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff8b80b8080 con 0x7ff8c8073260 2024-05-27T09:27:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.813+0000 7ff8c5400640 1 --2- 172.21.15.114:0/2207765326 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff8a80762d0 0x7ff8a80787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:47.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.814+0000 7ff8c5400640 1 --2- 172.21.15.114:0/2207765326 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff8a80762d0 0x7ff8a80787e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff8c81ecb20 tx=0x7ff8b4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:47.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:47.818+0000 7ff8be000640 1 -- 172.21.15.114:0/2207765326 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff8b80812f0 con 0x7ff8c8073260 2024-05-27T09:27:48.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.028+0000 7ff8ccc00640 1 -- 172.21.15.114:0/2207765326 --> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff8c80c41f0 con 0x7ff8a80762d0 2024-05-27T09:27:48.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.035+0000 7ff8be000640 1 -- 172.21.15.114:0/2207765326 <== mgr.4108 v2:172.21.15.163:6832/1122018204 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251982 (secure 0 0 0) 0x7ff8c80c41f0 con 0x7ff8a80762d0 2024-05-27T09:27:48.041 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:48.042 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-05-27T09:27:48.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.040+0000 7ff8ccc00640 1 -- 172.21.15.114:0/2207765326 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff8a80762d0 msgr2=0x7ff8a80787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:48.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.040+0000 7ff8ccc00640 1 --2- 172.21.15.114:0/2207765326 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff8a80762d0 0x7ff8a80787e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff8c81ecb20 tx=0x7ff8b4009040 comp rx=0 tx=0).stop 2024-05-27T09:27:48.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.040+0000 7ff8ccc00640 1 -- 172.21.15.114:0/2207765326 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8c8073260 msgr2=0x7ff8c81f2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:48.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.040+0000 7ff8ccc00640 1 --2- 172.21.15.114:0/2207765326 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8c8073260 0x7ff8c81f2a90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009870 tx=0x7ff8b80046c0 comp rx=0 tx=0).stop 2024-05-27T09:27:48.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.040+0000 7ff8ccc00640 1 -- 172.21.15.114:0/2207765326 shutdown_connections 2024-05-27T09:27:48.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.040+0000 7ff8ccc00640 1 --2- 172.21.15.114:0/2207765326 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff8a80762d0 0x7ff8a80787e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:48.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.040+0000 7ff8ccc00640 1 --2- 172.21.15.114:0/2207765326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8c81593b0 0x7ff8c813d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:48.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.040+0000 7ff8ccc00640 1 --2- 172.21.15.114:0/2207765326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8c80c2990 0x7ff8c81413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:48.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.040+0000 7ff8ccc00640 1 --2- 172.21.15.114:0/2207765326 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8c8073260 0x7ff8c81f2a90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:48.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.040+0000 7ff8ccc00640 1 -- 172.21.15.114:0/2207765326 >> 172.21.15.114:0/2207765326 conn(0x7ff8c806f6e0 msgr2=0x7ff8c813f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:48.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.041+0000 7ff8ccc00640 1 -- 172.21.15.114:0/2207765326 shutdown_connections 2024-05-27T09:27:48.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.041+0000 7ff8ccc00640 1 -- 172.21.15.114:0/2207765326 wait complete. 2024-05-27T09:27:48.070 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-05-27T09:27:46.769776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595844,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":227,"num_write_kb":1667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":182,"ondisk_log_size":182,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218048,"kb_used_data":1920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756672,"statfs":{"total":773094113280,"available":772870832128,"internally_reserved":0,"allocated":1966080,"data_stored":1773342,"data_compressed":65078,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1660,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003132"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544950+0000","last_change":"2024-05-27T09:27:18.585655+0000","last_active":"2024-05-27T09:27:19.544950+0000","last_peered":"2024-05-27T09:27:19.544950+0000","last_clean":"2024-05-27T09:27:19.544950+0000","last_became_active":"2024-05-27T09:27:18.585169+0000","last_became_peered":"2024-05-27T09:27:18.585169+0000","last_unstale":"2024-05-27T09:27:19.544950+0000","last_undegraded":"2024-05-27T09:27:19.544950+0000","last_fullsized":"2024-05-27T09:27:19.544950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:39:59.034873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835983+0000","last_change":"2024-05-27T09:27:18.566621+0000","last_active":"2024-05-27T09:27:20.835983+0000","last_peered":"2024-05-27T09:27:20.835983+0000","last_clean":"2024-05-27T09:27:20.835983+0000","last_became_active":"2024-05-27T09:27:18.566269+0000","last_became_peered":"2024-05-27T09:27:18.566269+0000","last_unstale":"2024-05-27T09:27:20.835983+0000","last_undegraded":"2024-05-27T09:27:20.835983+0000","last_fullsized":"2024-05-27T09:27:20.835983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:24:24.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835531+0000","last_change":"2024-05-27T09:27:18.560958+0000","last_active":"2024-05-27T09:27:20.835531+0000","last_peered":"2024-05-27T09:27:20.835531+0000","last_clean":"2024-05-27T09:27:20.835531+0000","last_became_active":"2024-05-27T09:27:18.560620+0000","last_became_peered":"2024-05-27T09:27:18.560620+0000","last_unstale":"2024-05-27T09:27:20.835531+0000","last_undegraded":"2024-05-27T09:27:20.835531+0000","last_fullsized":"2024-05-27T09:27:20.835531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:42:38.351682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542664+0000","last_change":"2024-05-27T09:27:18.569804+0000","last_active":"2024-05-27T09:27:19.542664+0000","last_peered":"2024-05-27T09:27:19.542664+0000","last_clean":"2024-05-27T09:27:19.542664+0000","last_became_active":"2024-05-27T09:27:18.569481+0000","last_became_peered":"2024-05-27T09:27:18.569481+0000","last_unstale":"2024-05-27T09:27:19.542664+0000","last_undegraded":"2024-05-27T09:27:19.542664+0000","last_fullsized":"2024-05-27T09:27:19.542664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:55:03.583659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543808+0000","last_change":"2024-05-27T09:27:18.568138+0000","last_active":"2024-05-27T09:27:19.543808+0000","last_peered":"2024-05-27T09:27:19.543808+0000","last_clean":"2024-05-27T09:27:19.543808+0000","last_became_active":"2024-05-27T09:27:18.567795+0000","last_became_peered":"2024-05-27T09:27:18.567795+0000","last_unstale":"2024-05-27T09:27:19.543808+0000","last_undegraded":"2024-05-27T09:27:19.543808+0000","last_fullsized":"2024-05-27T09:27:19.543808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:17:42.005593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.540719+0000","last_change":"2024-05-27T09:27:18.573984+0000","last_active":"2024-05-27T09:27:19.540719+0000","last_peered":"2024-05-27T09:27:19.540719+0000","last_clean":"2024-05-27T09:27:19.540719+0000","last_became_active":"2024-05-27T09:27:18.573576+0000","last_became_peered":"2024-05-27T09:27:18.573576+0000","last_unstale":"2024-05-27T09:27:19.540719+0000","last_undegraded":"2024-05-27T09:27:19.540719+0000","last_fullsized":"2024-05-27T09:27:19.540719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:06.224637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543010+0000","last_change":"2024-05-27T09:27:18.543122+0000","last_active":"2024-05-27T09:27:19.543010+0000","last_peered":"2024-05-27T09:27:19.543010+0000","last_clean":"2024-05-27T09:27:19.543010+0000","last_became_active":"2024-05-27T09:27:18.542593+0000","last_became_peered":"2024-05-27T09:27:18.542593+0000","last_unstale":"2024-05-27T09:27:19.543010+0000","last_undegraded":"2024-05-27T09:27:19.543010+0000","last_fullsized":"2024-05-27T09:27:19.543010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:28:19.579432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835946+0000","last_change":"2024-05-27T09:27:18.562292+0000","last_active":"2024-05-27T09:27:20.835946+0000","last_peered":"2024-05-27T09:27:20.835946+0000","last_clean":"2024-05-27T09:27:20.835946+0000","last_became_active":"2024-05-27T09:27:18.561788+0000","last_became_peered":"2024-05-27T09:27:18.561788+0000","last_unstale":"2024-05-27T09:27:20.835946+0000","last_undegraded":"2024-05-27T09:27:20.835946+0000","last_fullsized":"2024-05-27T09:27:20.835946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:32:39.043268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542996+0000","last_change":"2024-05-27T09:27:18.583770+0000","last_active":"2024-05-27T09:27:19.542996+0000","last_peered":"2024-05-27T09:27:19.542996+0000","last_clean":"2024-05-27T09:27:19.542996+0000","last_became_active":"2024-05-27T09:27:18.582041+0000","last_became_peered":"2024-05-27T09:27:18.582041+0000","last_unstale":"2024-05-27T09:27:19.542996+0000","last_undegraded":"2024-05-27T09:27:19.542996+0000","last_fullsized":"2024-05-27T09:27:19.542996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:19.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.545125+0000","last_change":"2024-05-27T09:27:18.574658+0000","last_active":"2024-05-27T09:27:19.545125+0000","last_peered":"2024-05-27T09:27:19.545125+0000","last_clean":"2024-05-27T09:27:19.545125+0000","last_became_active":"2024-05-27T09:27:18.574409+0000","last_became_peered":"2024-05-27T09:27:18.574409+0000","last_unstale":"2024-05-27T09:27:19.545125+0000","last_undegraded":"2024-05-27T09:27:19.545125+0000","last_fullsized":"2024-05-27T09:27:19.545125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:19.131343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544558+0000","last_change":"2024-05-27T09:27:18.579141+0000","last_active":"2024-05-27T09:27:19.544558+0000","last_peered":"2024-05-27T09:27:19.544558+0000","last_clean":"2024-05-27T09:27:19.544558+0000","last_became_active":"2024-05-27T09:27:18.578679+0000","last_became_peered":"2024-05-27T09:27:18.578679+0000","last_unstale":"2024-05-27T09:27:19.544558+0000","last_undegraded":"2024-05-27T09:27:19.544558+0000","last_fullsized":"2024-05-27T09:27:19.544558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:44.336453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544110+0000","last_change":"2024-05-27T09:27:18.547375+0000","last_active":"2024-05-27T09:27:19.544110+0000","last_peered":"2024-05-27T09:27:19.544110+0000","last_clean":"2024-05-27T09:27:19.544110+0000","last_became_active":"2024-05-27T09:27:18.544328+0000","last_became_peered":"2024-05-27T09:27:18.544328+0000","last_unstale":"2024-05-27T09:27:19.544110+0000","last_undegraded":"2024-05-27T09:27:19.544110+0000","last_fullsized":"2024-05-27T09:27:19.544110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:59:50.499894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836590+0000","last_change":"2024-05-27T09:27:18.561903+0000","last_active":"2024-05-27T09:27:20.836590+0000","last_peered":"2024-05-27T09:27:20.836590+0000","last_clean":"2024-05-27T09:27:20.836590+0000","last_became_active":"2024-05-27T09:27:18.561493+0000","last_became_peered":"2024-05-27T09:27:18.561493+0000","last_unstale":"2024-05-27T09:27:20.836590+0000","last_undegraded":"2024-05-27T09:27:20.836590+0000","last_fullsized":"2024-05-27T09:27:20.836590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:28:31.470269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836378+0000","last_change":"2024-05-27T09:27:18.563168+0000","last_active":"2024-05-27T09:27:20.836378+0000","last_peered":"2024-05-27T09:27:20.836378+0000","last_clean":"2024-05-27T09:27:20.836378+0000","last_became_active":"2024-05-27T09:27:18.562843+0000","last_became_peered":"2024-05-27T09:27:18.562843+0000","last_unstale":"2024-05-27T09:27:20.836378+0000","last_undegraded":"2024-05-27T09:27:20.836378+0000","last_fullsized":"2024-05-27T09:27:20.836378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:33:36.671764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835457+0000","last_change":"2024-05-27T09:27:18.566502+0000","last_active":"2024-05-27T09:27:20.835457+0000","last_peered":"2024-05-27T09:27:20.835457+0000","last_clean":"2024-05-27T09:27:20.835457+0000","last_became_active":"2024-05-27T09:27:18.566108+0000","last_became_peered":"2024-05-27T09:27:18.566108+0000","last_unstale":"2024-05-27T09:27:20.835457+0000","last_undegraded":"2024-05-27T09:27:20.835457+0000","last_fullsized":"2024-05-27T09:27:20.835457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:44:47.447245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836471+0000","last_change":"2024-05-27T09:27:18.582595+0000","last_active":"2024-05-27T09:27:20.836471+0000","last_peered":"2024-05-27T09:27:20.836471+0000","last_clean":"2024-05-27T09:27:20.836471+0000","last_became_active":"2024-05-27T09:27:18.581719+0000","last_became_peered":"2024-05-27T09:27:18.581719+0000","last_unstale":"2024-05-27T09:27:20.836471+0000","last_undegraded":"2024-05-27T09:27:20.836471+0000","last_fullsized":"2024-05-27T09:27:20.836471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:48:19.573654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837234+0000","last_change":"2024-05-27T09:27:18.577565+0000","last_active":"2024-05-27T09:27:20.837234+0000","last_peered":"2024-05-27T09:27:20.837234+0000","last_clean":"2024-05-27T09:27:20.837234+0000","last_became_active":"2024-05-27T09:27:18.577193+0000","last_became_peered":"2024-05-27T09:27:18.577193+0000","last_unstale":"2024-05-27T09:27:20.837234+0000","last_undegraded":"2024-05-27T09:27:20.837234+0000","last_fullsized":"2024-05-27T09:27:20.837234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:43:20.065873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835530+0000","last_change":"2024-05-27T09:27:18.571089+0000","last_active":"2024-05-27T09:27:20.835530+0000","last_peered":"2024-05-27T09:27:20.835530+0000","last_clean":"2024-05-27T09:27:20.835530+0000","last_became_active":"2024-05-27T09:27:18.570712+0000","last_became_peered":"2024-05-27T09:27:18.570712+0000","last_unstale":"2024-05-27T09:27:20.835530+0000","last_undegraded":"2024-05-27T09:27:20.835530+0000","last_fullsized":"2024-05-27T09:27:20.835530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:40:19.310581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837956+0000","last_change":"2024-05-27T09:27:18.572675+0000","last_active":"2024-05-27T09:27:20.837956+0000","last_peered":"2024-05-27T09:27:20.837956+0000","last_clean":"2024-05-27T09:27:20.837956+0000","last_became_active":"2024-05-27T09:27:18.564643+0000","last_became_peered":"2024-05-27T09:27:18.564643+0000","last_unstale":"2024-05-27T09:27:20.837956+0000","last_undegraded":"2024-05-27T09:27:20.837956+0000","last_fullsized":"2024-05-27T09:27:20.837956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:36:15.602241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544179+0000","last_change":"2024-05-27T09:27:18.544554+0000","last_active":"2024-05-27T09:27:19.544179+0000","last_peered":"2024-05-27T09:27:19.544179+0000","last_clean":"2024-05-27T09:27:19.544179+0000","last_became_active":"2024-05-27T09:27:18.543777+0000","last_became_peered":"2024-05-27T09:27:18.543777+0000","last_unstale":"2024-05-27T09:27:19.544179+0000","last_undegraded":"2024-05-27T09:27:19.544179+0000","last_fullsized":"2024-05-27T09:27:19.544179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:35:43.354270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835033+0000","last_change":"2024-05-27T09:27:18.571321+0000","last_active":"2024-05-27T09:27:20.835033+0000","last_peered":"2024-05-27T09:27:20.835033+0000","last_clean":"2024-05-27T09:27:20.835033+0000","last_became_active":"2024-05-27T09:27:18.570948+0000","last_became_peered":"2024-05-27T09:27:18.570948+0000","last_unstale":"2024-05-27T09:27:20.835033+0000","last_undegraded":"2024-05-27T09:27:20.835033+0000","last_fullsized":"2024-05-27T09:27:20.835033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:28:31.470269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539490+0000","last_change":"2024-05-27T09:27:18.544282+0000","last_active":"2024-05-27T09:27:19.539490+0000","last_peered":"2024-05-27T09:27:19.539490+0000","last_clean":"2024-05-27T09:27:19.539490+0000","last_became_active":"2024-05-27T09:27:18.543921+0000","last_became_peered":"2024-05-27T09:27:18.543921+0000","last_unstale":"2024-05-27T09:27:19.539490+0000","last_undegraded":"2024-05-27T09:27:19.539490+0000","last_fullsized":"2024-05-27T09:27:19.539490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:27.225150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835732+0000","last_change":"2024-05-27T09:27:18.575415+0000","last_active":"2024-05-27T09:27:20.835732+0000","last_peered":"2024-05-27T09:27:20.835732+0000","last_clean":"2024-05-27T09:27:20.835732+0000","last_became_active":"2024-05-27T09:27:18.575115+0000","last_became_peered":"2024-05-27T09:27:18.575115+0000","last_unstale":"2024-05-27T09:27:20.835732+0000","last_undegraded":"2024-05-27T09:27:20.835732+0000","last_fullsized":"2024-05-27T09:27:20.835732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:50:48.091125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836496+0000","last_change":"2024-05-27T09:27:18.564422+0000","last_active":"2024-05-27T09:27:20.836496+0000","last_peered":"2024-05-27T09:27:20.836496+0000","last_clean":"2024-05-27T09:27:20.836496+0000","last_became_active":"2024-05-27T09:27:18.563991+0000","last_became_peered":"2024-05-27T09:27:18.563991+0000","last_unstale":"2024-05-27T09:27:20.836496+0000","last_undegraded":"2024-05-27T09:27:20.836496+0000","last_fullsized":"2024-05-27T09:27:20.836496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:19.131343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836032+0000","last_change":"2024-05-27T09:27:18.563557+0000","last_active":"2024-05-27T09:27:20.836032+0000","last_peered":"2024-05-27T09:27:20.836032+0000","last_clean":"2024-05-27T09:27:20.836032+0000","last_became_active":"2024-05-27T09:27:18.562803+0000","last_became_peered":"2024-05-27T09:27:18.562803+0000","last_unstale":"2024-05-27T09:27:20.836032+0000","last_undegraded":"2024-05-27T09:27:20.836032+0000","last_fullsized":"2024-05-27T09:27:20.836032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:47:54.271208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542223+0000","last_change":"2024-05-27T09:27:18.590430+0000","last_active":"2024-05-27T09:27:19.542223+0000","last_peered":"2024-05-27T09:27:19.542223+0000","last_clean":"2024-05-27T09:27:19.542223+0000","last_became_active":"2024-05-27T09:27:18.590083+0000","last_became_peered":"2024-05-27T09:27:18.590083+0000","last_unstale":"2024-05-27T09:27:19.542223+0000","last_undegraded":"2024-05-27T09:27:19.542223+0000","last_fullsized":"2024-05-27T09:27:19.542223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:03.762818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837167+0000","last_change":"2024-05-27T09:27:18.558248+0000","last_active":"2024-05-27T09:27:20.837167+0000","last_peered":"2024-05-27T09:27:20.837167+0000","last_clean":"2024-05-27T09:27:20.837167+0000","last_became_active":"2024-05-27T09:27:18.557953+0000","last_became_peered":"2024-05-27T09:27:18.557953+0000","last_unstale":"2024-05-27T09:27:20.837167+0000","last_undegraded":"2024-05-27T09:27:20.837167+0000","last_fullsized":"2024-05-27T09:27:20.837167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:44:47.447245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836514+0000","last_change":"2024-05-27T09:27:18.563213+0000","last_active":"2024-05-27T09:27:20.836514+0000","last_peered":"2024-05-27T09:27:20.836514+0000","last_clean":"2024-05-27T09:27:20.836514+0000","last_became_active":"2024-05-27T09:27:18.562839+0000","last_became_peered":"2024-05-27T09:27:18.562839+0000","last_unstale":"2024-05-27T09:27:20.836514+0000","last_undegraded":"2024-05-27T09:27:20.836514+0000","last_fullsized":"2024-05-27T09:27:20.836514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:43.851698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836053+0000","last_change":"2024-05-27T09:27:18.562237+0000","last_active":"2024-05-27T09:27:20.836053+0000","last_peered":"2024-05-27T09:27:20.836053+0000","last_clean":"2024-05-27T09:27:20.836053+0000","last_became_active":"2024-05-27T09:27:18.561475+0000","last_became_peered":"2024-05-27T09:27:18.561475+0000","last_unstale":"2024-05-27T09:27:20.836053+0000","last_undegraded":"2024-05-27T09:27:20.836053+0000","last_fullsized":"2024-05-27T09:27:20.836053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:39:58.110527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837611+0000","last_change":"2024-05-27T09:27:18.563469+0000","last_active":"2024-05-27T09:27:20.837611+0000","last_peered":"2024-05-27T09:27:20.837611+0000","last_clean":"2024-05-27T09:27:20.837611+0000","last_became_active":"2024-05-27T09:27:18.563103+0000","last_became_peered":"2024-05-27T09:27:18.563103+0000","last_unstale":"2024-05-27T09:27:20.837611+0000","last_undegraded":"2024-05-27T09:27:20.837611+0000","last_fullsized":"2024-05-27T09:27:20.837611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:53:57.281169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.834984+0000","last_change":"2024-05-27T09:27:18.560551+0000","last_active":"2024-05-27T09:27:20.834984+0000","last_peered":"2024-05-27T09:27:20.834984+0000","last_clean":"2024-05-27T09:27:20.834984+0000","last_became_active":"2024-05-27T09:27:18.560244+0000","last_became_peered":"2024-05-27T09:27:18.560244+0000","last_unstale":"2024-05-27T09:27:20.834984+0000","last_undegraded":"2024-05-27T09:27:20.834984+0000","last_fullsized":"2024-05-27T09:27:20.834984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:00.131180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.538536+0000","last_change":"2024-05-27T09:27:18.568200+0000","last_active":"2024-05-27T09:27:19.538536+0000","last_peered":"2024-05-27T09:27:19.538536+0000","last_clean":"2024-05-27T09:27:19.538536+0000","last_became_active":"2024-05-27T09:27:18.567845+0000","last_became_peered":"2024-05-27T09:27:18.567845+0000","last_unstale":"2024-05-27T09:27:19.538536+0000","last_undegraded":"2024-05-27T09:27:19.538536+0000","last_fullsized":"2024-05-27T09:27:19.538536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:54.726617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.540019+0000","last_change":"2024-05-27T09:27:18.560521+0000","last_active":"2024-05-27T09:27:19.540019+0000","last_peered":"2024-05-27T09:27:19.540019+0000","last_clean":"2024-05-27T09:27:19.540019+0000","last_became_active":"2024-05-27T09:27:18.556156+0000","last_became_peered":"2024-05-27T09:27:18.556156+0000","last_unstale":"2024-05-27T09:27:19.540019+0000","last_undegraded":"2024-05-27T09:27:19.540019+0000","last_fullsized":"2024-05-27T09:27:19.540019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:03.762818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.631378+0000","last_change":"2024-05-27T09:27:17.548038+0000","last_active":"2024-05-27T09:27:28.631378+0000","last_peered":"2024-05-27T09:27:28.631378+0000","last_clean":"2024-05-27T09:27:28.631378+0000","last_became_active":"2024-05-27T09:27:17.547653+0000","last_became_peered":"2024-05-27T09:27:17.547653+0000","last_unstale":"2024-05-27T09:27:28.631378+0000","last_undegraded":"2024-05-27T09:27:28.631378+0000","last_fullsized":"2024-05-27T09:27:28.631378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":876,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835265+0000","last_change":"2024-05-27T09:27:18.543048+0000","last_active":"2024-05-27T09:27:20.835265+0000","last_peered":"2024-05-27T09:27:20.835265+0000","last_clean":"2024-05-27T09:27:20.835265+0000","last_became_active":"2024-05-27T09:27:18.542715+0000","last_became_peered":"2024-05-27T09:27:18.542715+0000","last_unstale":"2024-05-27T09:27:20.835265+0000","last_undegraded":"2024-05-27T09:27:20.835265+0000","last_fullsized":"2024-05-27T09:27:20.835265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:46:49.444709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.738235+0000","last_change":"2024-05-27T09:27:17.570014+0000","last_active":"2024-05-27T09:27:33.738235+0000","last_peered":"2024-05-27T09:27:33.738235+0000","last_clean":"2024-05-27T09:27:33.738235+0000","last_became_active":"2024-05-27T09:27:17.569673+0000","last_became_peered":"2024-05-27T09:27:17.569673+0000","last_unstale":"2024-05-27T09:27:33.738235+0000","last_undegraded":"2024-05-27T09:27:33.738235+0000","last_fullsized":"2024-05-27T09:27:33.738235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:26.210041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.738367+0000","last_change":"2024-05-27T09:27:17.565489+0000","last_active":"2024-05-27T09:27:33.738367+0000","last_peered":"2024-05-27T09:27:33.738367+0000","last_clean":"2024-05-27T09:27:33.738367+0000","last_became_active":"2024-05-27T09:27:17.565137+0000","last_became_peered":"2024-05-27T09:27:17.565137+0000","last_unstale":"2024-05-27T09:27:33.738367+0000","last_undegraded":"2024-05-27T09:27:33.738367+0000","last_fullsized":"2024-05-27T09:27:33.738367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836835+0000","last_change":"2024-05-27T09:27:18.565294+0000","last_active":"2024-05-27T09:27:20.836835+0000","last_peered":"2024-05-27T09:27:20.836835+0000","last_clean":"2024-05-27T09:27:20.836835+0000","last_became_active":"2024-05-27T09:27:18.564115+0000","last_became_peered":"2024-05-27T09:27:18.564115+0000","last_unstale":"2024-05-27T09:27:20.836835+0000","last_undegraded":"2024-05-27T09:27:20.836835+0000","last_fullsized":"2024-05-27T09:27:20.836835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:57:20.488849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836816+0000","last_change":"2024-05-27T09:27:17.561204+0000","last_active":"2024-05-27T09:27:20.836816+0000","last_peered":"2024-05-27T09:27:20.836816+0000","last_clean":"2024-05-27T09:27:20.836816+0000","last_became_active":"2024-05-27T09:27:17.560149+0000","last_became_peered":"2024-05-27T09:27:17.560149+0000","last_unstale":"2024-05-27T09:27:20.836816+0000","last_undegraded":"2024-05-27T09:27:20.836816+0000","last_fullsized":"2024-05-27T09:27:20.836816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.284270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836343+0000","last_change":"2024-05-27T09:27:18.542204+0000","last_active":"2024-05-27T09:27:20.836343+0000","last_peered":"2024-05-27T09:27:20.836343+0000","last_clean":"2024-05-27T09:27:20.836343+0000","last_became_active":"2024-05-27T09:27:18.541946+0000","last_became_peered":"2024-05-27T09:27:18.541946+0000","last_unstale":"2024-05-27T09:27:20.836343+0000","last_undegraded":"2024-05-27T09:27:20.836343+0000","last_fullsized":"2024-05-27T09:27:20.836343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:50:48.091125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.639056+0000","last_change":"2024-05-27T09:27:17.561714+0000","last_active":"2024-05-27T09:27:28.639056+0000","last_peered":"2024-05-27T09:27:28.639056+0000","last_clean":"2024-05-27T09:27:28.639056+0000","last_became_active":"2024-05-27T09:27:17.561355+0000","last_became_peered":"2024-05-27T09:27:17.561355+0000","last_unstale":"2024-05-27T09:27:28.639056+0000","last_undegraded":"2024-05-27T09:27:28.639056+0000","last_fullsized":"2024-05-27T09:27:28.639056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:07:02.481451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542128+0000","last_change":"2024-05-27T09:27:18.588199+0000","last_active":"2024-05-27T09:27:19.542128+0000","last_peered":"2024-05-27T09:27:19.542128+0000","last_clean":"2024-05-27T09:27:19.542128+0000","last_became_active":"2024-05-27T09:27:18.587921+0000","last_became_peered":"2024-05-27T09:27:18.587921+0000","last_unstale":"2024-05-27T09:27:19.542128+0000","last_undegraded":"2024-05-27T09:27:19.542128+0000","last_fullsized":"2024-05-27T09:27:19.542128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:17.895211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.620737+0000","last_change":"2024-05-27T09:27:17.565854+0000","last_active":"2024-05-27T09:27:28.620737+0000","last_peered":"2024-05-27T09:27:28.620737+0000","last_clean":"2024-05-27T09:27:28.620737+0000","last_became_active":"2024-05-27T09:27:17.565488+0000","last_became_peered":"2024-05-27T09:27:17.565488+0000","last_unstale":"2024-05-27T09:27:28.620737+0000","last_undegraded":"2024-05-27T09:27:28.620737+0000","last_fullsized":"2024-05-27T09:27:28.620737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:53.711508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836966+0000","last_change":"2024-05-27T09:27:18.585592+0000","last_active":"2024-05-27T09:27:20.836966+0000","last_peered":"2024-05-27T09:27:20.836966+0000","last_clean":"2024-05-27T09:27:20.836966+0000","last_became_active":"2024-05-27T09:27:18.585243+0000","last_became_peered":"2024-05-27T09:27:18.585243+0000","last_unstale":"2024-05-27T09:27:20.836966+0000","last_undegraded":"2024-05-27T09:27:20.836966+0000","last_fullsized":"2024-05-27T09:27:20.836966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:28.872653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.548249+0000","last_change":"2024-05-27T09:27:17.548651+0000","last_active":"2024-05-27T09:27:23.548249+0000","last_peered":"2024-05-27T09:27:23.548249+0000","last_clean":"2024-05-27T09:27:23.548249+0000","last_became_active":"2024-05-27T09:27:17.548307+0000","last_became_peered":"2024-05-27T09:27:17.548307+0000","last_unstale":"2024-05-27T09:27:23.548249+0000","last_undegraded":"2024-05-27T09:27:23.548249+0000","last_fullsized":"2024-05-27T09:27:23.548249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:48.905422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837273+0000","last_change":"2024-05-27T09:27:18.582895+0000","last_active":"2024-05-27T09:27:20.837273+0000","last_peered":"2024-05-27T09:27:20.837273+0000","last_clean":"2024-05-27T09:27:20.837273+0000","last_became_active":"2024-05-27T09:27:18.581742+0000","last_became_peered":"2024-05-27T09:27:18.581742+0000","last_unstale":"2024-05-27T09:27:20.837273+0000","last_undegraded":"2024-05-27T09:27:20.837273+0000","last_fullsized":"2024-05-27T09:27:20.837273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:23:26.888905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.625723+0000","last_change":"2024-05-27T09:27:17.556419+0000","last_active":"2024-05-27T09:27:28.625723+0000","last_peered":"2024-05-27T09:27:28.625723+0000","last_clean":"2024-05-27T09:27:28.625723+0000","last_became_active":"2024-05-27T09:27:17.556036+0000","last_became_peered":"2024-05-27T09:27:17.556036+0000","last_unstale":"2024-05-27T09:27:28.625723+0000","last_undegraded":"2024-05-27T09:27:28.625723+0000","last_fullsized":"2024-05-27T09:27:28.625723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:48:20.718085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544281+0000","last_change":"2024-05-27T09:27:18.568187+0000","last_active":"2024-05-27T09:27:19.544281+0000","last_peered":"2024-05-27T09:27:19.544281+0000","last_clean":"2024-05-27T09:27:19.544281+0000","last_became_active":"2024-05-27T09:27:18.565500+0000","last_became_peered":"2024-05-27T09:27:18.565500+0000","last_unstale":"2024-05-27T09:27:19.544281+0000","last_undegraded":"2024-05-27T09:27:19.544281+0000","last_fullsized":"2024-05-27T09:27:19.544281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:16:08.688048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.734325+0000","last_change":"2024-05-27T09:27:17.572168+0000","last_active":"2024-05-27T09:27:33.734325+0000","last_peered":"2024-05-27T09:27:33.734325+0000","last_clean":"2024-05-27T09:27:33.734325+0000","last_became_active":"2024-05-27T09:27:17.571709+0000","last_became_peered":"2024-05-27T09:27:17.571709+0000","last_unstale":"2024-05-27T09:27:33.734325+0000","last_undegraded":"2024-05-27T09:27:33.734325+0000","last_fullsized":"2024-05-27T09:27:33.734325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836574+0000","last_change":"2024-05-27T09:27:18.564930+0000","last_active":"2024-05-27T09:27:20.836574+0000","last_peered":"2024-05-27T09:27:20.836574+0000","last_clean":"2024-05-27T09:27:20.836574+0000","last_became_active":"2024-05-27T09:27:18.564589+0000","last_became_peered":"2024-05-27T09:27:18.564589+0000","last_unstale":"2024-05-27T09:27:20.836574+0000","last_undegraded":"2024-05-27T09:27:20.836574+0000","last_fullsized":"2024-05-27T09:27:20.836574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:21:00.644551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.650158+0000","last_change":"2024-05-27T09:27:17.561344+0000","last_active":"2024-05-27T09:27:28.650158+0000","last_peered":"2024-05-27T09:27:28.650158+0000","last_clean":"2024-05-27T09:27:28.650158+0000","last_became_active":"2024-05-27T09:27:17.554860+0000","last_became_peered":"2024-05-27T09:27:17.554860+0000","last_unstale":"2024-05-27T09:27:28.650158+0000","last_undegraded":"2024-05-27T09:27:28.650158+0000","last_fullsized":"2024-05-27T09:27:28.650158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:16.880101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837236+0000","last_change":"2024-05-27T09:27:18.575317+0000","last_active":"2024-05-27T09:27:20.837236+0000","last_peered":"2024-05-27T09:27:20.837236+0000","last_clean":"2024-05-27T09:27:20.837236+0000","last_became_active":"2024-05-27T09:27:18.572024+0000","last_became_peered":"2024-05-27T09:27:18.572024+0000","last_unstale":"2024-05-27T09:27:20.837236+0000","last_undegraded":"2024-05-27T09:27:20.837236+0000","last_fullsized":"2024-05-27T09:27:20.837236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:23:35.418697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.739197+0000","last_change":"2024-05-27T09:27:17.575859+0000","last_active":"2024-05-27T09:27:33.739197+0000","last_peered":"2024-05-27T09:27:33.739197+0000","last_clean":"2024-05-27T09:27:33.739197+0000","last_became_active":"2024-05-27T09:27:17.574695+0000","last_became_peered":"2024-05-27T09:27:17.574695+0000","last_unstale":"2024-05-27T09:27:33.739197+0000","last_undegraded":"2024-05-27T09:27:33.739197+0000","last_fullsized":"2024-05-27T09:27:33.739197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:24:19.379819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539060+0000","last_change":"2024-05-27T09:27:18.555137+0000","last_active":"2024-05-27T09:27:19.539060+0000","last_peered":"2024-05-27T09:27:19.539060+0000","last_clean":"2024-05-27T09:27:19.539060+0000","last_became_active":"2024-05-27T09:27:18.554932+0000","last_became_peered":"2024-05-27T09:27:18.554932+0000","last_unstale":"2024-05-27T09:27:19.539060+0000","last_undegraded":"2024-05-27T09:27:19.539060+0000","last_fullsized":"2024-05-27T09:27:19.539060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:06.224637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'15","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.739565+0000","last_change":"2024-05-27T09:27:17.547895+0000","last_active":"2024-05-27T09:27:33.739565+0000","last_peered":"2024-05-27T09:27:33.739565+0000","last_clean":"2024-05-27T09:27:33.739565+0000","last_became_active":"2024-05-27T09:27:17.547523+0000","last_became_peered":"2024-05-27T09:27:17.547523+0000","last_unstale":"2024-05-27T09:27:33.739565+0000","last_undegraded":"2024-05-27T09:27:33.739565+0000","last_fullsized":"2024-05-27T09:27:33.739565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836706+0000","last_change":"2024-05-27T09:27:18.584284+0000","last_active":"2024-05-27T09:27:20.836706+0000","last_peered":"2024-05-27T09:27:20.836706+0000","last_clean":"2024-05-27T09:27:20.836706+0000","last_became_active":"2024-05-27T09:27:18.582360+0000","last_became_peered":"2024-05-27T09:27:18.582360+0000","last_unstale":"2024-05-27T09:27:20.836706+0000","last_undegraded":"2024-05-27T09:27:20.836706+0000","last_fullsized":"2024-05-27T09:27:20.836706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:32.535070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.550902+0000","last_change":"2024-05-27T09:27:17.571481+0000","last_active":"2024-05-27T09:27:23.550902+0000","last_peered":"2024-05-27T09:27:23.550902+0000","last_clean":"2024-05-27T09:27:23.550902+0000","last_became_active":"2024-05-27T09:27:17.571146+0000","last_became_peered":"2024-05-27T09:27:17.571146+0000","last_unstale":"2024-05-27T09:27:23.550902+0000","last_undegraded":"2024-05-27T09:27:23.550902+0000","last_fullsized":"2024-05-27T09:27:23.550902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:18.038221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542481+0000","last_change":"2024-05-27T09:27:18.580425+0000","last_active":"2024-05-27T09:27:19.542481+0000","last_peered":"2024-05-27T09:27:19.542481+0000","last_clean":"2024-05-27T09:27:19.542481+0000","last_became_active":"2024-05-27T09:27:18.579440+0000","last_became_peered":"2024-05-27T09:27:18.579440+0000","last_unstale":"2024-05-27T09:27:19.542481+0000","last_undegraded":"2024-05-27T09:27:19.542481+0000","last_fullsized":"2024-05-27T09:27:19.542481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:19.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.554242+0000","last_change":"2024-05-27T09:27:17.551295+0000","last_active":"2024-05-27T09:27:23.554242+0000","last_peered":"2024-05-27T09:27:23.554242+0000","last_clean":"2024-05-27T09:27:23.554242+0000","last_became_active":"2024-05-27T09:27:17.550861+0000","last_became_peered":"2024-05-27T09:27:17.550861+0000","last_unstale":"2024-05-27T09:27:23.554242+0000","last_undegraded":"2024-05-27T09:27:23.554242+0000","last_fullsized":"2024-05-27T09:27:23.554242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:13.955191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.545103+0000","last_change":"2024-05-27T09:27:18.547199+0000","last_active":"2024-05-27T09:27:19.545103+0000","last_peered":"2024-05-27T09:27:19.545103+0000","last_clean":"2024-05-27T09:27:19.545103+0000","last_became_active":"2024-05-27T09:27:18.543683+0000","last_became_peered":"2024-05-27T09:27:18.543683+0000","last_unstale":"2024-05-27T09:27:19.545103+0000","last_undegraded":"2024-05-27T09:27:19.545103+0000","last_fullsized":"2024-05-27T09:27:19.545103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:48:19.573654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:21.370295+0000","last_change":"2024-05-27T09:27:17.559255+0000","last_active":"2024-05-27T09:27:21.370295+0000","last_peered":"2024-05-27T09:27:21.370295+0000","last_clean":"2024-05-27T09:27:21.370295+0000","last_became_active":"2024-05-27T09:27:17.558973+0000","last_became_peered":"2024-05-27T09:27:17.558973+0000","last_unstale":"2024-05-27T09:27:21.370295+0000","last_undegraded":"2024-05-27T09:27:21.370295+0000","last_fullsized":"2024-05-27T09:27:21.370295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544602+0000","last_change":"2024-05-27T09:27:18.557423+0000","last_active":"2024-05-27T09:27:19.544602+0000","last_peered":"2024-05-27T09:27:19.544602+0000","last_clean":"2024-05-27T09:27:19.544602+0000","last_became_active":"2024-05-27T09:27:18.557076+0000","last_became_peered":"2024-05-27T09:27:18.557076+0000","last_unstale":"2024-05-27T09:27:19.544602+0000","last_undegraded":"2024-05-27T09:27:19.544602+0000","last_fullsized":"2024-05-27T09:27:19.544602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:11:04.612363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.738040+0000","last_change":"2024-05-27T09:27:17.573700+0000","last_active":"2024-05-27T09:27:33.738040+0000","last_peered":"2024-05-27T09:27:33.738040+0000","last_clean":"2024-05-27T09:27:33.738040+0000","last_became_active":"2024-05-27T09:27:17.573484+0000","last_became_peered":"2024-05-27T09:27:17.573484+0000","last_unstale":"2024-05-27T09:27:33.738040+0000","last_undegraded":"2024-05-27T09:27:33.738040+0000","last_fullsized":"2024-05-27T09:27:33.738040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:48.905422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837609+0000","last_change":"2024-05-27T09:27:18.582777+0000","last_active":"2024-05-27T09:27:20.837609+0000","last_peered":"2024-05-27T09:27:20.837609+0000","last_clean":"2024-05-27T09:27:20.837609+0000","last_became_active":"2024-05-27T09:27:18.574959+0000","last_became_peered":"2024-05-27T09:27:18.574959+0000","last_unstale":"2024-05-27T09:27:20.837609+0000","last_undegraded":"2024-05-27T09:27:20.837609+0000","last_fullsized":"2024-05-27T09:27:20.837609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:40:41.206755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":110,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539210+0000","last_change":"2024-05-27T09:27:12.693423+0000","last_active":"2024-05-27T09:27:19.539210+0000","last_peered":"2024-05-27T09:27:19.539210+0000","last_clean":"2024-05-27T09:27:19.539210+0000","last_became_active":"2024-05-27T09:27:12.690301+0000","last_became_peered":"2024-05-27T09:27:12.690301+0000","last_unstale":"2024-05-27T09:27:19.539210+0000","last_undegraded":"2024-05-27T09:27:19.539210+0000","last_fullsized":"2024-05-27T09:27:19.539210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:11.463226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:11.463226+0000","last_clean_scrub_stamp":"2024-05-27T09:27:11.463226+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:01.810071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.650695+0000","last_change":"2024-05-27T09:27:17.567670+0000","last_active":"2024-05-27T09:27:28.650695+0000","last_peered":"2024-05-27T09:27:28.650695+0000","last_clean":"2024-05-27T09:27:28.650695+0000","last_became_active":"2024-05-27T09:27:17.566292+0000","last_became_peered":"2024-05-27T09:27:17.566292+0000","last_unstale":"2024-05-27T09:27:28.650695+0000","last_undegraded":"2024-05-27T09:27:28.650695+0000","last_fullsized":"2024-05-27T09:27:28.650695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:18.038221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836742+0000","last_change":"2024-05-27T09:27:18.571124+0000","last_active":"2024-05-27T09:27:20.836742+0000","last_peered":"2024-05-27T09:27:20.836742+0000","last_clean":"2024-05-27T09:27:20.836742+0000","last_became_active":"2024-05-27T09:27:18.570782+0000","last_became_peered":"2024-05-27T09:27:18.570782+0000","last_unstale":"2024-05-27T09:27:20.836742+0000","last_undegraded":"2024-05-27T09:27:20.836742+0000","last_fullsized":"2024-05-27T09:27:20.836742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:39:58.110527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.734194+0000","last_change":"2024-05-27T09:27:17.563259+0000","last_active":"2024-05-27T09:27:33.734194+0000","last_peered":"2024-05-27T09:27:33.734194+0000","last_clean":"2024-05-27T09:27:33.734194+0000","last_became_active":"2024-05-27T09:27:17.563010+0000","last_became_peered":"2024-05-27T09:27:17.563010+0000","last_unstale":"2024-05-27T09:27:33.734194+0000","last_undegraded":"2024-05-27T09:27:33.734194+0000","last_fullsized":"2024-05-27T09:27:33.734194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:53.711508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836371+0000","last_change":"2024-05-27T09:27:18.574466+0000","last_active":"2024-05-27T09:27:20.836371+0000","last_peered":"2024-05-27T09:27:20.836371+0000","last_clean":"2024-05-27T09:27:20.836371+0000","last_became_active":"2024-05-27T09:27:18.574090+0000","last_became_peered":"2024-05-27T09:27:18.574090+0000","last_unstale":"2024-05-27T09:27:20.836371+0000","last_undegraded":"2024-05-27T09:27:20.836371+0000","last_fullsized":"2024-05-27T09:27:20.836371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:41:22.515484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.737344+0000","last_change":"2024-05-27T09:27:17.551310+0000","last_active":"2024-05-27T09:27:33.737344+0000","last_peered":"2024-05-27T09:27:33.737344+0000","last_clean":"2024-05-27T09:27:33.737344+0000","last_became_active":"2024-05-27T09:27:17.550982+0000","last_became_peered":"2024-05-27T09:27:17.550982+0000","last_unstale":"2024-05-27T09:27:33.737344+0000","last_undegraded":"2024-05-27T09:27:33.737344+0000","last_fullsized":"2024-05-27T09:27:33.737344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836574+0000","last_change":"2024-05-27T09:27:18.559416+0000","last_active":"2024-05-27T09:27:20.836574+0000","last_peered":"2024-05-27T09:27:20.836574+0000","last_clean":"2024-05-27T09:27:20.836574+0000","last_became_active":"2024-05-27T09:27:18.559017+0000","last_became_peered":"2024-05-27T09:27:18.559017+0000","last_unstale":"2024-05-27T09:27:20.836574+0000","last_undegraded":"2024-05-27T09:27:20.836574+0000","last_fullsized":"2024-05-27T09:27:20.836574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:33:48.400306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837736+0000","last_change":"2024-05-27T09:27:17.563947+0000","last_active":"2024-05-27T09:27:20.837736+0000","last_peered":"2024-05-27T09:27:20.837736+0000","last_clean":"2024-05-27T09:27:20.837736+0000","last_became_active":"2024-05-27T09:27:17.563640+0000","last_became_peered":"2024-05-27T09:27:17.563640+0000","last_unstale":"2024-05-27T09:27:20.837736+0000","last_undegraded":"2024-05-27T09:27:20.837736+0000","last_fullsized":"2024-05-27T09:27:20.837736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:02.747708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543615+0000","last_change":"2024-05-27T09:27:18.575835+0000","last_active":"2024-05-27T09:27:19.543615+0000","last_peered":"2024-05-27T09:27:19.543615+0000","last_clean":"2024-05-27T09:27:19.543615+0000","last_became_active":"2024-05-27T09:27:18.574710+0000","last_became_peered":"2024-05-27T09:27:18.574710+0000","last_unstale":"2024-05-27T09:27:19.543615+0000","last_undegraded":"2024-05-27T09:27:19.543615+0000","last_fullsized":"2024-05-27T09:27:19.543615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:16:00.934068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835781+0000","last_change":"2024-05-27T09:27:17.550749+0000","last_active":"2024-05-27T09:27:20.835781+0000","last_peered":"2024-05-27T09:27:20.835781+0000","last_clean":"2024-05-27T09:27:20.835781+0000","last_became_active":"2024-05-27T09:27:17.550378+0000","last_became_peered":"2024-05-27T09:27:17.550378+0000","last_unstale":"2024-05-27T09:27:20.835781+0000","last_undegraded":"2024-05-27T09:27:20.835781+0000","last_fullsized":"2024-05-27T09:27:20.835781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:20.167629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.541045+0000","last_change":"2024-05-27T09:27:18.567233+0000","last_active":"2024-05-27T09:27:19.541045+0000","last_peered":"2024-05-27T09:27:19.541045+0000","last_clean":"2024-05-27T09:27:19.541045+0000","last_became_active":"2024-05-27T09:27:18.566326+0000","last_became_peered":"2024-05-27T09:27:18.566326+0000","last_unstale":"2024-05-27T09:27:19.541045+0000","last_undegraded":"2024-05-27T09:27:19.541045+0000","last_fullsized":"2024-05-27T09:27:19.541045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:27.225150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837888+0000","last_change":"2024-05-27T09:27:17.557912+0000","last_active":"2024-05-27T09:27:20.837888+0000","last_peered":"2024-05-27T09:27:20.837888+0000","last_clean":"2024-05-27T09:27:20.837888+0000","last_became_active":"2024-05-27T09:27:17.557216+0000","last_became_peered":"2024-05-27T09:27:17.557216+0000","last_unstale":"2024-05-27T09:27:20.837888+0000","last_undegraded":"2024-05-27T09:27:20.837888+0000","last_fullsized":"2024-05-27T09:27:20.837888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:21:22.020724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836847+0000","last_change":"2024-05-27T09:27:18.563649+0000","last_active":"2024-05-27T09:27:20.836847+0000","last_peered":"2024-05-27T09:27:20.836847+0000","last_clean":"2024-05-27T09:27:20.836847+0000","last_became_active":"2024-05-27T09:27:18.562565+0000","last_became_peered":"2024-05-27T09:27:18.562565+0000","last_unstale":"2024-05-27T09:27:20.836847+0000","last_undegraded":"2024-05-27T09:27:20.836847+0000","last_fullsized":"2024-05-27T09:27:20.836847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:46:49.444709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.734132+0000","last_change":"2024-05-27T09:27:17.571911+0000","last_active":"2024-05-27T09:27:33.734132+0000","last_peered":"2024-05-27T09:27:33.734132+0000","last_clean":"2024-05-27T09:27:33.734132+0000","last_became_active":"2024-05-27T09:27:17.571551+0000","last_became_peered":"2024-05-27T09:27:17.571551+0000","last_unstale":"2024-05-27T09:27:33.734132+0000","last_undegraded":"2024-05-27T09:27:33.734132+0000","last_fullsized":"2024-05-27T09:27:33.734132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:20.167629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835899+0000","last_change":"2024-05-27T09:27:18.558780+0000","last_active":"2024-05-27T09:27:20.835899+0000","last_peered":"2024-05-27T09:27:20.835899+0000","last_clean":"2024-05-27T09:27:20.835899+0000","last_became_active":"2024-05-27T09:27:18.558463+0000","last_became_peered":"2024-05-27T09:27:18.558463+0000","last_unstale":"2024-05-27T09:27:20.835899+0000","last_undegraded":"2024-05-27T09:27:20.835899+0000","last_fullsized":"2024-05-27T09:27:20.835899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:42:38.351682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.646698+0000","last_change":"2024-05-27T09:27:17.567973+0000","last_active":"2024-05-27T09:27:28.646698+0000","last_peered":"2024-05-27T09:27:28.646698+0000","last_clean":"2024-05-27T09:27:28.646698+0000","last_became_active":"2024-05-27T09:27:17.567650+0000","last_became_peered":"2024-05-27T09:27:17.567650+0000","last_unstale":"2024-05-27T09:27:28.646698+0000","last_undegraded":"2024-05-27T09:27:28.646698+0000","last_fullsized":"2024-05-27T09:27:28.646698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:25:52.924185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835408+0000","last_change":"2024-05-27T09:27:18.554845+0000","last_active":"2024-05-27T09:27:20.835408+0000","last_peered":"2024-05-27T09:27:20.835408+0000","last_clean":"2024-05-27T09:27:20.835408+0000","last_became_active":"2024-05-27T09:27:18.554462+0000","last_became_peered":"2024-05-27T09:27:18.554462+0000","last_unstale":"2024-05-27T09:27:20.835408+0000","last_undegraded":"2024-05-27T09:27:20.835408+0000","last_fullsized":"2024-05-27T09:27:20.835408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:44:47.447245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.625211+0000","last_change":"2024-05-27T09:27:17.570300+0000","last_active":"2024-05-27T09:27:28.625211+0000","last_peered":"2024-05-27T09:27:28.625211+0000","last_clean":"2024-05-27T09:27:28.625211+0000","last_became_active":"2024-05-27T09:27:17.569685+0000","last_became_peered":"2024-05-27T09:27:17.569685+0000","last_unstale":"2024-05-27T09:27:28.625211+0000","last_undegraded":"2024-05-27T09:27:28.625211+0000","last_fullsized":"2024-05-27T09:27:28.625211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:26.210041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836430+0000","last_change":"2024-05-27T09:27:18.562774+0000","last_active":"2024-05-27T09:27:20.836430+0000","last_peered":"2024-05-27T09:27:20.836430+0000","last_clean":"2024-05-27T09:27:20.836430+0000","last_became_active":"2024-05-27T09:27:18.562455+0000","last_became_peered":"2024-05-27T09:27:18.562455+0000","last_unstale":"2024-05-27T09:27:20.836430+0000","last_undegraded":"2024-05-27T09:27:20.836430+0000","last_fullsized":"2024-05-27T09:27:20.836430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:39:59.034873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.627233+0000","last_change":"2024-05-27T09:27:17.563172+0000","last_active":"2024-05-27T09:27:28.627233+0000","last_peered":"2024-05-27T09:27:28.627233+0000","last_clean":"2024-05-27T09:27:28.627233+0000","last_became_active":"2024-05-27T09:27:17.562941+0000","last_became_peered":"2024-05-27T09:27:17.562941+0000","last_unstale":"2024-05-27T09:27:28.627233+0000","last_undegraded":"2024-05-27T09:27:28.627233+0000","last_fullsized":"2024-05-27T09:27:28.627233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:51.742078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836477+0000","last_change":"2024-05-27T09:27:18.555617+0000","last_active":"2024-05-27T09:27:20.836477+0000","last_peered":"2024-05-27T09:27:20.836477+0000","last_clean":"2024-05-27T09:27:20.836477+0000","last_became_active":"2024-05-27T09:27:18.555280+0000","last_became_peered":"2024-05-27T09:27:18.555280+0000","last_unstale":"2024-05-27T09:27:20.836477+0000","last_undegraded":"2024-05-27T09:27:20.836477+0000","last_fullsized":"2024-05-27T09:27:20.836477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:32:39.043268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:21.367453+0000","last_change":"2024-05-27T09:27:17.548266+0000","last_active":"2024-05-27T09:27:21.367453+0000","last_peered":"2024-05-27T09:27:21.367453+0000","last_clean":"2024-05-27T09:27:21.367453+0000","last_became_active":"2024-05-27T09:27:17.547938+0000","last_became_peered":"2024-05-27T09:27:17.547938+0000","last_unstale":"2024-05-27T09:27:21.367453+0000","last_undegraded":"2024-05-27T09:27:21.367453+0000","last_fullsized":"2024-05-27T09:27:21.367453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835677+0000","last_change":"2024-05-27T09:27:18.563586+0000","last_active":"2024-05-27T09:27:20.835677+0000","last_peered":"2024-05-27T09:27:20.835677+0000","last_clean":"2024-05-27T09:27:20.835677+0000","last_became_active":"2024-05-27T09:27:18.563227+0000","last_became_peered":"2024-05-27T09:27:18.563227+0000","last_unstale":"2024-05-27T09:27:20.835677+0000","last_undegraded":"2024-05-27T09:27:20.835677+0000","last_fullsized":"2024-05-27T09:27:20.835677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:53.222040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.647667+0000","last_change":"2024-05-27T09:27:17.562200+0000","last_active":"2024-05-27T09:27:28.647667+0000","last_peered":"2024-05-27T09:27:28.647667+0000","last_clean":"2024-05-27T09:27:28.647667+0000","last_became_active":"2024-05-27T09:27:17.561857+0000","last_became_peered":"2024-05-27T09:27:17.561857+0000","last_unstale":"2024-05-27T09:27:28.647667+0000","last_undegraded":"2024-05-27T09:27:28.647667+0000","last_fullsized":"2024-05-27T09:27:28.647667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:05.209528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836936+0000","last_change":"2024-05-27T09:27:18.572676+0000","last_active":"2024-05-27T09:27:20.836936+0000","last_peered":"2024-05-27T09:27:20.836936+0000","last_clean":"2024-05-27T09:27:20.836936+0000","last_became_active":"2024-05-27T09:27:18.572339+0000","last_became_peered":"2024-05-27T09:27:18.572339+0000","last_unstale":"2024-05-27T09:27:20.836936+0000","last_undegraded":"2024-05-27T09:27:20.836936+0000","last_fullsized":"2024-05-27T09:27:20.836936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:09:23.795528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837085+0000","last_change":"2024-05-27T09:27:18.575394+0000","last_active":"2024-05-27T09:27:20.837085+0000","last_peered":"2024-05-27T09:27:20.837085+0000","last_clean":"2024-05-27T09:27:20.837085+0000","last_became_active":"2024-05-27T09:27:18.572307+0000","last_became_peered":"2024-05-27T09:27:18.572307+0000","last_unstale":"2024-05-27T09:27:20.837085+0000","last_undegraded":"2024-05-27T09:27:20.837085+0000","last_fullsized":"2024-05-27T09:27:20.837085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:29:05.116582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.555603+0000","last_change":"2024-05-27T09:27:17.570634+0000","last_active":"2024-05-27T09:27:23.555603+0000","last_peered":"2024-05-27T09:27:23.555603+0000","last_clean":"2024-05-27T09:27:23.555603+0000","last_became_active":"2024-05-27T09:27:17.569866+0000","last_became_peered":"2024-05-27T09:27:17.569866+0000","last_unstale":"2024-05-27T09:27:23.555603+0000","last_undegraded":"2024-05-27T09:27:23.555603+0000","last_fullsized":"2024-05-27T09:27:23.555603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:21:38.268535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539980+0000","last_change":"2024-05-27T09:27:18.549339+0000","last_active":"2024-05-27T09:27:19.539980+0000","last_peered":"2024-05-27T09:27:19.539980+0000","last_clean":"2024-05-27T09:27:19.539980+0000","last_became_active":"2024-05-27T09:27:18.549035+0000","last_became_peered":"2024-05-27T09:27:18.549035+0000","last_unstale":"2024-05-27T09:27:19.539980+0000","last_undegraded":"2024-05-27T09:27:19.539980+0000","last_fullsized":"2024-05-27T09:27:19.539980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:17.895211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.737950+0000","last_change":"2024-05-27T09:27:17.568143+0000","last_active":"2024-05-27T09:27:33.737950+0000","last_peered":"2024-05-27T09:27:33.737950+0000","last_clean":"2024-05-27T09:27:33.737950+0000","last_became_active":"2024-05-27T09:27:17.567235+0000","last_became_peered":"2024-05-27T09:27:17.567235+0000","last_unstale":"2024-05-27T09:27:33.737950+0000","last_undegraded":"2024-05-27T09:27:33.737950+0000","last_fullsized":"2024-05-27T09:27:33.737950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.284270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836871+0000","last_change":"2024-05-27T09:27:18.563270+0000","last_active":"2024-05-27T09:27:20.836871+0000","last_peered":"2024-05-27T09:27:20.836871+0000","last_clean":"2024-05-27T09:27:20.836871+0000","last_became_active":"2024-05-27T09:27:18.562919+0000","last_became_peered":"2024-05-27T09:27:18.562919+0000","last_unstale":"2024-05-27T09:27:20.836871+0000","last_undegraded":"2024-05-27T09:27:20.836871+0000","last_fullsized":"2024-05-27T09:27:20.836871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:55:47.799529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.554180+0000","last_change":"2024-05-27T09:27:17.551193+0000","last_active":"2024-05-27T09:27:23.554180+0000","last_peered":"2024-05-27T09:27:23.554180+0000","last_clean":"2024-05-27T09:27:23.554180+0000","last_became_active":"2024-05-27T09:27:17.550849+0000","last_became_peered":"2024-05-27T09:27:17.550849+0000","last_unstale":"2024-05-27T09:27:23.554180+0000","last_undegraded":"2024-05-27T09:27:23.554180+0000","last_fullsized":"2024-05-27T09:27:23.554180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:51.742078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836183+0000","last_change":"2024-05-27T09:27:18.579803+0000","last_active":"2024-05-27T09:27:20.836183+0000","last_peered":"2024-05-27T09:27:20.836183+0000","last_clean":"2024-05-27T09:27:20.836183+0000","last_became_active":"2024-05-27T09:27:18.579574+0000","last_became_peered":"2024-05-27T09:27:18.579574+0000","last_unstale":"2024-05-27T09:27:20.836183+0000","last_undegraded":"2024-05-27T09:27:20.836183+0000","last_fullsized":"2024-05-27T09:27:20.836183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:11:22.320778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'9","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.737701+0000","last_change":"2024-05-27T09:27:17.549711+0000","last_active":"2024-05-27T09:27:33.737701+0000","last_peered":"2024-05-27T09:27:33.737701+0000","last_clean":"2024-05-27T09:27:33.737701+0000","last_became_active":"2024-05-27T09:27:17.549291+0000","last_became_peered":"2024-05-27T09:27:17.549291+0000","last_unstale":"2024-05-27T09:27:33.737701+0000","last_undegraded":"2024-05-27T09:27:33.737701+0000","last_fullsized":"2024-05-27T09:27:33.737701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:16:22.057803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5476,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":11206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65078,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":225280,"data_stored":73901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":225280,"data_stored":74321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":663203,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":73991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542154,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":73583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":74133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542154,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":664101,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":76109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T09:27:48.074 INFO:tasks.mds_thrash.ceph_manager:clean! 2024-05-27T09:27:48.075 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T09:27:48.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.378+0000 7fc394a00640 1 Processor -- start 2024-05-27T09:27:48.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.379+0000 7fc394a00640 1 -- start start 2024-05-27T09:27:48.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.381+0000 7fc394a00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc3900b3640 0x7fc390070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:48.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.381+0000 7fc394a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc390070350 0x7fc390070760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:48.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.381+0000 7fc394a00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3900c93b0 con 0x7fc3900b3c00 2024-05-27T09:27:48.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.381+0000 7fc394a00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fc3900cbbb0 con 0x7fc3900b3640 2024-05-27T09:27:48.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.381+0000 7fc394a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fc390072410 con 0x7fc390070350 2024-05-27T09:27:48.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.381+0000 7fc38e800640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fc3900b3c00 0x7fc3900cb4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:45402/0 (socket says 172.21.15.114:45402) 2024-05-27T09:27:48.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.381+0000 7fc38e800640 1 -- 172.21.15.114:0/3393653930 learned_addr learned my addr 172.21.15.114:0/3393653930 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:48.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.381+0000 7fc38f200640 1 --2- 172.21.15.114:0/3393653930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc390070350 0x7fc390070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:48.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.381+0000 7fc387e00640 1 --2- 172.21.15.114:0/3393653930 >> v2:172.21.15.163:3300/0 conn(0x7fc3900b3640 0x7fc390070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:48.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.381+0000 7fc38de00640 1 -- 172.21.15.114:0/3393653930 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1235194285 0 0) 0x7fc380001110 con 0x7fc3900b3c00 2024-05-27T09:27:48.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.381+0000 7fc38de00640 1 -- 172.21.15.114:0/3393653930 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc374003520 con 0x7fc3900b3c00 2024-05-27T09:27:48.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.382+0000 7fc387e00640 1 -- 172.21.15.114:0/3393653930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc390070350 msgr2=0x7fc390070760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:48.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.382+0000 7fc387e00640 1 --2- 172.21.15.114:0/3393653930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc390070350 0x7fc390070760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:48.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.382+0000 7fc387e00640 1 -- 172.21.15.114:0/3393653930 >> v1:172.21.15.114:6789/0 conn(0x7fc3900b3c00 legacy=0x7fc3900cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:48.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.382+0000 7fc387e00640 1 -- 172.21.15.114:0/3393653930 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3900c9630 con 0x7fc3900b3640 2024-05-27T09:27:48.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.382+0000 7fc38f200640 1 --2- 172.21.15.114:0/3393653930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc390070350 0x7fc390070760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:48.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.383+0000 7fc387e00640 1 --2- 172.21.15.114:0/3393653930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3900b3640 0x7fc390070e80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc3780097a0 tx=0x7fc37802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=551cd7afdcb6c6c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:48.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.383+0000 7fc38de00640 1 -- 172.21.15.114:0/3393653930 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378037070 con 0x7fc3900b3640 2024-05-27T09:27:48.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.383+0000 7fc38de00640 1 -- 172.21.15.114:0/3393653930 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc378032440 con 0x7fc3900b3640 2024-05-27T09:27:48.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.383+0000 7fc38de00640 1 -- 172.21.15.114:0/3393653930 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3780327f0 con 0x7fc3900b3640 2024-05-27T09:27:48.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.384+0000 7fc394a00640 1 -- 172.21.15.114:0/3393653930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3900b3640 msgr2=0x7fc390070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:48.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.384+0000 7fc394a00640 1 --2- 172.21.15.114:0/3393653930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3900b3640 0x7fc390070e80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc3780097a0 tx=0x7fc37802a180 comp rx=0 tx=0).stop 2024-05-27T09:27:48.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.384+0000 7fc394a00640 1 -- 172.21.15.114:0/3393653930 shutdown_connections 2024-05-27T09:27:48.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.384+0000 7fc394a00640 1 --2- 172.21.15.114:0/3393653930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc390070350 0x7fc390070760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:48.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.384+0000 7fc394a00640 1 --2- 172.21.15.114:0/3393653930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3900b3640 0x7fc390070e80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:48.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.384+0000 7fc394a00640 1 -- 172.21.15.114:0/3393653930 >> 172.21.15.114:0/3393653930 conn(0x7fc39006d7a0 msgr2=0x7fc39006dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:48.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.384+0000 7fc394a00640 1 -- 172.21.15.114:0/3393653930 shutdown_connections 2024-05-27T09:27:48.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.384+0000 7fc394a00640 1 -- 172.21.15.114:0/3393653930 wait complete. 2024-05-27T09:27:48.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.386+0000 7fc394a00640 1 Processor -- start 2024-05-27T09:27:48.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.386+0000 7fc394a00640 1 -- start start 2024-05-27T09:27:48.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.388+0000 7fc394a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc390070350 0x7fc3901f6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:48.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.388+0000 7fc394a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3900b3640 0x7fc390158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:48.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.388+0000 7fc394a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3900b3c00 0x7fc3900c4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:48.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.388+0000 7fc394a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc390156410 con 0x7fc390070350 2024-05-27T09:27:48.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.388+0000 7fc394a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc390072940 con 0x7fc3900b3640 2024-05-27T09:27:48.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.388+0000 7fc394a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fc390156590 con 0x7fc3900b3c00 2024-05-27T09:27:48.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.388+0000 7fc38e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc390070350 0x7fc3901f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:48.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.388+0000 7fc38e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc390070350 0x7fc3901f6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38034/0 (socket says 172.21.15.114:38034) 2024-05-27T09:27:48.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.388+0000 7fc38e800640 1 -- 172.21.15.114:0/372952316 learned_addr learned my addr 172.21.15.114:0/372952316 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:48.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.388+0000 7fc38f200640 1 --2- 172.21.15.114:0/372952316 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3900b3c00 0x7fc3900c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:48.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.388+0000 7fc387e00640 1 --2- 172.21.15.114:0/372952316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3900b3640 0x7fc390158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:48.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.388+0000 7fc38e800640 1 -- 172.21.15.114:0/372952316 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3900b3c00 msgr2=0x7fc3900c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:48.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.388+0000 7fc38e800640 1 --2- 172.21.15.114:0/372952316 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3900b3c00 0x7fc3900c4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:48.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.388+0000 7fc38e800640 1 -- 172.21.15.114:0/372952316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3900b3640 msgr2=0x7fc390158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:48.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.388+0000 7fc38e800640 1 --2- 172.21.15.114:0/372952316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3900b3640 0x7fc390158b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:48.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.389+0000 7fc38e800640 1 -- 172.21.15.114:0/372952316 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3901f6840 con 0x7fc390070350 2024-05-27T09:27:48.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.389+0000 7fc387e00640 1 --2- 172.21.15.114:0/372952316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3900b3640 0x7fc390158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:48.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.389+0000 7fc38e800640 1 --2- 172.21.15.114:0/372952316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc390070350 0x7fc3901f6300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc38000c700 tx=0x7fc38000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:48.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.389+0000 7fc386a00640 1 -- 172.21.15.114:0/372952316 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380017070 con 0x7fc390070350 2024-05-27T09:27:48.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.389+0000 7fc386a00640 1 -- 172.21.15.114:0/372952316 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc380005ce0 con 0x7fc390070350 2024-05-27T09:27:48.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.389+0000 7fc394a00640 1 -- 172.21.15.114:0/372952316 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3901596b0 con 0x7fc390070350 2024-05-27T09:27:48.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.389+0000 7fc386a00640 1 -- 172.21.15.114:0/372952316 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3800072d0 con 0x7fc390070350 2024-05-27T09:27:48.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.389+0000 7fc394a00640 1 -- 172.21.15.114:0/372952316 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3901f27a0 con 0x7fc390070350 2024-05-27T09:27:48.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.391+0000 7fc394a00640 1 -- 172.21.15.114:0/372952316 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc390050860 con 0x7fc390070350 2024-05-27T09:27:48.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.392+0000 7fc386a00640 1 -- 172.21.15.114:0/372952316 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc38001a070 con 0x7fc390070350 2024-05-27T09:27:48.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.393+0000 7fc386a00640 1 --2- 172.21.15.114:0/372952316 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc36c076300 0x7fc36c078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:48.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.393+0000 7fc386a00640 1 -- 172.21.15.114:0/372952316 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc380096960 con 0x7fc390070350 2024-05-27T09:27:48.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.393+0000 7fc387e00640 1 --2- 172.21.15.114:0/372952316 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc36c076300 0x7fc36c078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:48.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.394+0000 7fc387e00640 1 --2- 172.21.15.114:0/372952316 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc36c076300 0x7fc36c078810 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc378034000 tx=0x7fc37802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:48.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.398+0000 7fc386a00640 1 -- 172.21.15.114:0/372952316 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc3900c93b0 con 0x7fc390070350 2024-05-27T09:27:48.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.600+0000 7fc394a00640 1 -- 172.21.15.114:0/372952316 --> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc390156d80 con 0x7fc36c076300 2024-05-27T09:27:48.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.607+0000 7fc386a00640 1 -- 172.21.15.114:0/372952316 <== mgr.4108 v2:172.21.15.163:6832/1122018204 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251982 (secure 0 0 0) 0x7fc390156d80 con 0x7fc36c076300 2024-05-27T09:27:48.611 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:48.613 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-05-27T09:27:48.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.613+0000 7fc394a00640 1 -- 172.21.15.114:0/372952316 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc36c076300 msgr2=0x7fc36c078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:48.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.613+0000 7fc394a00640 1 --2- 172.21.15.114:0/372952316 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc36c076300 0x7fc36c078810 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc378034000 tx=0x7fc37802e000 comp rx=0 tx=0).stop 2024-05-27T09:27:48.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.613+0000 7fc394a00640 1 -- 172.21.15.114:0/372952316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc390070350 msgr2=0x7fc3901f6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:48.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.613+0000 7fc394a00640 1 --2- 172.21.15.114:0/372952316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc390070350 0x7fc3901f6300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc38000c700 tx=0x7fc38000cb00 comp rx=0 tx=0).stop 2024-05-27T09:27:48.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.613+0000 7fc394a00640 1 -- 172.21.15.114:0/372952316 shutdown_connections 2024-05-27T09:27:48.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.613+0000 7fc394a00640 1 --2- 172.21.15.114:0/372952316 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc36c076300 0x7fc36c078810 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:48.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.613+0000 7fc394a00640 1 --2- 172.21.15.114:0/372952316 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3900b3c00 0x7fc3900c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:48.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.613+0000 7fc394a00640 1 --2- 172.21.15.114:0/372952316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3900b3640 0x7fc390158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:48.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.613+0000 7fc394a00640 1 --2- 172.21.15.114:0/372952316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc390070350 0x7fc3901f6300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:48.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.613+0000 7fc394a00640 1 -- 172.21.15.114:0/372952316 >> 172.21.15.114:0/372952316 conn(0x7fc39006d7a0 msgr2=0x7fc39006dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:48.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.613+0000 7fc394a00640 1 -- 172.21.15.114:0/372952316 shutdown_connections 2024-05-27T09:27:48.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.613+0000 7fc394a00640 1 -- 172.21.15.114:0/372952316 wait complete. 2024-05-27T09:27:48.637 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-05-27T09:27:46.769776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595844,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":227,"num_write_kb":1667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":182,"ondisk_log_size":182,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218048,"kb_used_data":1920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756672,"statfs":{"total":773094113280,"available":772870832128,"internally_reserved":0,"allocated":1966080,"data_stored":1773342,"data_compressed":65078,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1660,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003132"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544950+0000","last_change":"2024-05-27T09:27:18.585655+0000","last_active":"2024-05-27T09:27:19.544950+0000","last_peered":"2024-05-27T09:27:19.544950+0000","last_clean":"2024-05-27T09:27:19.544950+0000","last_became_active":"2024-05-27T09:27:18.585169+0000","last_became_peered":"2024-05-27T09:27:18.585169+0000","last_unstale":"2024-05-27T09:27:19.544950+0000","last_undegraded":"2024-05-27T09:27:19.544950+0000","last_fullsized":"2024-05-27T09:27:19.544950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:39:59.034873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835983+0000","last_change":"2024-05-27T09:27:18.566621+0000","last_active":"2024-05-27T09:27:20.835983+0000","last_peered":"2024-05-27T09:27:20.835983+0000","last_clean":"2024-05-27T09:27:20.835983+0000","last_became_active":"2024-05-27T09:27:18.566269+0000","last_became_peered":"2024-05-27T09:27:18.566269+0000","last_unstale":"2024-05-27T09:27:20.835983+0000","last_undegraded":"2024-05-27T09:27:20.835983+0000","last_fullsized":"2024-05-27T09:27:20.835983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:24:24.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835531+0000","last_change":"2024-05-27T09:27:18.560958+0000","last_active":"2024-05-27T09:27:20.835531+0000","last_peered":"2024-05-27T09:27:20.835531+0000","last_clean":"2024-05-27T09:27:20.835531+0000","last_became_active":"2024-05-27T09:27:18.560620+0000","last_became_peered":"2024-05-27T09:27:18.560620+0000","last_unstale":"2024-05-27T09:27:20.835531+0000","last_undegraded":"2024-05-27T09:27:20.835531+0000","last_fullsized":"2024-05-27T09:27:20.835531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:42:38.351682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542664+0000","last_change":"2024-05-27T09:27:18.569804+0000","last_active":"2024-05-27T09:27:19.542664+0000","last_peered":"2024-05-27T09:27:19.542664+0000","last_clean":"2024-05-27T09:27:19.542664+0000","last_became_active":"2024-05-27T09:27:18.569481+0000","last_became_peered":"2024-05-27T09:27:18.569481+0000","last_unstale":"2024-05-27T09:27:19.542664+0000","last_undegraded":"2024-05-27T09:27:19.542664+0000","last_fullsized":"2024-05-27T09:27:19.542664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:55:03.583659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543808+0000","last_change":"2024-05-27T09:27:18.568138+0000","last_active":"2024-05-27T09:27:19.543808+0000","last_peered":"2024-05-27T09:27:19.543808+0000","last_clean":"2024-05-27T09:27:19.543808+0000","last_became_active":"2024-05-27T09:27:18.567795+0000","last_became_peered":"2024-05-27T09:27:18.567795+0000","last_unstale":"2024-05-27T09:27:19.543808+0000","last_undegraded":"2024-05-27T09:27:19.543808+0000","last_fullsized":"2024-05-27T09:27:19.543808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:17:42.005593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.540719+0000","last_change":"2024-05-27T09:27:18.573984+0000","last_active":"2024-05-27T09:27:19.540719+0000","last_peered":"2024-05-27T09:27:19.540719+0000","last_clean":"2024-05-27T09:27:19.540719+0000","last_became_active":"2024-05-27T09:27:18.573576+0000","last_became_peered":"2024-05-27T09:27:18.573576+0000","last_unstale":"2024-05-27T09:27:19.540719+0000","last_undegraded":"2024-05-27T09:27:19.540719+0000","last_fullsized":"2024-05-27T09:27:19.540719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:06.224637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543010+0000","last_change":"2024-05-27T09:27:18.543122+0000","last_active":"2024-05-27T09:27:19.543010+0000","last_peered":"2024-05-27T09:27:19.543010+0000","last_clean":"2024-05-27T09:27:19.543010+0000","last_became_active":"2024-05-27T09:27:18.542593+0000","last_became_peered":"2024-05-27T09:27:18.542593+0000","last_unstale":"2024-05-27T09:27:19.543010+0000","last_undegraded":"2024-05-27T09:27:19.543010+0000","last_fullsized":"2024-05-27T09:27:19.543010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:28:19.579432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835946+0000","last_change":"2024-05-27T09:27:18.562292+0000","last_active":"2024-05-27T09:27:20.835946+0000","last_peered":"2024-05-27T09:27:20.835946+0000","last_clean":"2024-05-27T09:27:20.835946+0000","last_became_active":"2024-05-27T09:27:18.561788+0000","last_became_peered":"2024-05-27T09:27:18.561788+0000","last_unstale":"2024-05-27T09:27:20.835946+0000","last_undegraded":"2024-05-27T09:27:20.835946+0000","last_fullsized":"2024-05-27T09:27:20.835946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:32:39.043268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542996+0000","last_change":"2024-05-27T09:27:18.583770+0000","last_active":"2024-05-27T09:27:19.542996+0000","last_peered":"2024-05-27T09:27:19.542996+0000","last_clean":"2024-05-27T09:27:19.542996+0000","last_became_active":"2024-05-27T09:27:18.582041+0000","last_became_peered":"2024-05-27T09:27:18.582041+0000","last_unstale":"2024-05-27T09:27:19.542996+0000","last_undegraded":"2024-05-27T09:27:19.542996+0000","last_fullsized":"2024-05-27T09:27:19.542996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:19.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.545125+0000","last_change":"2024-05-27T09:27:18.574658+0000","last_active":"2024-05-27T09:27:19.545125+0000","last_peered":"2024-05-27T09:27:19.545125+0000","last_clean":"2024-05-27T09:27:19.545125+0000","last_became_active":"2024-05-27T09:27:18.574409+0000","last_became_peered":"2024-05-27T09:27:18.574409+0000","last_unstale":"2024-05-27T09:27:19.545125+0000","last_undegraded":"2024-05-27T09:27:19.545125+0000","last_fullsized":"2024-05-27T09:27:19.545125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:19.131343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544558+0000","last_change":"2024-05-27T09:27:18.579141+0000","last_active":"2024-05-27T09:27:19.544558+0000","last_peered":"2024-05-27T09:27:19.544558+0000","last_clean":"2024-05-27T09:27:19.544558+0000","last_became_active":"2024-05-27T09:27:18.578679+0000","last_became_peered":"2024-05-27T09:27:18.578679+0000","last_unstale":"2024-05-27T09:27:19.544558+0000","last_undegraded":"2024-05-27T09:27:19.544558+0000","last_fullsized":"2024-05-27T09:27:19.544558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:40:44.336453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544110+0000","last_change":"2024-05-27T09:27:18.547375+0000","last_active":"2024-05-27T09:27:19.544110+0000","last_peered":"2024-05-27T09:27:19.544110+0000","last_clean":"2024-05-27T09:27:19.544110+0000","last_became_active":"2024-05-27T09:27:18.544328+0000","last_became_peered":"2024-05-27T09:27:18.544328+0000","last_unstale":"2024-05-27T09:27:19.544110+0000","last_undegraded":"2024-05-27T09:27:19.544110+0000","last_fullsized":"2024-05-27T09:27:19.544110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:59:50.499894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836590+0000","last_change":"2024-05-27T09:27:18.561903+0000","last_active":"2024-05-27T09:27:20.836590+0000","last_peered":"2024-05-27T09:27:20.836590+0000","last_clean":"2024-05-27T09:27:20.836590+0000","last_became_active":"2024-05-27T09:27:18.561493+0000","last_became_peered":"2024-05-27T09:27:18.561493+0000","last_unstale":"2024-05-27T09:27:20.836590+0000","last_undegraded":"2024-05-27T09:27:20.836590+0000","last_fullsized":"2024-05-27T09:27:20.836590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:28:31.470269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836378+0000","last_change":"2024-05-27T09:27:18.563168+0000","last_active":"2024-05-27T09:27:20.836378+0000","last_peered":"2024-05-27T09:27:20.836378+0000","last_clean":"2024-05-27T09:27:20.836378+0000","last_became_active":"2024-05-27T09:27:18.562843+0000","last_became_peered":"2024-05-27T09:27:18.562843+0000","last_unstale":"2024-05-27T09:27:20.836378+0000","last_undegraded":"2024-05-27T09:27:20.836378+0000","last_fullsized":"2024-05-27T09:27:20.836378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:33:36.671764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835457+0000","last_change":"2024-05-27T09:27:18.566502+0000","last_active":"2024-05-27T09:27:20.835457+0000","last_peered":"2024-05-27T09:27:20.835457+0000","last_clean":"2024-05-27T09:27:20.835457+0000","last_became_active":"2024-05-27T09:27:18.566108+0000","last_became_peered":"2024-05-27T09:27:18.566108+0000","last_unstale":"2024-05-27T09:27:20.835457+0000","last_undegraded":"2024-05-27T09:27:20.835457+0000","last_fullsized":"2024-05-27T09:27:20.835457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:44:47.447245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836471+0000","last_change":"2024-05-27T09:27:18.582595+0000","last_active":"2024-05-27T09:27:20.836471+0000","last_peered":"2024-05-27T09:27:20.836471+0000","last_clean":"2024-05-27T09:27:20.836471+0000","last_became_active":"2024-05-27T09:27:18.581719+0000","last_became_peered":"2024-05-27T09:27:18.581719+0000","last_unstale":"2024-05-27T09:27:20.836471+0000","last_undegraded":"2024-05-27T09:27:20.836471+0000","last_fullsized":"2024-05-27T09:27:20.836471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:48:19.573654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837234+0000","last_change":"2024-05-27T09:27:18.577565+0000","last_active":"2024-05-27T09:27:20.837234+0000","last_peered":"2024-05-27T09:27:20.837234+0000","last_clean":"2024-05-27T09:27:20.837234+0000","last_became_active":"2024-05-27T09:27:18.577193+0000","last_became_peered":"2024-05-27T09:27:18.577193+0000","last_unstale":"2024-05-27T09:27:20.837234+0000","last_undegraded":"2024-05-27T09:27:20.837234+0000","last_fullsized":"2024-05-27T09:27:20.837234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:43:20.065873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835530+0000","last_change":"2024-05-27T09:27:18.571089+0000","last_active":"2024-05-27T09:27:20.835530+0000","last_peered":"2024-05-27T09:27:20.835530+0000","last_clean":"2024-05-27T09:27:20.835530+0000","last_became_active":"2024-05-27T09:27:18.570712+0000","last_became_peered":"2024-05-27T09:27:18.570712+0000","last_unstale":"2024-05-27T09:27:20.835530+0000","last_undegraded":"2024-05-27T09:27:20.835530+0000","last_fullsized":"2024-05-27T09:27:20.835530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:40:19.310581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837956+0000","last_change":"2024-05-27T09:27:18.572675+0000","last_active":"2024-05-27T09:27:20.837956+0000","last_peered":"2024-05-27T09:27:20.837956+0000","last_clean":"2024-05-27T09:27:20.837956+0000","last_became_active":"2024-05-27T09:27:18.564643+0000","last_became_peered":"2024-05-27T09:27:18.564643+0000","last_unstale":"2024-05-27T09:27:20.837956+0000","last_undegraded":"2024-05-27T09:27:20.837956+0000","last_fullsized":"2024-05-27T09:27:20.837956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:36:15.602241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544179+0000","last_change":"2024-05-27T09:27:18.544554+0000","last_active":"2024-05-27T09:27:19.544179+0000","last_peered":"2024-05-27T09:27:19.544179+0000","last_clean":"2024-05-27T09:27:19.544179+0000","last_became_active":"2024-05-27T09:27:18.543777+0000","last_became_peered":"2024-05-27T09:27:18.543777+0000","last_unstale":"2024-05-27T09:27:19.544179+0000","last_undegraded":"2024-05-27T09:27:19.544179+0000","last_fullsized":"2024-05-27T09:27:19.544179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:35:43.354270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835033+0000","last_change":"2024-05-27T09:27:18.571321+0000","last_active":"2024-05-27T09:27:20.835033+0000","last_peered":"2024-05-27T09:27:20.835033+0000","last_clean":"2024-05-27T09:27:20.835033+0000","last_became_active":"2024-05-27T09:27:18.570948+0000","last_became_peered":"2024-05-27T09:27:18.570948+0000","last_unstale":"2024-05-27T09:27:20.835033+0000","last_undegraded":"2024-05-27T09:27:20.835033+0000","last_fullsized":"2024-05-27T09:27:20.835033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:28:31.470269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539490+0000","last_change":"2024-05-27T09:27:18.544282+0000","last_active":"2024-05-27T09:27:19.539490+0000","last_peered":"2024-05-27T09:27:19.539490+0000","last_clean":"2024-05-27T09:27:19.539490+0000","last_became_active":"2024-05-27T09:27:18.543921+0000","last_became_peered":"2024-05-27T09:27:18.543921+0000","last_unstale":"2024-05-27T09:27:19.539490+0000","last_undegraded":"2024-05-27T09:27:19.539490+0000","last_fullsized":"2024-05-27T09:27:19.539490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:27.225150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835732+0000","last_change":"2024-05-27T09:27:18.575415+0000","last_active":"2024-05-27T09:27:20.835732+0000","last_peered":"2024-05-27T09:27:20.835732+0000","last_clean":"2024-05-27T09:27:20.835732+0000","last_became_active":"2024-05-27T09:27:18.575115+0000","last_became_peered":"2024-05-27T09:27:18.575115+0000","last_unstale":"2024-05-27T09:27:20.835732+0000","last_undegraded":"2024-05-27T09:27:20.835732+0000","last_fullsized":"2024-05-27T09:27:20.835732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:50:48.091125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836496+0000","last_change":"2024-05-27T09:27:18.564422+0000","last_active":"2024-05-27T09:27:20.836496+0000","last_peered":"2024-05-27T09:27:20.836496+0000","last_clean":"2024-05-27T09:27:20.836496+0000","last_became_active":"2024-05-27T09:27:18.563991+0000","last_became_peered":"2024-05-27T09:27:18.563991+0000","last_unstale":"2024-05-27T09:27:20.836496+0000","last_undegraded":"2024-05-27T09:27:20.836496+0000","last_fullsized":"2024-05-27T09:27:20.836496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:19.131343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836032+0000","last_change":"2024-05-27T09:27:18.563557+0000","last_active":"2024-05-27T09:27:20.836032+0000","last_peered":"2024-05-27T09:27:20.836032+0000","last_clean":"2024-05-27T09:27:20.836032+0000","last_became_active":"2024-05-27T09:27:18.562803+0000","last_became_peered":"2024-05-27T09:27:18.562803+0000","last_unstale":"2024-05-27T09:27:20.836032+0000","last_undegraded":"2024-05-27T09:27:20.836032+0000","last_fullsized":"2024-05-27T09:27:20.836032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:47:54.271208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542223+0000","last_change":"2024-05-27T09:27:18.590430+0000","last_active":"2024-05-27T09:27:19.542223+0000","last_peered":"2024-05-27T09:27:19.542223+0000","last_clean":"2024-05-27T09:27:19.542223+0000","last_became_active":"2024-05-27T09:27:18.590083+0000","last_became_peered":"2024-05-27T09:27:18.590083+0000","last_unstale":"2024-05-27T09:27:19.542223+0000","last_undegraded":"2024-05-27T09:27:19.542223+0000","last_fullsized":"2024-05-27T09:27:19.542223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:03.762818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837167+0000","last_change":"2024-05-27T09:27:18.558248+0000","last_active":"2024-05-27T09:27:20.837167+0000","last_peered":"2024-05-27T09:27:20.837167+0000","last_clean":"2024-05-27T09:27:20.837167+0000","last_became_active":"2024-05-27T09:27:18.557953+0000","last_became_peered":"2024-05-27T09:27:18.557953+0000","last_unstale":"2024-05-27T09:27:20.837167+0000","last_undegraded":"2024-05-27T09:27:20.837167+0000","last_fullsized":"2024-05-27T09:27:20.837167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:44:47.447245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836514+0000","last_change":"2024-05-27T09:27:18.563213+0000","last_active":"2024-05-27T09:27:20.836514+0000","last_peered":"2024-05-27T09:27:20.836514+0000","last_clean":"2024-05-27T09:27:20.836514+0000","last_became_active":"2024-05-27T09:27:18.562839+0000","last_became_peered":"2024-05-27T09:27:18.562839+0000","last_unstale":"2024-05-27T09:27:20.836514+0000","last_undegraded":"2024-05-27T09:27:20.836514+0000","last_fullsized":"2024-05-27T09:27:20.836514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:19:43.851698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836053+0000","last_change":"2024-05-27T09:27:18.562237+0000","last_active":"2024-05-27T09:27:20.836053+0000","last_peered":"2024-05-27T09:27:20.836053+0000","last_clean":"2024-05-27T09:27:20.836053+0000","last_became_active":"2024-05-27T09:27:18.561475+0000","last_became_peered":"2024-05-27T09:27:18.561475+0000","last_unstale":"2024-05-27T09:27:20.836053+0000","last_undegraded":"2024-05-27T09:27:20.836053+0000","last_fullsized":"2024-05-27T09:27:20.836053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:39:58.110527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837611+0000","last_change":"2024-05-27T09:27:18.563469+0000","last_active":"2024-05-27T09:27:20.837611+0000","last_peered":"2024-05-27T09:27:20.837611+0000","last_clean":"2024-05-27T09:27:20.837611+0000","last_became_active":"2024-05-27T09:27:18.563103+0000","last_became_peered":"2024-05-27T09:27:18.563103+0000","last_unstale":"2024-05-27T09:27:20.837611+0000","last_undegraded":"2024-05-27T09:27:20.837611+0000","last_fullsized":"2024-05-27T09:27:20.837611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:53:57.281169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.834984+0000","last_change":"2024-05-27T09:27:18.560551+0000","last_active":"2024-05-27T09:27:20.834984+0000","last_peered":"2024-05-27T09:27:20.834984+0000","last_clean":"2024-05-27T09:27:20.834984+0000","last_became_active":"2024-05-27T09:27:18.560244+0000","last_became_peered":"2024-05-27T09:27:18.560244+0000","last_unstale":"2024-05-27T09:27:20.834984+0000","last_undegraded":"2024-05-27T09:27:20.834984+0000","last_fullsized":"2024-05-27T09:27:20.834984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:00.131180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.538536+0000","last_change":"2024-05-27T09:27:18.568200+0000","last_active":"2024-05-27T09:27:19.538536+0000","last_peered":"2024-05-27T09:27:19.538536+0000","last_clean":"2024-05-27T09:27:19.538536+0000","last_became_active":"2024-05-27T09:27:18.567845+0000","last_became_peered":"2024-05-27T09:27:18.567845+0000","last_unstale":"2024-05-27T09:27:19.538536+0000","last_undegraded":"2024-05-27T09:27:19.538536+0000","last_fullsized":"2024-05-27T09:27:19.538536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:54.726617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.540019+0000","last_change":"2024-05-27T09:27:18.560521+0000","last_active":"2024-05-27T09:27:19.540019+0000","last_peered":"2024-05-27T09:27:19.540019+0000","last_clean":"2024-05-27T09:27:19.540019+0000","last_became_active":"2024-05-27T09:27:18.556156+0000","last_became_peered":"2024-05-27T09:27:18.556156+0000","last_unstale":"2024-05-27T09:27:19.540019+0000","last_undegraded":"2024-05-27T09:27:19.540019+0000","last_fullsized":"2024-05-27T09:27:19.540019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:03.762818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.631378+0000","last_change":"2024-05-27T09:27:17.548038+0000","last_active":"2024-05-27T09:27:28.631378+0000","last_peered":"2024-05-27T09:27:28.631378+0000","last_clean":"2024-05-27T09:27:28.631378+0000","last_became_active":"2024-05-27T09:27:17.547653+0000","last_became_peered":"2024-05-27T09:27:17.547653+0000","last_unstale":"2024-05-27T09:27:28.631378+0000","last_undegraded":"2024-05-27T09:27:28.631378+0000","last_fullsized":"2024-05-27T09:27:28.631378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":876,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835265+0000","last_change":"2024-05-27T09:27:18.543048+0000","last_active":"2024-05-27T09:27:20.835265+0000","last_peered":"2024-05-27T09:27:20.835265+0000","last_clean":"2024-05-27T09:27:20.835265+0000","last_became_active":"2024-05-27T09:27:18.542715+0000","last_became_peered":"2024-05-27T09:27:18.542715+0000","last_unstale":"2024-05-27T09:27:20.835265+0000","last_undegraded":"2024-05-27T09:27:20.835265+0000","last_fullsized":"2024-05-27T09:27:20.835265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:46:49.444709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.738235+0000","last_change":"2024-05-27T09:27:17.570014+0000","last_active":"2024-05-27T09:27:33.738235+0000","last_peered":"2024-05-27T09:27:33.738235+0000","last_clean":"2024-05-27T09:27:33.738235+0000","last_became_active":"2024-05-27T09:27:17.569673+0000","last_became_peered":"2024-05-27T09:27:17.569673+0000","last_unstale":"2024-05-27T09:27:33.738235+0000","last_undegraded":"2024-05-27T09:27:33.738235+0000","last_fullsized":"2024-05-27T09:27:33.738235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:26.210041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.738367+0000","last_change":"2024-05-27T09:27:17.565489+0000","last_active":"2024-05-27T09:27:33.738367+0000","last_peered":"2024-05-27T09:27:33.738367+0000","last_clean":"2024-05-27T09:27:33.738367+0000","last_became_active":"2024-05-27T09:27:17.565137+0000","last_became_peered":"2024-05-27T09:27:17.565137+0000","last_unstale":"2024-05-27T09:27:33.738367+0000","last_undegraded":"2024-05-27T09:27:33.738367+0000","last_fullsized":"2024-05-27T09:27:33.738367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836835+0000","last_change":"2024-05-27T09:27:18.565294+0000","last_active":"2024-05-27T09:27:20.836835+0000","last_peered":"2024-05-27T09:27:20.836835+0000","last_clean":"2024-05-27T09:27:20.836835+0000","last_became_active":"2024-05-27T09:27:18.564115+0000","last_became_peered":"2024-05-27T09:27:18.564115+0000","last_unstale":"2024-05-27T09:27:20.836835+0000","last_undegraded":"2024-05-27T09:27:20.836835+0000","last_fullsized":"2024-05-27T09:27:20.836835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:57:20.488849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836816+0000","last_change":"2024-05-27T09:27:17.561204+0000","last_active":"2024-05-27T09:27:20.836816+0000","last_peered":"2024-05-27T09:27:20.836816+0000","last_clean":"2024-05-27T09:27:20.836816+0000","last_became_active":"2024-05-27T09:27:17.560149+0000","last_became_peered":"2024-05-27T09:27:17.560149+0000","last_unstale":"2024-05-27T09:27:20.836816+0000","last_undegraded":"2024-05-27T09:27:20.836816+0000","last_fullsized":"2024-05-27T09:27:20.836816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.284270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836343+0000","last_change":"2024-05-27T09:27:18.542204+0000","last_active":"2024-05-27T09:27:20.836343+0000","last_peered":"2024-05-27T09:27:20.836343+0000","last_clean":"2024-05-27T09:27:20.836343+0000","last_became_active":"2024-05-27T09:27:18.541946+0000","last_became_peered":"2024-05-27T09:27:18.541946+0000","last_unstale":"2024-05-27T09:27:20.836343+0000","last_undegraded":"2024-05-27T09:27:20.836343+0000","last_fullsized":"2024-05-27T09:27:20.836343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:50:48.091125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.639056+0000","last_change":"2024-05-27T09:27:17.561714+0000","last_active":"2024-05-27T09:27:28.639056+0000","last_peered":"2024-05-27T09:27:28.639056+0000","last_clean":"2024-05-27T09:27:28.639056+0000","last_became_active":"2024-05-27T09:27:17.561355+0000","last_became_peered":"2024-05-27T09:27:17.561355+0000","last_unstale":"2024-05-27T09:27:28.639056+0000","last_undegraded":"2024-05-27T09:27:28.639056+0000","last_fullsized":"2024-05-27T09:27:28.639056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:07:02.481451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542128+0000","last_change":"2024-05-27T09:27:18.588199+0000","last_active":"2024-05-27T09:27:19.542128+0000","last_peered":"2024-05-27T09:27:19.542128+0000","last_clean":"2024-05-27T09:27:19.542128+0000","last_became_active":"2024-05-27T09:27:18.587921+0000","last_became_peered":"2024-05-27T09:27:18.587921+0000","last_unstale":"2024-05-27T09:27:19.542128+0000","last_undegraded":"2024-05-27T09:27:19.542128+0000","last_fullsized":"2024-05-27T09:27:19.542128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:17.895211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.620737+0000","last_change":"2024-05-27T09:27:17.565854+0000","last_active":"2024-05-27T09:27:28.620737+0000","last_peered":"2024-05-27T09:27:28.620737+0000","last_clean":"2024-05-27T09:27:28.620737+0000","last_became_active":"2024-05-27T09:27:17.565488+0000","last_became_peered":"2024-05-27T09:27:17.565488+0000","last_unstale":"2024-05-27T09:27:28.620737+0000","last_undegraded":"2024-05-27T09:27:28.620737+0000","last_fullsized":"2024-05-27T09:27:28.620737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:53.711508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836966+0000","last_change":"2024-05-27T09:27:18.585592+0000","last_active":"2024-05-27T09:27:20.836966+0000","last_peered":"2024-05-27T09:27:20.836966+0000","last_clean":"2024-05-27T09:27:20.836966+0000","last_became_active":"2024-05-27T09:27:18.585243+0000","last_became_peered":"2024-05-27T09:27:18.585243+0000","last_unstale":"2024-05-27T09:27:20.836966+0000","last_undegraded":"2024-05-27T09:27:20.836966+0000","last_fullsized":"2024-05-27T09:27:20.836966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:28.872653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.548249+0000","last_change":"2024-05-27T09:27:17.548651+0000","last_active":"2024-05-27T09:27:23.548249+0000","last_peered":"2024-05-27T09:27:23.548249+0000","last_clean":"2024-05-27T09:27:23.548249+0000","last_became_active":"2024-05-27T09:27:17.548307+0000","last_became_peered":"2024-05-27T09:27:17.548307+0000","last_unstale":"2024-05-27T09:27:23.548249+0000","last_undegraded":"2024-05-27T09:27:23.548249+0000","last_fullsized":"2024-05-27T09:27:23.548249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:48.905422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837273+0000","last_change":"2024-05-27T09:27:18.582895+0000","last_active":"2024-05-27T09:27:20.837273+0000","last_peered":"2024-05-27T09:27:20.837273+0000","last_clean":"2024-05-27T09:27:20.837273+0000","last_became_active":"2024-05-27T09:27:18.581742+0000","last_became_peered":"2024-05-27T09:27:18.581742+0000","last_unstale":"2024-05-27T09:27:20.837273+0000","last_undegraded":"2024-05-27T09:27:20.837273+0000","last_fullsized":"2024-05-27T09:27:20.837273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:23:26.888905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.625723+0000","last_change":"2024-05-27T09:27:17.556419+0000","last_active":"2024-05-27T09:27:28.625723+0000","last_peered":"2024-05-27T09:27:28.625723+0000","last_clean":"2024-05-27T09:27:28.625723+0000","last_became_active":"2024-05-27T09:27:17.556036+0000","last_became_peered":"2024-05-27T09:27:17.556036+0000","last_unstale":"2024-05-27T09:27:28.625723+0000","last_undegraded":"2024-05-27T09:27:28.625723+0000","last_fullsized":"2024-05-27T09:27:28.625723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:48:20.718085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544281+0000","last_change":"2024-05-27T09:27:18.568187+0000","last_active":"2024-05-27T09:27:19.544281+0000","last_peered":"2024-05-27T09:27:19.544281+0000","last_clean":"2024-05-27T09:27:19.544281+0000","last_became_active":"2024-05-27T09:27:18.565500+0000","last_became_peered":"2024-05-27T09:27:18.565500+0000","last_unstale":"2024-05-27T09:27:19.544281+0000","last_undegraded":"2024-05-27T09:27:19.544281+0000","last_fullsized":"2024-05-27T09:27:19.544281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:16:08.688048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.734325+0000","last_change":"2024-05-27T09:27:17.572168+0000","last_active":"2024-05-27T09:27:33.734325+0000","last_peered":"2024-05-27T09:27:33.734325+0000","last_clean":"2024-05-27T09:27:33.734325+0000","last_became_active":"2024-05-27T09:27:17.571709+0000","last_became_peered":"2024-05-27T09:27:17.571709+0000","last_unstale":"2024-05-27T09:27:33.734325+0000","last_undegraded":"2024-05-27T09:27:33.734325+0000","last_fullsized":"2024-05-27T09:27:33.734325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836574+0000","last_change":"2024-05-27T09:27:18.564930+0000","last_active":"2024-05-27T09:27:20.836574+0000","last_peered":"2024-05-27T09:27:20.836574+0000","last_clean":"2024-05-27T09:27:20.836574+0000","last_became_active":"2024-05-27T09:27:18.564589+0000","last_became_peered":"2024-05-27T09:27:18.564589+0000","last_unstale":"2024-05-27T09:27:20.836574+0000","last_undegraded":"2024-05-27T09:27:20.836574+0000","last_fullsized":"2024-05-27T09:27:20.836574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:21:00.644551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.650158+0000","last_change":"2024-05-27T09:27:17.561344+0000","last_active":"2024-05-27T09:27:28.650158+0000","last_peered":"2024-05-27T09:27:28.650158+0000","last_clean":"2024-05-27T09:27:28.650158+0000","last_became_active":"2024-05-27T09:27:17.554860+0000","last_became_peered":"2024-05-27T09:27:17.554860+0000","last_unstale":"2024-05-27T09:27:28.650158+0000","last_undegraded":"2024-05-27T09:27:28.650158+0000","last_fullsized":"2024-05-27T09:27:28.650158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:16.880101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837236+0000","last_change":"2024-05-27T09:27:18.575317+0000","last_active":"2024-05-27T09:27:20.837236+0000","last_peered":"2024-05-27T09:27:20.837236+0000","last_clean":"2024-05-27T09:27:20.837236+0000","last_became_active":"2024-05-27T09:27:18.572024+0000","last_became_peered":"2024-05-27T09:27:18.572024+0000","last_unstale":"2024-05-27T09:27:20.837236+0000","last_undegraded":"2024-05-27T09:27:20.837236+0000","last_fullsized":"2024-05-27T09:27:20.837236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:23:35.418697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.739197+0000","last_change":"2024-05-27T09:27:17.575859+0000","last_active":"2024-05-27T09:27:33.739197+0000","last_peered":"2024-05-27T09:27:33.739197+0000","last_clean":"2024-05-27T09:27:33.739197+0000","last_became_active":"2024-05-27T09:27:17.574695+0000","last_became_peered":"2024-05-27T09:27:17.574695+0000","last_unstale":"2024-05-27T09:27:33.739197+0000","last_undegraded":"2024-05-27T09:27:33.739197+0000","last_fullsized":"2024-05-27T09:27:33.739197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:24:19.379819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539060+0000","last_change":"2024-05-27T09:27:18.555137+0000","last_active":"2024-05-27T09:27:19.539060+0000","last_peered":"2024-05-27T09:27:19.539060+0000","last_clean":"2024-05-27T09:27:19.539060+0000","last_became_active":"2024-05-27T09:27:18.554932+0000","last_became_peered":"2024-05-27T09:27:18.554932+0000","last_unstale":"2024-05-27T09:27:19.539060+0000","last_undegraded":"2024-05-27T09:27:19.539060+0000","last_fullsized":"2024-05-27T09:27:19.539060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:06.224637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'15","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.739565+0000","last_change":"2024-05-27T09:27:17.547895+0000","last_active":"2024-05-27T09:27:33.739565+0000","last_peered":"2024-05-27T09:27:33.739565+0000","last_clean":"2024-05-27T09:27:33.739565+0000","last_became_active":"2024-05-27T09:27:17.547523+0000","last_became_peered":"2024-05-27T09:27:17.547523+0000","last_unstale":"2024-05-27T09:27:33.739565+0000","last_undegraded":"2024-05-27T09:27:33.739565+0000","last_fullsized":"2024-05-27T09:27:33.739565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836706+0000","last_change":"2024-05-27T09:27:18.584284+0000","last_active":"2024-05-27T09:27:20.836706+0000","last_peered":"2024-05-27T09:27:20.836706+0000","last_clean":"2024-05-27T09:27:20.836706+0000","last_became_active":"2024-05-27T09:27:18.582360+0000","last_became_peered":"2024-05-27T09:27:18.582360+0000","last_unstale":"2024-05-27T09:27:20.836706+0000","last_undegraded":"2024-05-27T09:27:20.836706+0000","last_fullsized":"2024-05-27T09:27:20.836706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:32.535070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.550902+0000","last_change":"2024-05-27T09:27:17.571481+0000","last_active":"2024-05-27T09:27:23.550902+0000","last_peered":"2024-05-27T09:27:23.550902+0000","last_clean":"2024-05-27T09:27:23.550902+0000","last_became_active":"2024-05-27T09:27:17.571146+0000","last_became_peered":"2024-05-27T09:27:17.571146+0000","last_unstale":"2024-05-27T09:27:23.550902+0000","last_undegraded":"2024-05-27T09:27:23.550902+0000","last_fullsized":"2024-05-27T09:27:23.550902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:18.038221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.542481+0000","last_change":"2024-05-27T09:27:18.580425+0000","last_active":"2024-05-27T09:27:19.542481+0000","last_peered":"2024-05-27T09:27:19.542481+0000","last_clean":"2024-05-27T09:27:19.542481+0000","last_became_active":"2024-05-27T09:27:18.579440+0000","last_became_peered":"2024-05-27T09:27:18.579440+0000","last_unstale":"2024-05-27T09:27:19.542481+0000","last_undegraded":"2024-05-27T09:27:19.542481+0000","last_fullsized":"2024-05-27T09:27:19.542481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:19.053330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.554242+0000","last_change":"2024-05-27T09:27:17.551295+0000","last_active":"2024-05-27T09:27:23.554242+0000","last_peered":"2024-05-27T09:27:23.554242+0000","last_clean":"2024-05-27T09:27:23.554242+0000","last_became_active":"2024-05-27T09:27:17.550861+0000","last_became_peered":"2024-05-27T09:27:17.550861+0000","last_unstale":"2024-05-27T09:27:23.554242+0000","last_undegraded":"2024-05-27T09:27:23.554242+0000","last_fullsized":"2024-05-27T09:27:23.554242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:13.955191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.545103+0000","last_change":"2024-05-27T09:27:18.547199+0000","last_active":"2024-05-27T09:27:19.545103+0000","last_peered":"2024-05-27T09:27:19.545103+0000","last_clean":"2024-05-27T09:27:19.545103+0000","last_became_active":"2024-05-27T09:27:18.543683+0000","last_became_peered":"2024-05-27T09:27:18.543683+0000","last_unstale":"2024-05-27T09:27:19.545103+0000","last_undegraded":"2024-05-27T09:27:19.545103+0000","last_fullsized":"2024-05-27T09:27:19.545103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:48:19.573654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:21.370295+0000","last_change":"2024-05-27T09:27:17.559255+0000","last_active":"2024-05-27T09:27:21.370295+0000","last_peered":"2024-05-27T09:27:21.370295+0000","last_clean":"2024-05-27T09:27:21.370295+0000","last_became_active":"2024-05-27T09:27:17.558973+0000","last_became_peered":"2024-05-27T09:27:17.558973+0000","last_unstale":"2024-05-27T09:27:21.370295+0000","last_undegraded":"2024-05-27T09:27:21.370295+0000","last_fullsized":"2024-05-27T09:27:21.370295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.544602+0000","last_change":"2024-05-27T09:27:18.557423+0000","last_active":"2024-05-27T09:27:19.544602+0000","last_peered":"2024-05-27T09:27:19.544602+0000","last_clean":"2024-05-27T09:27:19.544602+0000","last_became_active":"2024-05-27T09:27:18.557076+0000","last_became_peered":"2024-05-27T09:27:18.557076+0000","last_unstale":"2024-05-27T09:27:19.544602+0000","last_undegraded":"2024-05-27T09:27:19.544602+0000","last_fullsized":"2024-05-27T09:27:19.544602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:11:04.612363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.738040+0000","last_change":"2024-05-27T09:27:17.573700+0000","last_active":"2024-05-27T09:27:33.738040+0000","last_peered":"2024-05-27T09:27:33.738040+0000","last_clean":"2024-05-27T09:27:33.738040+0000","last_became_active":"2024-05-27T09:27:17.573484+0000","last_became_peered":"2024-05-27T09:27:17.573484+0000","last_unstale":"2024-05-27T09:27:33.738040+0000","last_undegraded":"2024-05-27T09:27:33.738040+0000","last_fullsized":"2024-05-27T09:27:33.738040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:48.905422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837609+0000","last_change":"2024-05-27T09:27:18.582777+0000","last_active":"2024-05-27T09:27:20.837609+0000","last_peered":"2024-05-27T09:27:20.837609+0000","last_clean":"2024-05-27T09:27:20.837609+0000","last_became_active":"2024-05-27T09:27:18.574959+0000","last_became_peered":"2024-05-27T09:27:18.574959+0000","last_unstale":"2024-05-27T09:27:20.837609+0000","last_undegraded":"2024-05-27T09:27:20.837609+0000","last_fullsized":"2024-05-27T09:27:20.837609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:40:41.206755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":110,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539210+0000","last_change":"2024-05-27T09:27:12.693423+0000","last_active":"2024-05-27T09:27:19.539210+0000","last_peered":"2024-05-27T09:27:19.539210+0000","last_clean":"2024-05-27T09:27:19.539210+0000","last_became_active":"2024-05-27T09:27:12.690301+0000","last_became_peered":"2024-05-27T09:27:12.690301+0000","last_unstale":"2024-05-27T09:27:19.539210+0000","last_undegraded":"2024-05-27T09:27:19.539210+0000","last_fullsized":"2024-05-27T09:27:19.539210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:11.463226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:11.463226+0000","last_clean_scrub_stamp":"2024-05-27T09:27:11.463226+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:01.810071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.650695+0000","last_change":"2024-05-27T09:27:17.567670+0000","last_active":"2024-05-27T09:27:28.650695+0000","last_peered":"2024-05-27T09:27:28.650695+0000","last_clean":"2024-05-27T09:27:28.650695+0000","last_became_active":"2024-05-27T09:27:17.566292+0000","last_became_peered":"2024-05-27T09:27:17.566292+0000","last_unstale":"2024-05-27T09:27:28.650695+0000","last_undegraded":"2024-05-27T09:27:28.650695+0000","last_fullsized":"2024-05-27T09:27:28.650695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:18.038221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836742+0000","last_change":"2024-05-27T09:27:18.571124+0000","last_active":"2024-05-27T09:27:20.836742+0000","last_peered":"2024-05-27T09:27:20.836742+0000","last_clean":"2024-05-27T09:27:20.836742+0000","last_became_active":"2024-05-27T09:27:18.570782+0000","last_became_peered":"2024-05-27T09:27:18.570782+0000","last_unstale":"2024-05-27T09:27:20.836742+0000","last_undegraded":"2024-05-27T09:27:20.836742+0000","last_fullsized":"2024-05-27T09:27:20.836742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:39:58.110527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.734194+0000","last_change":"2024-05-27T09:27:17.563259+0000","last_active":"2024-05-27T09:27:33.734194+0000","last_peered":"2024-05-27T09:27:33.734194+0000","last_clean":"2024-05-27T09:27:33.734194+0000","last_became_active":"2024-05-27T09:27:17.563010+0000","last_became_peered":"2024-05-27T09:27:17.563010+0000","last_unstale":"2024-05-27T09:27:33.734194+0000","last_undegraded":"2024-05-27T09:27:33.734194+0000","last_fullsized":"2024-05-27T09:27:33.734194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:53.711508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836371+0000","last_change":"2024-05-27T09:27:18.574466+0000","last_active":"2024-05-27T09:27:20.836371+0000","last_peered":"2024-05-27T09:27:20.836371+0000","last_clean":"2024-05-27T09:27:20.836371+0000","last_became_active":"2024-05-27T09:27:18.574090+0000","last_became_peered":"2024-05-27T09:27:18.574090+0000","last_unstale":"2024-05-27T09:27:20.836371+0000","last_undegraded":"2024-05-27T09:27:20.836371+0000","last_fullsized":"2024-05-27T09:27:20.836371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:41:22.515484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.737344+0000","last_change":"2024-05-27T09:27:17.551310+0000","last_active":"2024-05-27T09:27:33.737344+0000","last_peered":"2024-05-27T09:27:33.737344+0000","last_clean":"2024-05-27T09:27:33.737344+0000","last_became_active":"2024-05-27T09:27:17.550982+0000","last_became_peered":"2024-05-27T09:27:17.550982+0000","last_unstale":"2024-05-27T09:27:33.737344+0000","last_undegraded":"2024-05-27T09:27:33.737344+0000","last_fullsized":"2024-05-27T09:27:33.737344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836574+0000","last_change":"2024-05-27T09:27:18.559416+0000","last_active":"2024-05-27T09:27:20.836574+0000","last_peered":"2024-05-27T09:27:20.836574+0000","last_clean":"2024-05-27T09:27:20.836574+0000","last_became_active":"2024-05-27T09:27:18.559017+0000","last_became_peered":"2024-05-27T09:27:18.559017+0000","last_unstale":"2024-05-27T09:27:20.836574+0000","last_undegraded":"2024-05-27T09:27:20.836574+0000","last_fullsized":"2024-05-27T09:27:20.836574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:33:48.400306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837736+0000","last_change":"2024-05-27T09:27:17.563947+0000","last_active":"2024-05-27T09:27:20.837736+0000","last_peered":"2024-05-27T09:27:20.837736+0000","last_clean":"2024-05-27T09:27:20.837736+0000","last_became_active":"2024-05-27T09:27:17.563640+0000","last_became_peered":"2024-05-27T09:27:17.563640+0000","last_unstale":"2024-05-27T09:27:20.837736+0000","last_undegraded":"2024-05-27T09:27:20.837736+0000","last_fullsized":"2024-05-27T09:27:20.837736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:02.747708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.543615+0000","last_change":"2024-05-27T09:27:18.575835+0000","last_active":"2024-05-27T09:27:19.543615+0000","last_peered":"2024-05-27T09:27:19.543615+0000","last_clean":"2024-05-27T09:27:19.543615+0000","last_became_active":"2024-05-27T09:27:18.574710+0000","last_became_peered":"2024-05-27T09:27:18.574710+0000","last_unstale":"2024-05-27T09:27:19.543615+0000","last_undegraded":"2024-05-27T09:27:19.543615+0000","last_fullsized":"2024-05-27T09:27:19.543615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:16:00.934068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835781+0000","last_change":"2024-05-27T09:27:17.550749+0000","last_active":"2024-05-27T09:27:20.835781+0000","last_peered":"2024-05-27T09:27:20.835781+0000","last_clean":"2024-05-27T09:27:20.835781+0000","last_became_active":"2024-05-27T09:27:17.550378+0000","last_became_peered":"2024-05-27T09:27:17.550378+0000","last_unstale":"2024-05-27T09:27:20.835781+0000","last_undegraded":"2024-05-27T09:27:20.835781+0000","last_fullsized":"2024-05-27T09:27:20.835781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:20.167629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.541045+0000","last_change":"2024-05-27T09:27:18.567233+0000","last_active":"2024-05-27T09:27:19.541045+0000","last_peered":"2024-05-27T09:27:19.541045+0000","last_clean":"2024-05-27T09:27:19.541045+0000","last_became_active":"2024-05-27T09:27:18.566326+0000","last_became_peered":"2024-05-27T09:27:18.566326+0000","last_unstale":"2024-05-27T09:27:19.541045+0000","last_undegraded":"2024-05-27T09:27:19.541045+0000","last_fullsized":"2024-05-27T09:27:19.541045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:27.225150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837888+0000","last_change":"2024-05-27T09:27:17.557912+0000","last_active":"2024-05-27T09:27:20.837888+0000","last_peered":"2024-05-27T09:27:20.837888+0000","last_clean":"2024-05-27T09:27:20.837888+0000","last_became_active":"2024-05-27T09:27:17.557216+0000","last_became_peered":"2024-05-27T09:27:17.557216+0000","last_unstale":"2024-05-27T09:27:20.837888+0000","last_undegraded":"2024-05-27T09:27:20.837888+0000","last_fullsized":"2024-05-27T09:27:20.837888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:21:22.020724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836847+0000","last_change":"2024-05-27T09:27:18.563649+0000","last_active":"2024-05-27T09:27:20.836847+0000","last_peered":"2024-05-27T09:27:20.836847+0000","last_clean":"2024-05-27T09:27:20.836847+0000","last_became_active":"2024-05-27T09:27:18.562565+0000","last_became_peered":"2024-05-27T09:27:18.562565+0000","last_unstale":"2024-05-27T09:27:20.836847+0000","last_undegraded":"2024-05-27T09:27:20.836847+0000","last_fullsized":"2024-05-27T09:27:20.836847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:46:49.444709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.734132+0000","last_change":"2024-05-27T09:27:17.571911+0000","last_active":"2024-05-27T09:27:33.734132+0000","last_peered":"2024-05-27T09:27:33.734132+0000","last_clean":"2024-05-27T09:27:33.734132+0000","last_became_active":"2024-05-27T09:27:17.571551+0000","last_became_peered":"2024-05-27T09:27:17.571551+0000","last_unstale":"2024-05-27T09:27:33.734132+0000","last_undegraded":"2024-05-27T09:27:33.734132+0000","last_fullsized":"2024-05-27T09:27:33.734132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:20.167629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835899+0000","last_change":"2024-05-27T09:27:18.558780+0000","last_active":"2024-05-27T09:27:20.835899+0000","last_peered":"2024-05-27T09:27:20.835899+0000","last_clean":"2024-05-27T09:27:20.835899+0000","last_became_active":"2024-05-27T09:27:18.558463+0000","last_became_peered":"2024-05-27T09:27:18.558463+0000","last_unstale":"2024-05-27T09:27:20.835899+0000","last_undegraded":"2024-05-27T09:27:20.835899+0000","last_fullsized":"2024-05-27T09:27:20.835899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:42:38.351682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.646698+0000","last_change":"2024-05-27T09:27:17.567973+0000","last_active":"2024-05-27T09:27:28.646698+0000","last_peered":"2024-05-27T09:27:28.646698+0000","last_clean":"2024-05-27T09:27:28.646698+0000","last_became_active":"2024-05-27T09:27:17.567650+0000","last_became_peered":"2024-05-27T09:27:17.567650+0000","last_unstale":"2024-05-27T09:27:28.646698+0000","last_undegraded":"2024-05-27T09:27:28.646698+0000","last_fullsized":"2024-05-27T09:27:28.646698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:25:52.924185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835408+0000","last_change":"2024-05-27T09:27:18.554845+0000","last_active":"2024-05-27T09:27:20.835408+0000","last_peered":"2024-05-27T09:27:20.835408+0000","last_clean":"2024-05-27T09:27:20.835408+0000","last_became_active":"2024-05-27T09:27:18.554462+0000","last_became_peered":"2024-05-27T09:27:18.554462+0000","last_unstale":"2024-05-27T09:27:20.835408+0000","last_undegraded":"2024-05-27T09:27:20.835408+0000","last_fullsized":"2024-05-27T09:27:20.835408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:44:47.447245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.625211+0000","last_change":"2024-05-27T09:27:17.570300+0000","last_active":"2024-05-27T09:27:28.625211+0000","last_peered":"2024-05-27T09:27:28.625211+0000","last_clean":"2024-05-27T09:27:28.625211+0000","last_became_active":"2024-05-27T09:27:17.569685+0000","last_became_peered":"2024-05-27T09:27:17.569685+0000","last_unstale":"2024-05-27T09:27:28.625211+0000","last_undegraded":"2024-05-27T09:27:28.625211+0000","last_fullsized":"2024-05-27T09:27:28.625211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:26.210041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836430+0000","last_change":"2024-05-27T09:27:18.562774+0000","last_active":"2024-05-27T09:27:20.836430+0000","last_peered":"2024-05-27T09:27:20.836430+0000","last_clean":"2024-05-27T09:27:20.836430+0000","last_became_active":"2024-05-27T09:27:18.562455+0000","last_became_peered":"2024-05-27T09:27:18.562455+0000","last_unstale":"2024-05-27T09:27:20.836430+0000","last_undegraded":"2024-05-27T09:27:20.836430+0000","last_fullsized":"2024-05-27T09:27:20.836430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:39:59.034873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.627233+0000","last_change":"2024-05-27T09:27:17.563172+0000","last_active":"2024-05-27T09:27:28.627233+0000","last_peered":"2024-05-27T09:27:28.627233+0000","last_clean":"2024-05-27T09:27:28.627233+0000","last_became_active":"2024-05-27T09:27:17.562941+0000","last_became_peered":"2024-05-27T09:27:17.562941+0000","last_unstale":"2024-05-27T09:27:28.627233+0000","last_undegraded":"2024-05-27T09:27:28.627233+0000","last_fullsized":"2024-05-27T09:27:28.627233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:51.742078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836477+0000","last_change":"2024-05-27T09:27:18.555617+0000","last_active":"2024-05-27T09:27:20.836477+0000","last_peered":"2024-05-27T09:27:20.836477+0000","last_clean":"2024-05-27T09:27:20.836477+0000","last_became_active":"2024-05-27T09:27:18.555280+0000","last_became_peered":"2024-05-27T09:27:18.555280+0000","last_unstale":"2024-05-27T09:27:20.836477+0000","last_undegraded":"2024-05-27T09:27:20.836477+0000","last_fullsized":"2024-05-27T09:27:20.836477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:32:39.043268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:21.367453+0000","last_change":"2024-05-27T09:27:17.548266+0000","last_active":"2024-05-27T09:27:21.367453+0000","last_peered":"2024-05-27T09:27:21.367453+0000","last_clean":"2024-05-27T09:27:21.367453+0000","last_became_active":"2024-05-27T09:27:17.547938+0000","last_became_peered":"2024-05-27T09:27:17.547938+0000","last_unstale":"2024-05-27T09:27:21.367453+0000","last_undegraded":"2024-05-27T09:27:21.367453+0000","last_fullsized":"2024-05-27T09:27:21.367453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.835677+0000","last_change":"2024-05-27T09:27:18.563586+0000","last_active":"2024-05-27T09:27:20.835677+0000","last_peered":"2024-05-27T09:27:20.835677+0000","last_clean":"2024-05-27T09:27:20.835677+0000","last_became_active":"2024-05-27T09:27:18.563227+0000","last_became_peered":"2024-05-27T09:27:18.563227+0000","last_unstale":"2024-05-27T09:27:20.835677+0000","last_undegraded":"2024-05-27T09:27:20.835677+0000","last_fullsized":"2024-05-27T09:27:20.835677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:38:53.222040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:28.647667+0000","last_change":"2024-05-27T09:27:17.562200+0000","last_active":"2024-05-27T09:27:28.647667+0000","last_peered":"2024-05-27T09:27:28.647667+0000","last_clean":"2024-05-27T09:27:28.647667+0000","last_became_active":"2024-05-27T09:27:17.561857+0000","last_became_peered":"2024-05-27T09:27:17.561857+0000","last_unstale":"2024-05-27T09:27:28.647667+0000","last_undegraded":"2024-05-27T09:27:28.647667+0000","last_fullsized":"2024-05-27T09:27:28.647667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:05.209528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836936+0000","last_change":"2024-05-27T09:27:18.572676+0000","last_active":"2024-05-27T09:27:20.836936+0000","last_peered":"2024-05-27T09:27:20.836936+0000","last_clean":"2024-05-27T09:27:20.836936+0000","last_became_active":"2024-05-27T09:27:18.572339+0000","last_became_peered":"2024-05-27T09:27:18.572339+0000","last_unstale":"2024-05-27T09:27:20.836936+0000","last_undegraded":"2024-05-27T09:27:20.836936+0000","last_fullsized":"2024-05-27T09:27:20.836936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:09:23.795528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.837085+0000","last_change":"2024-05-27T09:27:18.575394+0000","last_active":"2024-05-27T09:27:20.837085+0000","last_peered":"2024-05-27T09:27:20.837085+0000","last_clean":"2024-05-27T09:27:20.837085+0000","last_became_active":"2024-05-27T09:27:18.572307+0000","last_became_peered":"2024-05-27T09:27:18.572307+0000","last_unstale":"2024-05-27T09:27:20.837085+0000","last_undegraded":"2024-05-27T09:27:20.837085+0000","last_fullsized":"2024-05-27T09:27:20.837085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:29:05.116582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.555603+0000","last_change":"2024-05-27T09:27:17.570634+0000","last_active":"2024-05-27T09:27:23.555603+0000","last_peered":"2024-05-27T09:27:23.555603+0000","last_clean":"2024-05-27T09:27:23.555603+0000","last_became_active":"2024-05-27T09:27:17.569866+0000","last_became_peered":"2024-05-27T09:27:17.569866+0000","last_unstale":"2024-05-27T09:27:23.555603+0000","last_undegraded":"2024-05-27T09:27:23.555603+0000","last_fullsized":"2024-05-27T09:27:23.555603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:21:38.268535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:19.539980+0000","last_change":"2024-05-27T09:27:18.549339+0000","last_active":"2024-05-27T09:27:19.539980+0000","last_peered":"2024-05-27T09:27:19.539980+0000","last_clean":"2024-05-27T09:27:19.539980+0000","last_became_active":"2024-05-27T09:27:18.549035+0000","last_became_peered":"2024-05-27T09:27:18.549035+0000","last_unstale":"2024-05-27T09:27:19.539980+0000","last_undegraded":"2024-05-27T09:27:19.539980+0000","last_fullsized":"2024-05-27T09:27:19.539980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:17.895211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.737950+0000","last_change":"2024-05-27T09:27:17.568143+0000","last_active":"2024-05-27T09:27:33.737950+0000","last_peered":"2024-05-27T09:27:33.737950+0000","last_clean":"2024-05-27T09:27:33.737950+0000","last_became_active":"2024-05-27T09:27:17.567235+0000","last_became_peered":"2024-05-27T09:27:17.567235+0000","last_unstale":"2024-05-27T09:27:33.737950+0000","last_undegraded":"2024-05-27T09:27:33.737950+0000","last_fullsized":"2024-05-27T09:27:33.737950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.284270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836871+0000","last_change":"2024-05-27T09:27:18.563270+0000","last_active":"2024-05-27T09:27:20.836871+0000","last_peered":"2024-05-27T09:27:20.836871+0000","last_clean":"2024-05-27T09:27:20.836871+0000","last_became_active":"2024-05-27T09:27:18.562919+0000","last_became_peered":"2024-05-27T09:27:18.562919+0000","last_unstale":"2024-05-27T09:27:20.836871+0000","last_undegraded":"2024-05-27T09:27:20.836871+0000","last_fullsized":"2024-05-27T09:27:20.836871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:55:47.799529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:23.554180+0000","last_change":"2024-05-27T09:27:17.551193+0000","last_active":"2024-05-27T09:27:23.554180+0000","last_peered":"2024-05-27T09:27:23.554180+0000","last_clean":"2024-05-27T09:27:23.554180+0000","last_became_active":"2024-05-27T09:27:17.550849+0000","last_became_peered":"2024-05-27T09:27:17.550849+0000","last_unstale":"2024-05-27T09:27:23.554180+0000","last_undegraded":"2024-05-27T09:27:23.554180+0000","last_fullsized":"2024-05-27T09:27:23.554180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:51.742078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:20.836183+0000","last_change":"2024-05-27T09:27:18.579803+0000","last_active":"2024-05-27T09:27:20.836183+0000","last_peered":"2024-05-27T09:27:20.836183+0000","last_clean":"2024-05-27T09:27:20.836183+0000","last_became_active":"2024-05-27T09:27:18.579574+0000","last_became_peered":"2024-05-27T09:27:18.579574+0000","last_unstale":"2024-05-27T09:27:20.836183+0000","last_undegraded":"2024-05-27T09:27:20.836183+0000","last_fullsized":"2024-05-27T09:27:20.836183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:11:22.320778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'9","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-27T09:27:33.737701+0000","last_change":"2024-05-27T09:27:17.549711+0000","last_active":"2024-05-27T09:27:33.737701+0000","last_peered":"2024-05-27T09:27:33.737701+0000","last_clean":"2024-05-27T09:27:33.737701+0000","last_became_active":"2024-05-27T09:27:17.549291+0000","last_became_peered":"2024-05-27T09:27:17.549291+0000","last_unstale":"2024-05-27T09:27:33.737701+0000","last_undegraded":"2024-05-27T09:27:33.737701+0000","last_fullsized":"2024-05-27T09:27:33.737701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:16:22.057803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5476,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":11206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65078,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":225280,"data_stored":73901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":225280,"data_stored":74321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":663203,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":73991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542154,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":73583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":74133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542154,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":664101,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":76109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T09:27:48.641 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:27:48.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.926+0000 7fcd64800640 1 Processor -- start 2024-05-27T09:27:48.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.926+0000 7fcd64800640 1 -- start start 2024-05-27T09:27:48.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.928+0000 7fcd64800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd5c06d700 0x7fcd5c156730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:48.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.929+0000 7fcd64800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fcd5c0ccf10 0x7fcd5c0cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:48.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.929+0000 7fcd64800640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd5c156c70 con 0x7fcd5c0cdc30 2024-05-27T09:27:48.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.929+0000 7fcd64800640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fcd5c160a20 con 0x7fcd5c0ccf10 2024-05-27T09:27:48.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.929+0000 7fcd64800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fcd5c06dd00 con 0x7fcd5c06d700 2024-05-27T09:27:48.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.929+0000 7fcd62800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fcd5c0ccf10 0x7fcd5c0cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:48.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.929+0000 7fcd61e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd5c06d700 0x7fcd5c156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:48.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.929+0000 7fcd61400640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fcd5c0cdc30 0x7fcd5c160310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:41236/0 (socket says 172.21.15.114:41236) 2024-05-27T09:27:48.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.929+0000 7fcd61400640 1 -- 172.21.15.114:0/3106627155 learned_addr learned my addr 172.21.15.114:0/3106627155 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:48.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.929+0000 7fcd60a00640 1 -- 172.21.15.114:0/3106627155 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4041549554 0 0) 0x7fcd4c001110 con 0x7fcd5c0cdc30 2024-05-27T09:27:48.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.929+0000 7fcd60a00640 1 -- 172.21.15.114:0/3106627155 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd44003520 con 0x7fcd5c0cdc30 2024-05-27T09:27:48.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.929+0000 7fcd62800640 1 -- 172.21.15.114:0/3106627155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd5c06d700 msgr2=0x7fcd5c156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:27:48.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.930+0000 7fcd62800640 1 --2- 172.21.15.114:0/3106627155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd5c06d700 0x7fcd5c156730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:48.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.930+0000 7fcd62800640 1 -- 172.21.15.114:0/3106627155 >> v1:172.21.15.163:6789/0 conn(0x7fcd5c0cdc30 legacy=0x7fcd5c160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:48.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.930+0000 7fcd62800640 1 -- 172.21.15.114:0/3106627155 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5c06dfe0 con 0x7fcd5c0ccf10 2024-05-27T09:27:48.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.930+0000 7fcd61e00640 1 --2- 172.21.15.114:0/3106627155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd5c06d700 0x7fcd5c156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:48.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.931+0000 7fcd62800640 1 --2- 172.21.15.114:0/3106627155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd5c0ccf10 0x7fcd5c0cd320 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009d90 tx=0x7fcd5802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90836755040fb84 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:48.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.931+0000 7fcd60a00640 1 -- 172.21.15.114:0/3106627155 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58007740 con 0x7fcd5c0ccf10 2024-05-27T09:27:48.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.931+0000 7fcd60a00640 1 -- 172.21.15.114:0/3106627155 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd580078e0 con 0x7fcd5c0ccf10 2024-05-27T09:27:48.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.931+0000 7fcd60a00640 1 -- 172.21.15.114:0/3106627155 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58007cb0 con 0x7fcd5c0ccf10 2024-05-27T09:27:48.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.932+0000 7fcd64800640 1 -- 172.21.15.114:0/3106627155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd5c0ccf10 msgr2=0x7fcd5c0cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:48.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.932+0000 7fcd64800640 1 --2- 172.21.15.114:0/3106627155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd5c0ccf10 0x7fcd5c0cd320 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009d90 tx=0x7fcd5802a6f0 comp rx=0 tx=0).stop 2024-05-27T09:27:48.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.932+0000 7fcd64800640 1 -- 172.21.15.114:0/3106627155 shutdown_connections 2024-05-27T09:27:48.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.932+0000 7fcd64800640 1 --2- 172.21.15.114:0/3106627155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd5c0ccf10 0x7fcd5c0cd320 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:48.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.932+0000 7fcd64800640 1 --2- 172.21.15.114:0/3106627155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd5c06d700 0x7fcd5c156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:48.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.932+0000 7fcd64800640 1 -- 172.21.15.114:0/3106627155 >> 172.21.15.114:0/3106627155 conn(0x7fcd5c08dd80 msgr2=0x7fcd5c08e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:48.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.932+0000 7fcd64800640 1 -- 172.21.15.114:0/3106627155 shutdown_connections 2024-05-27T09:27:48.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.932+0000 7fcd64800640 1 -- 172.21.15.114:0/3106627155 wait complete. 2024-05-27T09:27:48.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.934+0000 7fcd64800640 1 Processor -- start 2024-05-27T09:27:48.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.934+0000 7fcd64800640 1 -- start start 2024-05-27T09:27:48.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.935+0000 7fcd64800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd5c06d700 0x7fcd5c148420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:48.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.935+0000 7fcd64800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd5c0ccf10 0x7fcd5c1f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:48.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.935+0000 7fcd64800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd5c0cdc30 0x7fcd5c154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:48.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.935+0000 7fcd64800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c15fb10 con 0x7fcd5c0ccf10 2024-05-27T09:27:48.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.935+0000 7fcd64800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c06e5a0 con 0x7fcd5c06d700 2024-05-27T09:27:48.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.935+0000 7fcd64800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fcd5c15fdf0 con 0x7fcd5c0cdc30 2024-05-27T09:27:48.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.936+0000 7fcd61400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd5c0ccf10 0x7fcd5c1f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:48.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.936+0000 7fcd61400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd5c0ccf10 0x7fcd5c1f2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38044/0 (socket says 172.21.15.114:38044) 2024-05-27T09:27:48.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.936+0000 7fcd61400640 1 -- 172.21.15.114:0/1798649012 learned_addr learned my addr 172.21.15.114:0/1798649012 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:48.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.936+0000 7fcd62800640 1 --2- 172.21.15.114:0/1798649012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd5c0cdc30 0x7fcd5c154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:48.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.936+0000 7fcd61e00640 1 --2- 172.21.15.114:0/1798649012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd5c06d700 0x7fcd5c148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:48.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.936+0000 7fcd61400640 1 -- 172.21.15.114:0/1798649012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd5c0cdc30 msgr2=0x7fcd5c154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:48.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.936+0000 7fcd61400640 1 --2- 172.21.15.114:0/1798649012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd5c0cdc30 0x7fcd5c154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:48.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.936+0000 7fcd61400640 1 -- 172.21.15.114:0/1798649012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd5c06d700 msgr2=0x7fcd5c148420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:48.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.936+0000 7fcd61400640 1 --2- 172.21.15.114:0/1798649012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd5c06d700 0x7fcd5c148420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:48.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.936+0000 7fcd61400640 1 -- 172.21.15.114:0/1798649012 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5c1f2fc0 con 0x7fcd5c0ccf10 2024-05-27T09:27:48.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.937+0000 7fcd61400640 1 --2- 172.21.15.114:0/1798649012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd5c0ccf10 0x7fcd5c1f2a80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00c730 tx=0x7fcd4c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:48.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.937+0000 7fcd56000640 1 -- 172.21.15.114:0/1798649012 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c017070 con 0x7fcd5c0ccf10 2024-05-27T09:27:48.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.937+0000 7fcd56000640 1 -- 172.21.15.114:0/1798649012 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd4c005ab0 con 0x7fcd5c0ccf10 2024-05-27T09:27:48.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.937+0000 7fcd64800640 1 -- 172.21.15.114:0/1798649012 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5c150140 con 0x7fcd5c0ccf10 2024-05-27T09:27:48.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.937+0000 7fcd56000640 1 -- 172.21.15.114:0/1798649012 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c001d10 con 0x7fcd5c0ccf10 2024-05-27T09:27:48.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.937+0000 7fcd64800640 1 -- 172.21.15.114:0/1798649012 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd5c150700 con 0x7fcd5c0ccf10 2024-05-27T09:27:48.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.938+0000 7fcd64800640 1 -- 172.21.15.114:0/1798649012 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd5c06a380 con 0x7fcd5c0ccf10 2024-05-27T09:27:48.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.939+0000 7fcd56000640 1 -- 172.21.15.114:0/1798649012 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcd4c01a070 con 0x7fcd5c0ccf10 2024-05-27T09:27:48.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.941+0000 7fcd56000640 1 --2- 172.21.15.114:0/1798649012 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcd40076300 0x7fcd40078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:48.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.941+0000 7fcd56000640 1 -- 172.21.15.114:0/1798649012 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcd4c096810 con 0x7fcd5c0ccf10 2024-05-27T09:27:48.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.941+0000 7fcd61e00640 1 --2- 172.21.15.114:0/1798649012 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcd40076300 0x7fcd40078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:48.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.942+0000 7fcd61e00640 1 --2- 172.21.15.114:0/1798649012 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcd40076300 0x7fcd40078810 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009560 tx=0x7fcd50009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:48.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:48.945+0000 7fcd56000640 1 -- 172.21.15.114:0/1798649012 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcd5c156c70 con 0x7fcd5c0ccf10 2024-05-27T09:27:49.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.179+0000 7fcd64800640 1 -- 172.21.15.114:0/1798649012 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd5c1f2730 con 0x7fcd5c0ccf10 2024-05-27T09:27:49.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.181+0000 7fcd56000640 1 -- 172.21.15.114:0/1798649012 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+6877 (secure 0 0 0) 0x7fcd4c068150 con 0x7fcd5c0ccf10 2024-05-27T09:27:49.182 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:49.182 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:27:33.841352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"f","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6839/444766663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":444766663},{"type":"v1","addr":"172.21.15.163:6839","nonce":444766663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4279,4297,4300]},"id":1}]} 2024-05-27T09:27:49.182 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 12 2024-05-27T09:27:49.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.184+0000 7fcd64800640 1 -- 172.21.15.114:0/1798649012 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcd40076300 msgr2=0x7fcd40078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:49.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.184+0000 7fcd64800640 1 --2- 172.21.15.114:0/1798649012 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcd40076300 0x7fcd40078810 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009560 tx=0x7fcd50009190 comp rx=0 tx=0).stop 2024-05-27T09:27:49.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.184+0000 7fcd64800640 1 -- 172.21.15.114:0/1798649012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd5c0ccf10 msgr2=0x7fcd5c1f2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:49.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.184+0000 7fcd64800640 1 --2- 172.21.15.114:0/1798649012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd5c0ccf10 0x7fcd5c1f2a80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00c730 tx=0x7fcd4c00cbf0 comp rx=0 tx=0).stop 2024-05-27T09:27:49.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.184+0000 7fcd64800640 1 -- 172.21.15.114:0/1798649012 shutdown_connections 2024-05-27T09:27:49.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.184+0000 7fcd64800640 1 --2- 172.21.15.114:0/1798649012 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcd40076300 0x7fcd40078810 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:49.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.184+0000 7fcd64800640 1 --2- 172.21.15.114:0/1798649012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd5c0cdc30 0x7fcd5c154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:49.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.184+0000 7fcd64800640 1 --2- 172.21.15.114:0/1798649012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd5c0ccf10 0x7fcd5c1f2a80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:49.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.184+0000 7fcd64800640 1 --2- 172.21.15.114:0/1798649012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd5c06d700 0x7fcd5c148420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:49.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.184+0000 7fcd64800640 1 -- 172.21.15.114:0/1798649012 >> 172.21.15.114:0/1798649012 conn(0x7fcd5c08dd80 msgr2=0x7fcd5c0b6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:49.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.184+0000 7fcd64800640 1 -- 172.21.15.114:0/1798649012 shutdown_connections 2024-05-27T09:27:49.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.184+0000 7fcd64800640 1 -- 172.21.15.114:0/1798649012 wait complete. 2024-05-27T09:27:49.210 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T09:27:49.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.468+0000 7ffbc9600640 1 Processor -- start 2024-05-27T09:27:49.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.468+0000 7ffbc9600640 1 -- start start 2024-05-27T09:27:49.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.470+0000 7ffbc9600640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffbc4074a80 0x7ffbc415cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:49.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.471+0000 7ffbc9600640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbc40c8470 con 0x7ffbc4156880 2024-05-27T09:27:49.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.471+0000 7ffbc9600640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbc40c8dd0 con 0x7ffbc4070060 2024-05-27T09:27:49.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.471+0000 7ffbc9600640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7ffbc40c8220 con 0x7ffbc4074a80 2024-05-27T09:27:49.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.471+0000 7ffbbb400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7ffbc4156880 0x7ffbc406edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:45404/0 (socket says 172.21.15.114:45404) 2024-05-27T09:27:49.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.471+0000 7ffbbb400640 1 -- 172.21.15.114:0/3241012510 learned_addr learned my addr 172.21.15.114:0/3241012510 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:27:49.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.471+0000 7ffbc3e00640 1 --2- 172.21.15.114:0/3241012510 >> v2:172.21.15.163:3300/0 conn(0x7ffbc4074a80 0x7ffbc415cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:49.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.472+0000 7ffbc3400640 1 -- 172.21.15.114:0/3241012510 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1117469576 0 0) 0x7ffbc40c8470 con 0x7ffbc4156880 2024-05-27T09:27:49.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.472+0000 7ffbc3400640 1 -- 172.21.15.114:0/3241012510 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffba8003520 con 0x7ffbc4156880 2024-05-27T09:27:49.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.472+0000 7ffbc3400640 1 -- 172.21.15.114:0/3241012510 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2632384070 0 0) 0x7ffbc40c8dd0 con 0x7ffbc4070060 2024-05-27T09:27:49.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.472+0000 7ffbc3e00640 1 -- 172.21.15.114:0/3241012510 >> v1:172.21.15.163:6789/0 conn(0x7ffbc4070060 legacy=0x7ffbc4166de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:49.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.472+0000 7ffbc3e00640 1 -- 172.21.15.114:0/3241012510 >> v1:172.21.15.114:6789/0 conn(0x7ffbc4156880 legacy=0x7ffbc406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:49.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.472+0000 7ffbc3e00640 1 -- 172.21.15.114:0/3241012510 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc415c570 con 0x7ffbc4074a80 2024-05-27T09:27:49.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.473+0000 7ffbc3e00640 1 --2- 172.21.15.114:0/3241012510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbc4074a80 0x7ffbc415cd70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40097a0 tx=0x7ffbb402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f7e81b3d6f3092d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:49.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.474+0000 7ffbc3400640 1 -- 172.21.15.114:0/3241012510 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4037070 con 0x7ffbc4074a80 2024-05-27T09:27:49.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.474+0000 7ffbc3400640 1 -- 172.21.15.114:0/3241012510 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb4032440 con 0x7ffbc4074a80 2024-05-27T09:27:49.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.474+0000 7ffbc3400640 1 -- 172.21.15.114:0/3241012510 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb40327f0 con 0x7ffbc4074a80 2024-05-27T09:27:49.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.474+0000 7ffbc9600640 1 -- 172.21.15.114:0/3241012510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbc4074a80 msgr2=0x7ffbc415cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:49.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.474+0000 7ffbc9600640 1 --2- 172.21.15.114:0/3241012510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbc4074a80 0x7ffbc415cd70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40097a0 tx=0x7ffbb402a230 comp rx=0 tx=0).stop 2024-05-27T09:27:49.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.474+0000 7ffbc9600640 1 -- 172.21.15.114:0/3241012510 shutdown_connections 2024-05-27T09:27:49.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.474+0000 7ffbc9600640 1 --2- 172.21.15.114:0/3241012510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbc4074a80 0x7ffbc415cd70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:49.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.474+0000 7ffbc9600640 1 -- 172.21.15.114:0/3241012510 >> 172.21.15.114:0/3241012510 conn(0x7ffbc40ca320 msgr2=0x7ffbc40ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:49.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.475+0000 7ffbc9600640 1 -- 172.21.15.114:0/3241012510 shutdown_connections 2024-05-27T09:27:49.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.475+0000 7ffbc9600640 1 -- 172.21.15.114:0/3241012510 wait complete. 2024-05-27T09:27:49.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.477+0000 7ffbc9600640 1 Processor -- start 2024-05-27T09:27:49.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.477+0000 7ffbc9600640 1 -- start start 2024-05-27T09:27:49.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.478+0000 7ffbc9600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffbc4070060 0x7ffbc41f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:49.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.478+0000 7ffbc9600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbc4074a80 0x7ffbc4070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:49.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.478+0000 7ffbc9600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbc4156880 0x7ffbc414a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:49.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.478+0000 7ffbc9600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc415c8e0 con 0x7ffbc4070060 2024-05-27T09:27:49.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.478+0000 7ffbc9600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc40c8750 con 0x7ffbc4074a80 2024-05-27T09:27:49.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.478+0000 7ffbc9600640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7ffbc415fa50 con 0x7ffbc4156880 2024-05-27T09:27:49.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.478+0000 7ffbc3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffbc4070060 0x7ffbc41f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:49.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.478+0000 7ffbc3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffbc4070060 0x7ffbc41f2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38048/0 (socket says 172.21.15.114:38048) 2024-05-27T09:27:49.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.478+0000 7ffbc3e00640 1 -- 172.21.15.114:0/1647116093 learned_addr learned my addr 172.21.15.114:0/1647116093 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:27:49.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.478+0000 7ffbbb400640 1 --2- 172.21.15.114:0/1647116093 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbc4074a80 0x7ffbc4070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:49.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.479+0000 7ffbc8c00640 1 --2- 172.21.15.114:0/1647116093 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbc4156880 0x7ffbc414a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:49.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.479+0000 7ffbc3e00640 1 -- 172.21.15.114:0/1647116093 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbc4156880 msgr2=0x7ffbc414a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:49.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.479+0000 7ffbc3e00640 1 --2- 172.21.15.114:0/1647116093 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbc4156880 0x7ffbc414a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:49.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.479+0000 7ffbc3e00640 1 -- 172.21.15.114:0/1647116093 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbc4074a80 msgr2=0x7ffbc4070b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:49.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.479+0000 7ffbc3e00640 1 --2- 172.21.15.114:0/1647116093 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbc4074a80 0x7ffbc4070b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:49.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.479+0000 7ffbc3e00640 1 -- 172.21.15.114:0/1647116093 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc41f3170 con 0x7ffbc4070060 2024-05-27T09:27:49.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.479+0000 7ffbc8c00640 1 --2- 172.21.15.114:0/1647116093 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbc4156880 0x7ffbc414a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:27:49.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.479+0000 7ffbbb400640 1 --2- 172.21.15.114:0/1647116093 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbc4074a80 0x7ffbc4070b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:27:49.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.479+0000 7ffbc3e00640 1 --2- 172.21.15.114:0/1647116093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffbc4070060 0x7ffbc41f2c30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4035040 tx=0x7ffbb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:49.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.479+0000 7ffbc0c00640 1 -- 172.21.15.114:0/1647116093 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4037050 con 0x7ffbc4070060 2024-05-27T09:27:49.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.479+0000 7ffbc0c00640 1 -- 172.21.15.114:0/1647116093 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb4043070 con 0x7ffbc4070060 2024-05-27T09:27:49.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.480+0000 7ffbc9600640 1 -- 172.21.15.114:0/1647116093 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc414d280 con 0x7ffbc4070060 2024-05-27T09:27:49.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.480+0000 7ffbc0c00640 1 -- 172.21.15.114:0/1647116093 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4032a30 con 0x7ffbc4070060 2024-05-27T09:27:49.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.480+0000 7ffbc9600640 1 -- 172.21.15.114:0/1647116093 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc414eb90 con 0x7ffbc4070060 2024-05-27T09:27:49.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.481+0000 7ffbc9600640 1 -- 172.21.15.114:0/1647116093 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc4050860 con 0x7ffbc4070060 2024-05-27T09:27:49.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.482+0000 7ffbc0c00640 1 -- 172.21.15.114:0/1647116093 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ffbb4032bd0 con 0x7ffbc4070060 2024-05-27T09:27:49.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.483+0000 7ffbc0c00640 1 --2- 172.21.15.114:0/1647116093 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ffba00762d0 0x7ffba00787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:49.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.483+0000 7ffbc0c00640 1 -- 172.21.15.114:0/1647116093 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ffbb40b83a0 con 0x7ffbc4070060 2024-05-27T09:27:49.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.483+0000 7ffbbb400640 1 --2- 172.21.15.114:0/1647116093 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ffba00762d0 0x7ffba00787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:49.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.484+0000 7ffbbb400640 1 --2- 172.21.15.114:0/1647116093 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ffba00762d0 0x7ffba00787e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffbac007a60 tx=0x7ffbac0079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:49.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.488+0000 7ffbc0c00640 1 -- 172.21.15.114:0/1647116093 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ffbb4081c50 con 0x7ffbc4070060 2024-05-27T09:27:49.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.684+0000 7ffbc9600640 1 -- 172.21.15.114:0/1647116093 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffbc4074860 con 0x7ffbc4070060 2024-05-27T09:27:49.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.687+0000 7ffbc0c00640 1 -- 172.21.15.114:0/1647116093 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16645 (secure 0 0 0) 0x7ffbb4081550 con 0x7ffbc4070060 2024-05-27T09:27:49.687 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:27:49.688 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"fsid":"2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48","created":"2024-05-27T09:27:00.307278+0000","modified":"2024-05-27T09:27:19.530054+0000","last_up_change":"2024-05-27T09:27:11.463226+0000","last_in_change":"2024-05-27T09:27:02.972880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T09:27:10.796859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T09:27:16.063199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T09:27:17.064857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3561c750-1d63-42f6-a6ae-0ee63721e15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6817","nonce":1960550710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6819","nonce":1960550710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6824","nonce":1960550710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6821","nonce":1960550710}]},"public_addr":"172.21.15.114:6817/1960550710","cluster_addr":"172.21.15.114:6819/1960550710","heartbeat_back_addr":"172.21.15.114:6824/1960550710","heartbeat_front_addr":"172.21.15.114:6821/1960550710","state":["exists","up"]},{"osd":1,"uuid":"9a407d04-38c0-4676-928f-d04e8116bd19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6825","nonce":326580057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6827","nonce":326580057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6831","nonce":326580057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6829","nonce":326580057}]},"public_addr":"172.21.15.114:6825/326580057","cluster_addr":"172.21.15.114:6827/326580057","heartbeat_back_addr":"172.21.15.114:6831/326580057","heartbeat_front_addr":"172.21.15.114:6829/326580057","state":["exists","up"]},{"osd":2,"uuid":"255a1cc6-9357-43f7-9db5-d760f4806ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6809","nonce":865568619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6811","nonce":865568619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6815","nonce":865568619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6813","nonce":865568619}]},"public_addr":"172.21.15.114:6809/865568619","cluster_addr":"172.21.15.114:6811/865568619","heartbeat_back_addr":"172.21.15.114:6815/865568619","heartbeat_front_addr":"172.21.15.114:6813/865568619","state":["exists","up"]},{"osd":3,"uuid":"cb4d7692-9a17-41b9-927e-d8eac3e217c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6801","nonce":1601792164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6803","nonce":1601792164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6807","nonce":1601792164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6805","nonce":1601792164}]},"public_addr":"172.21.15.114:6801/1601792164","cluster_addr":"172.21.15.114:6803/1601792164","heartbeat_back_addr":"172.21.15.114:6807/1601792164","heartbeat_front_addr":"172.21.15.114:6805/1601792164","state":["exists","up"]},{"osd":4,"uuid":"cdef1940-3453-4fbd-b0f1-459f5f5c7142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6817","nonce":1982246015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6819","nonce":1982246015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6823","nonce":1982246015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6821","nonce":1982246015}]},"public_addr":"172.21.15.163:6817/1982246015","cluster_addr":"172.21.15.163:6819/1982246015","heartbeat_back_addr":"172.21.15.163:6823/1982246015","heartbeat_front_addr":"172.21.15.163:6821/1982246015","state":["exists","up"]},{"osd":5,"uuid":"220a7b32-abcc-492f-bca8-4615cc2ea111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6801","nonce":1695294004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6803","nonce":1695294004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6807","nonce":1695294004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6805","nonce":1695294004}]},"public_addr":"172.21.15.163:6801/1695294004","cluster_addr":"172.21.15.163:6803/1695294004","heartbeat_back_addr":"172.21.15.163:6807/1695294004","heartbeat_front_addr":"172.21.15.163:6805/1695294004","state":["exists","up"]},{"osd":6,"uuid":"43a74d13-d03f-493a-a6c0-e73b0a056c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6809","nonce":3649718505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6811","nonce":3649718505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6815","nonce":3649718505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6813","nonce":3649718505}]},"public_addr":"172.21.15.163:6809/3649718505","cluster_addr":"172.21.15.163:6811/3649718505","heartbeat_back_addr":"172.21.15.163:6815/3649718505","heartbeat_front_addr":"172.21.15.163:6813/3649718505","state":["exists","up"]},{"osd":7,"uuid":"134629c2-70af-4b84-83a6-7ddf53681baa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6825","nonce":2539451311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6827","nonce":2539451311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6831","nonce":2539451311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6829","nonce":2539451311}]},"public_addr":"172.21.15.163:6825/2539451311","cluster_addr":"172.21.15.163:6827/2539451311","heartbeat_back_addr":"172.21.15.163:6831/2539451311","heartbeat_front_addr":"172.21.15.163:6829/2539451311","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.388942+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.515675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.646761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.400212+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.220072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.079531+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.180419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.175008+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T09:27:49.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.689+0000 7ffbc9600640 1 -- 172.21.15.114:0/1647116093 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ffba00762d0 msgr2=0x7ffba00787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:49.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.689+0000 7ffbc9600640 1 --2- 172.21.15.114:0/1647116093 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ffba00762d0 0x7ffba00787e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffbac007a60 tx=0x7ffbac0079b0 comp rx=0 tx=0).stop 2024-05-27T09:27:49.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.689+0000 7ffbc9600640 1 -- 172.21.15.114:0/1647116093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffbc4070060 msgr2=0x7ffbc41f2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:49.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.689+0000 7ffbc9600640 1 --2- 172.21.15.114:0/1647116093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffbc4070060 0x7ffbc41f2c30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4035040 tx=0x7ffbb40046c0 comp rx=0 tx=0).stop 2024-05-27T09:27:49.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.689+0000 7ffbc9600640 1 -- 172.21.15.114:0/1647116093 shutdown_connections 2024-05-27T09:27:49.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.689+0000 7ffbc9600640 1 --2- 172.21.15.114:0/1647116093 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ffba00762d0 0x7ffba00787e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:49.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.689+0000 7ffbc9600640 1 --2- 172.21.15.114:0/1647116093 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbc4156880 0x7ffbc414a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:49.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.689+0000 7ffbc9600640 1 --2- 172.21.15.114:0/1647116093 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbc4074a80 0x7ffbc4070b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:49.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.689+0000 7ffbc9600640 1 --2- 172.21.15.114:0/1647116093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffbc4070060 0x7ffbc41f2c30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:49.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.689+0000 7ffbc9600640 1 -- 172.21.15.114:0/1647116093 >> 172.21.15.114:0/1647116093 conn(0x7ffbc40ca320 msgr2=0x7ffbc4152780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:49.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.690+0000 7ffbc9600640 1 -- 172.21.15.114:0/1647116093 shutdown_connections 2024-05-27T09:27:49.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:27:49.690+0000 7ffbc9600640 1 -- 172.21.15.114:0/1647116093 wait complete. 2024-05-27T09:27:49.717 DEBUG:tasks.mds_thrash:Yielding 2024-05-27T09:27:49.717 INFO:teuthology.run_tasks:Running task workunit... 2024-05-27T09:27:49.730 INFO:tasks.workunit:Pulling workunits from ref a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:27:49.730 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-27T09:27:49.730 INFO:tasks.workunit:timeout=6h 2024-05-27T09:27:49.730 INFO:tasks.workunit:cleanup=True 2024-05-27T09:27:49.731 DEBUG:teuthology.orchestra.run.smithi114:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-27T09:27:49.731 INFO:tasks.mds_thrash.fs.[cephfs]:starting mds_do_thrash for fs cephfs 2024-05-27T09:27:49.731 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 100 secs before thrashing 2024-05-27T09:27:49.783 INFO:teuthology.orchestra.run.smithi114.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-27T09:27:49.783 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-27T09:27:49.783 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 28h/40d Inode: 1 Links: 2 2024-05-27T09:27:49.783 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T09:27:49.783 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-27T09:27:49.783 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-05-27T09:27:49.784 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-05-27 09:27:23.535092897 +0000 2024-05-27T09:27:49.784 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-05-27 09:27:43.658410003 +0000 2024-05-27T09:27:49.784 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-05-27 09:27:23.535092897 +0000 2024-05-27T09:27:49.784 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-27T09:27:49.784 DEBUG:teuthology.orchestra.run.smithi114:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-27T09:27:49.879 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:27:49.935 INFO:tasks.workunit.client.0.smithi114.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-27T09:28:17.650 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:28:17.659 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:28:47.774 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:28:47.779 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:29:17.862 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:29:17.871 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:29:19.413 INFO:tasks.workunit.client.0.smithi114.stderr:Updating files: 62% (7699/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-27T09:29:20.264 INFO:tasks.workunit.client.0.smithi114.stderr:Note: switching to 'a6507388dd5057528934822c0163b0c347ef1d5d'. 2024-05-27T09:29:20.264 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-05-27T09:29:20.264 INFO:tasks.workunit.client.0.smithi114.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-27T09:29:20.265 INFO:tasks.workunit.client.0.smithi114.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-27T09:29:20.265 INFO:tasks.workunit.client.0.smithi114.stderr:state without impacting any branches by switching back to a branch. 2024-05-27T09:29:20.265 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-05-27T09:29:20.265 INFO:tasks.workunit.client.0.smithi114.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-27T09:29:20.265 INFO:tasks.workunit.client.0.smithi114.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-27T09:29:20.265 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-05-27T09:29:20.265 INFO:tasks.workunit.client.0.smithi114.stderr: git switch -c 2024-05-27T09:29:20.265 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-05-27T09:29:20.265 INFO:tasks.workunit.client.0.smithi114.stderr:Or undo this operation with: 2024-05-27T09:29:20.265 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-05-27T09:29:20.265 INFO:tasks.workunit.client.0.smithi114.stderr: git switch - 2024-05-27T09:29:20.265 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-05-27T09:29:20.265 INFO:tasks.workunit.client.0.smithi114.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-27T09:29:20.266 INFO:tasks.workunit.client.0.smithi114.stderr: 2024-05-27T09:29:20.266 INFO:tasks.workunit.client.0.smithi114.stderr:HEAD is now at a6507388dd5 Merge PR #53755 into wip-vshankar-testing-20240521.105740-debug 2024-05-27T09:29:20.277 DEBUG:teuthology.orchestra.run.smithi114:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-27T09:29:20.338 INFO:tasks.workunit.client.0.smithi114.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-27T09:29:20.345 INFO:tasks.workunit.client.0.smithi114.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-27T09:29:20.345 INFO:tasks.workunit.client.0.smithi114.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-27T09:29:21.080 INFO:tasks.workunit.client.0.smithi114.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-27T09:29:21.202 INFO:tasks.workunit.client.0.smithi114.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-27T09:29:21.281 INFO:tasks.workunit.client.0.smithi114.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-27T09:29:21.286 INFO:tasks.workunit.client.0.smithi114.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-27T09:29:21.287 INFO:tasks.workunit.client.0.smithi114.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-27T09:29:21.368 INFO:tasks.workunit.client.0.smithi114.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-27T09:29:21.378 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:29:21.378 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-27T09:29:21.430 INFO:tasks.workunit:Running workunits matching fs/snaps on client.0... 2024-05-27T09:29:21.431 INFO:tasks.workunit:Running workunit fs/snaps/snap-rm-diff.sh... 2024-05-27T09:29:21.431 DEBUG:teuthology.orchestra.run.smithi114:workunit test fs/snaps/snap-rm-diff.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6507388dd5057528934822c0163b0c347ef1d5d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh 2024-05-27T09:29:21.501 INFO:tasks.workunit.client.0.smithi114.stderr:+ wget -q http://download.ceph.com/qa/linux-2.6.33.tar.bz2 2024-05-27T09:29:26.583 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir foo 2024-05-27T09:29:26.594 INFO:tasks.workunit.client.0.smithi114.stderr:+ cp linux-2.6.33.tar.bz2 foo 2024-05-27T09:29:26.705 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir foo/.snap/barsnap 2024-05-27T09:29:26.744 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm foo/linux-2.6.33.tar.bz2 2024-05-27T09:29:26.775 INFO:tasks.workunit.client.0.smithi114.stderr:+ diff -q foo/.snap/barsnap/linux-2.6.33.tar.bz2 linux-2.6.33.tar.bz2 2024-05-27T09:29:27.411 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo 'passed: files are identical' 2024-05-27T09:29:27.411 INFO:tasks.workunit.client.0.smithi114.stdout:passed: files are identical 2024-05-27T09:29:27.412 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir foo/.snap/barsnap 2024-05-27T09:29:27.460 INFO:tasks.workunit.client.0.smithi114.stdout:OK 2024-05-27T09:29:27.460 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo OK 2024-05-27T09:29:27.462 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-27T09:29:27.462 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-27T09:29:27.576 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-1.sh... 2024-05-27T09:29:27.576 DEBUG:teuthology.orchestra.run.smithi114:workunit test fs/snaps/snaptest-1.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6507388dd5057528934822c0163b0c347ef1d5d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-1.sh 2024-05-27T09:29:27.653 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo 1 2024-05-27T09:29:27.654 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo 2 2024-05-27T09:29:27.655 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo 3 2024-05-27T09:29:27.656 INFO:tasks.workunit.client.0.smithi114.stderr:+ '[' -e file4 ']' 2024-05-27T09:29:27.656 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir .snap/snap1 2024-05-27T09:29:27.696 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo 4 2024-05-27T09:29:27.700 INFO:tasks.workunit.client.0.smithi114.stderr:++ ls 2024-05-27T09:29:27.707 INFO:tasks.workunit.client.0.smithi114.stderr:+ now='file1 2024-05-27T09:29:27.708 INFO:tasks.workunit.client.0.smithi114.stderr:file2 2024-05-27T09:29:27.708 INFO:tasks.workunit.client.0.smithi114.stderr:file3 2024-05-27T09:29:27.708 INFO:tasks.workunit.client.0.smithi114.stderr:file4' 2024-05-27T09:29:27.708 INFO:tasks.workunit.client.0.smithi114.stderr:++ ls .snap/snap1 2024-05-27T09:29:27.717 INFO:tasks.workunit.client.0.smithi114.stderr:+ then='file1 2024-05-27T09:29:27.718 INFO:tasks.workunit.client.0.smithi114.stderr:file2 2024-05-27T09:29:27.718 INFO:tasks.workunit.client.0.smithi114.stderr:file3' 2024-05-27T09:29:27.718 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir .snap/snap1 2024-05-27T09:29:27.728 INFO:tasks.workunit.client.0.smithi114.stderr:+ '[' 'file1 2024-05-27T09:29:27.728 INFO:tasks.workunit.client.0.smithi114.stderr:file2 2024-05-27T09:29:27.728 INFO:tasks.workunit.client.0.smithi114.stderr:file3 2024-05-27T09:29:27.728 INFO:tasks.workunit.client.0.smithi114.stderr:file4' = 'file1 2024-05-27T09:29:27.728 INFO:tasks.workunit.client.0.smithi114.stderr:file2 2024-05-27T09:29:27.729 INFO:tasks.workunit.client.0.smithi114.stderr:file3' ']' 2024-05-27T09:29:27.729 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo 1 2024-05-27T09:29:27.739 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo 2 2024-05-27T09:29:27.755 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo 3 2024-05-27T09:29:27.772 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir .snap/snap1 2024-05-27T09:29:27.816 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo 4 2024-05-27T09:29:27.822 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir .snap/snap1 2024-05-27T09:29:27.831 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm file1 file2 file3 file4 2024-05-27T09:29:27.846 INFO:tasks.workunit.client.0.smithi114.stdout:OK 2024-05-27T09:29:27.847 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo OK 2024-05-27T09:29:27.847 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-27T09:29:27.848 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-27T09:29:27.916 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-2.sh... 2024-05-27T09:29:27.916 DEBUG:teuthology.orchestra.run.smithi114:workunit test fs/snaps/snaptest-2.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6507388dd5057528934822c0163b0c347ef1d5d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-2.sh 2024-05-27T09:29:28.639 INFO:tasks.workunit.client.0.smithi114.stdout:Create dir 100 to 199 ... 2024-05-27T09:29:28.644 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 100 2024-05-27T09:29:28.670 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 101 2024-05-27T09:29:28.707 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 102 2024-05-27T09:29:28.735 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 103 2024-05-27T09:29:28.773 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 104 2024-05-27T09:29:28.806 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 105 2024-05-27T09:29:28.845 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 106 2024-05-27T09:29:28.880 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 107 2024-05-27T09:29:28.905 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 108 2024-05-27T09:29:28.937 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 109 2024-05-27T09:29:28.973 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 110 2024-05-27T09:29:29.012 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 111 2024-05-27T09:29:29.043 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 112 2024-05-27T09:29:29.081 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 113 2024-05-27T09:29:29.114 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 114 2024-05-27T09:29:29.151 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 115 2024-05-27T09:29:29.197 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 116 2024-05-27T09:29:29.225 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 117 2024-05-27T09:29:29.242 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 118 2024-05-27T09:29:29.259 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 119 2024-05-27T09:29:29.276 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 120 2024-05-27T09:29:29.295 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 121 2024-05-27T09:29:29.320 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 122 2024-05-27T09:29:29.343 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 123 2024-05-27T09:29:29.364 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 124 2024-05-27T09:29:29.385 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 125 2024-05-27T09:29:29.407 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 126 2024-05-27T09:29:29.428 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 127 2024-05-27T09:29:29.449 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 128 2024-05-27T09:29:29.470 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 129 2024-05-27T09:29:29.492 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 130 2024-05-27T09:29:29.516 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 131 2024-05-27T09:29:29.542 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 132 2024-05-27T09:29:29.573 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 133 2024-05-27T09:29:29.600 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 134 2024-05-27T09:29:29.624 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 135 2024-05-27T09:29:29.646 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 136 2024-05-27T09:29:29.669 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 137 2024-05-27T09:29:29.693 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 138 2024-05-27T09:29:29.722 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 139 2024-05-27T09:29:29.732 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:29:29.750 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 140 2024-05-27T09:29:29.771 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 141 2024-05-27T09:29:29.790 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 142 2024-05-27T09:29:29.811 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 143 2024-05-27T09:29:29.831 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 144 2024-05-27T09:29:29.852 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 145 2024-05-27T09:29:29.875 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 146 2024-05-27T09:29:29.900 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 147 2024-05-27T09:29:29.930 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 148 2024-05-27T09:29:29.953 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 149 2024-05-27T09:29:29.969 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 150 2024-05-27T09:29:29.987 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 151 2024-05-27T09:29:30.004 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 152 2024-05-27T09:29:30.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.020+0000 7feaa7200640 1 Processor -- start 2024-05-27T09:29:30.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.020+0000 7feaa7200640 1 -- start start 2024-05-27T09:29:30.021 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 153 2024-05-27T09:29:30.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.021+0000 7feaa7200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feaa80ca9d0 0x7feaa80ca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:30.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.021+0000 7feaa7200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7feaa815b220 0x7feaa81542a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:30.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.022+0000 7feaa7200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaa80c5270 con 0x7feaa80cc530 2024-05-27T09:29:30.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.022+0000 7feaa7200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7feaa808d9d0 con 0x7feaa815b220 2024-05-27T09:29:30.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.022+0000 7feaa7200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7feaa80c5510 con 0x7feaa80ca9d0 2024-05-27T09:29:30.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.022+0000 7feaa6800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7feaa815b220 0x7feaa81542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:30.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.022+0000 7feaa5400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7feaa80cc530 0x7feaa815ab10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:43886/0 (socket says 172.21.15.114:43886) 2024-05-27T09:29:30.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.022+0000 7feaa5400640 1 -- 172.21.15.114:0/3214741608 learned_addr learned my addr 172.21.15.114:0/3214741608 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:29:30.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.022+0000 7feaa5e00640 1 --2- 172.21.15.114:0/3214741608 >> v2:172.21.15.163:3300/0 conn(0x7feaa80ca9d0 0x7feaa80ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:30.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.022+0000 7feaa6800640 1 -- 172.21.15.114:0/3214741608 >> v2:172.21.15.163:3300/0 conn(0x7feaa80ca9d0 msgr2=0x7feaa80ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:29:30.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.022+0000 7feaa6800640 1 --2- 172.21.15.114:0/3214741608 >> v2:172.21.15.163:3300/0 conn(0x7feaa80ca9d0 0x7feaa80ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:30.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.022+0000 7feaa6800640 1 -- 172.21.15.114:0/3214741608 >> v1:172.21.15.114:6789/0 conn(0x7feaa80cc530 legacy=0x7feaa815ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:30.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.022+0000 7feaa6800640 1 -- 172.21.15.114:0/3214741608 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa80c57f0 con 0x7feaa815b220 2024-05-27T09:29:30.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.022+0000 7feaa4a00640 1 -- 172.21.15.114:0/3214741608 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1581031252 0 0) 0x7feaa80c5270 con 0x7feaa80cc530 2024-05-27T09:29:30.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.023+0000 7feaa6800640 1 --2- 172.21.15.114:0/3214741608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaa815b220 0x7feaa81542a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7feaa806d200 tx=0x7feaa002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cabb6bd0826dee0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:30.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.024+0000 7feaa4a00640 1 -- 172.21.15.114:0/3214741608 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa0007740 con 0x7feaa815b220 2024-05-27T09:29:30.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.024+0000 7feaa4a00640 1 -- 172.21.15.114:0/3214741608 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaa00078e0 con 0x7feaa815b220 2024-05-27T09:29:30.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.024+0000 7feaa4a00640 1 -- 172.21.15.114:0/3214741608 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa0007cb0 con 0x7feaa815b220 2024-05-27T09:29:30.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.025+0000 7feaa7200640 1 -- 172.21.15.114:0/3214741608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaa815b220 msgr2=0x7feaa81542a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:30.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.025+0000 7feaa7200640 1 --2- 172.21.15.114:0/3214741608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaa815b220 0x7feaa81542a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7feaa806d200 tx=0x7feaa002a5f0 comp rx=0 tx=0).stop 2024-05-27T09:29:30.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.025+0000 7feaa7200640 1 -- 172.21.15.114:0/3214741608 shutdown_connections 2024-05-27T09:29:30.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.025+0000 7feaa7200640 1 --2- 172.21.15.114:0/3214741608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaa815b220 0x7feaa81542a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:30.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.025+0000 7feaa7200640 1 --2- 172.21.15.114:0/3214741608 >> v2:172.21.15.163:3300/0 conn(0x7feaa80ca9d0 0x7feaa80ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:30.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.025+0000 7feaa7200640 1 -- 172.21.15.114:0/3214741608 >> 172.21.15.114:0/3214741608 conn(0x7feaa8071460 msgr2=0x7feaa8068600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:30.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.025+0000 7feaa7200640 1 -- 172.21.15.114:0/3214741608 shutdown_connections 2024-05-27T09:29:30.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.025+0000 7feaa7200640 1 -- 172.21.15.114:0/3214741608 wait complete. 2024-05-27T09:29:30.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.027+0000 7feaa7200640 1 Processor -- start 2024-05-27T09:29:30.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.027+0000 7feaa7200640 1 -- start start 2024-05-27T09:29:30.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.029+0000 7feaa7200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feaa80ca9d0 0x7feaa814e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:30.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.029+0000 7feaa7200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaa80cc530 0x7feaa81515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:30.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.029+0000 7feaa7200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaa814f030 0x7feaa814f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:30.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.029+0000 7feaa7200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7feaa80cb160 con 0x7feaa814f030 2024-05-27T09:29:30.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.029+0000 7feaa7200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7feaa80c5db0 con 0x7feaa80cc530 2024-05-27T09:29:30.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.029+0000 7feaa7200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7feaa80cb2e0 con 0x7feaa80ca9d0 2024-05-27T09:29:30.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.029+0000 7feaa6800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaa814f030 0x7feaa814f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:30.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.029+0000 7feaa5400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaa80cc530 0x7feaa81515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:30.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.029+0000 7feaa5e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feaa80ca9d0 0x7feaa814e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:30.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.029+0000 7feaa5400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaa80cc530 0x7feaa81515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:44060/0 (socket says 172.21.15.114:44060) 2024-05-27T09:29:30.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.029+0000 7feaa5400640 1 -- 172.21.15.114:0/4194008244 learned_addr learned my addr 172.21.15.114:0/4194008244 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:29:30.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.030+0000 7feaa5e00640 1 -- 172.21.15.114:0/4194008244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaa80cc530 msgr2=0x7feaa81515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.030+0000 7feaa5e00640 1 --2- 172.21.15.114:0/4194008244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaa80cc530 0x7feaa81515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.030+0000 7feaa5e00640 1 -- 172.21.15.114:0/4194008244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaa814f030 msgr2=0x7feaa814f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.030+0000 7feaa5e00640 1 --2- 172.21.15.114:0/4194008244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaa814f030 0x7feaa814f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.030+0000 7feaa5e00640 1 -- 172.21.15.114:0/4194008244 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa81f17b0 con 0x7feaa80ca9d0 2024-05-27T09:29:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.030+0000 7feaa6800640 1 --2- 172.21.15.114:0/4194008244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaa814f030 0x7feaa814f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:29:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.030+0000 7feaa5e00640 1 --2- 172.21.15.114:0/4194008244 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feaa80ca9d0 0x7feaa814e2e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fea9800e8a0 tx=0x7fea9800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.030+0000 7fea9e000640 1 -- 172.21.15.114:0/4194008244 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea98017070 con 0x7feaa80ca9d0 2024-05-27T09:29:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.030+0000 7fea9e000640 1 -- 172.21.15.114:0/4194008244 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea9800c940 con 0x7feaa80ca9d0 2024-05-27T09:29:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.031+0000 7feaa7200640 1 -- 172.21.15.114:0/4194008244 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa814e820 con 0x7feaa80ca9d0 2024-05-27T09:29:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.031+0000 7fea9e000640 1 -- 172.21.15.114:0/4194008244 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea98009e60 con 0x7feaa80ca9d0 2024-05-27T09:29:30.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.031+0000 7feaa7200640 1 -- 172.21.15.114:0/4194008244 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa80c9570 con 0x7feaa80ca9d0 2024-05-27T09:29:30.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.032+0000 7feaa7200640 1 -- 172.21.15.114:0/4194008244 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaa81f1980 con 0x7feaa80ca9d0 2024-05-27T09:29:30.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.033+0000 7fea9e000640 1 -- 172.21.15.114:0/4194008244 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fea9801a070 con 0x7feaa80ca9d0 2024-05-27T09:29:30.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.034+0000 7fea9e000640 1 --2- 172.21.15.114:0/4194008244 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fea88076300 0x7fea88078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:30.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.034+0000 7fea9e000640 1 -- 172.21.15.114:0/4194008244 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 5794+0+0 (secure 0 0 0) 0x7fea98097360 con 0x7feaa80ca9d0 2024-05-27T09:29:30.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.035+0000 7feaa5400640 1 --2- 172.21.15.114:0/4194008244 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fea88076300 0x7fea88078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:30.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.035+0000 7feaa5400640 1 --2- 172.21.15.114:0/4194008244 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fea88076300 0x7fea88078810 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7feaa80c6850 tx=0x7fea94007e70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:30.038 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 154 2024-05-27T09:29:30.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.039+0000 7fea9e000640 1 -- 172.21.15.114:0/4194008244 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fea98060600 con 0x7feaa80ca9d0 2024-05-27T09:29:30.054 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 155 2024-05-27T09:29:30.072 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 156 2024-05-27T09:29:30.089 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 157 2024-05-27T09:29:30.109 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 158 2024-05-27T09:29:30.132 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 159 2024-05-27T09:29:30.164 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 160 2024-05-27T09:29:30.199 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 161 2024-05-27T09:29:30.217 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 162 2024-05-27T09:29:30.239 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 163 2024-05-27T09:29:30.268 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 164 2024-05-27T09:29:30.300 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 165 2024-05-27T09:29:30.333 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 166 2024-05-27T09:29:30.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.338+0000 7feaa7200640 1 -- 172.21.15.114:0/4194008244 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feaa8050860 con 0x7feaa80ca9d0 2024-05-27T09:29:30.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.339+0000 7fea9e000640 1 -- 172.21.15.114:0/4194008244 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+6877 (secure 0 0 0) 0x7fea98064900 con 0x7feaa80ca9d0 2024-05-27T09:29:30.339 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:29:30.339 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:27:33.841352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"f","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6839/444766663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":444766663},{"type":"v1","addr":"172.21.15.163:6839","nonce":444766663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4279,4297,4300]},"id":1}]} 2024-05-27T09:29:30.340 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 12 2024-05-27T09:29:30.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.342+0000 7feaa7200640 1 -- 172.21.15.114:0/4194008244 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fea88076300 msgr2=0x7fea88078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:30.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.342+0000 7feaa7200640 1 --2- 172.21.15.114:0/4194008244 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fea88076300 0x7fea88078810 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7feaa80c6850 tx=0x7fea94007e70 comp rx=0 tx=0).stop 2024-05-27T09:29:30.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.342+0000 7feaa7200640 1 -- 172.21.15.114:0/4194008244 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feaa80ca9d0 msgr2=0x7feaa814e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:30.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.342+0000 7feaa7200640 1 --2- 172.21.15.114:0/4194008244 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feaa80ca9d0 0x7feaa814e2e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fea9800e8a0 tx=0x7fea9800ed60 comp rx=0 tx=0).stop 2024-05-27T09:29:30.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.342+0000 7feaa7200640 1 -- 172.21.15.114:0/4194008244 shutdown_connections 2024-05-27T09:29:30.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.342+0000 7feaa7200640 1 --2- 172.21.15.114:0/4194008244 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fea88076300 0x7fea88078810 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:30.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.342+0000 7feaa7200640 1 --2- 172.21.15.114:0/4194008244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feaa814f030 0x7feaa814f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:30.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.342+0000 7feaa7200640 1 --2- 172.21.15.114:0/4194008244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaa80cc530 0x7feaa81515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:30.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.342+0000 7feaa7200640 1 --2- 172.21.15.114:0/4194008244 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feaa80ca9d0 0x7feaa814e2e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:30.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.342+0000 7feaa7200640 1 -- 172.21.15.114:0/4194008244 >> 172.21.15.114:0/4194008244 conn(0x7feaa8071460 msgr2=0x7feaa80c9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:30.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.342+0000 7feaa7200640 1 -- 172.21.15.114:0/4194008244 shutdown_connections 2024-05-27T09:29:30.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.342+0000 7feaa7200640 1 -- 172.21.15.114:0/4194008244 wait complete. 2024-05-27T09:29:30.352 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 167 2024-05-27T09:29:30.373 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=2) 2024-05-27T09:29:30.374 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-05-27T09:29:30.374 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:29:30.374 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 168 2024-05-27T09:29:30.386 INFO:tasks.ceph.mds.f:Stopped 2024-05-27T09:29:30.386 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:29:30.390 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 169 2024-05-27T09:29:30.408 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 170 2024-05-27T09:29:30.426 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 171 2024-05-27T09:29:30.444 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 172 2024-05-27T09:29:30.462 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 173 2024-05-27T09:29:30.479 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 174 2024-05-27T09:29:30.495 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 175 2024-05-27T09:29:30.512 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 176 2024-05-27T09:29:30.529 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 177 2024-05-27T09:29:30.545 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 178 2024-05-27T09:29:30.562 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 179 2024-05-27T09:29:30.581 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 180 2024-05-27T09:29:30.598 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 181 2024-05-27T09:29:30.619 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 182 2024-05-27T09:29:30.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.657+0000 7f5b48c00640 1 Processor -- start 2024-05-27T09:29:30.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.657+0000 7f5b48c00640 1 -- start start 2024-05-27T09:29:30.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.659+0000 7f5b48c00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5b44156980 0x7f5b44160330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:30.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.659+0000 7f5b48c00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5b440cb740 0x7f5b440cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:30.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.659+0000 7f5b48c00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b4406e1c0 con 0x7f5b44160870 2024-05-27T09:29:30.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.659+0000 7f5b48c00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f5b4406ed40 con 0x7f5b440cb740 2024-05-27T09:29:30.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.659+0000 7f5b48c00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f5b4406e4a0 con 0x7f5b44156980 2024-05-27T09:29:30.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.659+0000 7f5b3be00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f5b44160870 0x7f5b440cc350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:43900/0 (socket says 172.21.15.114:43900) 2024-05-27T09:29:30.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.659+0000 7f5b3be00640 1 -- 172.21.15.114:0/347593653 learned_addr learned my addr 172.21.15.114:0/347593653 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:29:30.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.659+0000 7f5b43400640 1 --2- 172.21.15.114:0/347593653 >> v2:172.21.15.163:3300/0 conn(0x7f5b44156980 0x7f5b44160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:30.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.659+0000 7f5b43e00640 1 --2- 172.21.15.114:0/347593653 >> v2:172.21.15.114:3300/0 conn(0x7f5b440cb740 0x7f5b440cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:30.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.660+0000 7f5b43400640 1 -- 172.21.15.114:0/347593653 >> v2:172.21.15.114:3300/0 conn(0x7f5b440cb740 msgr2=0x7f5b440cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:29:30.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.660+0000 7f5b43400640 1 --2- 172.21.15.114:0/347593653 >> v2:172.21.15.114:3300/0 conn(0x7f5b440cb740 0x7f5b440cbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:30.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.660+0000 7f5b43400640 1 -- 172.21.15.114:0/347593653 >> v1:172.21.15.114:6789/0 conn(0x7f5b44160870 legacy=0x7f5b440cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:30.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.660+0000 7f5b43400640 1 -- 172.21.15.114:0/347593653 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4406e780 con 0x7f5b44156980 2024-05-27T09:29:30.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.661+0000 7f5b43400640 1 --2- 172.21.15.114:0/347593653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b44156980 0x7f5b44160330 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5b440d08f0 tx=0x7f5b3402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8f56d290dd053c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:30.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.662+0000 7f5b42a00640 1 -- 172.21.15.114:0/347593653 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34037070 con 0x7f5b44156980 2024-05-27T09:29:30.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.662+0000 7f5b42a00640 1 -- 172.21.15.114:0/347593653 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b34032440 con 0x7f5b44156980 2024-05-27T09:29:30.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.662+0000 7f5b42a00640 1 -- 172.21.15.114:0/347593653 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3402ae60 con 0x7f5b44156980 2024-05-27T09:29:30.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.663+0000 7f5b48c00640 1 -- 172.21.15.114:0/347593653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b44156980 msgr2=0x7f5b44160330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:30.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.663+0000 7f5b48c00640 1 --2- 172.21.15.114:0/347593653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b44156980 0x7f5b44160330 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5b440d08f0 tx=0x7f5b3402a110 comp rx=0 tx=0).stop 2024-05-27T09:29:30.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.663+0000 7f5b48c00640 1 -- 172.21.15.114:0/347593653 shutdown_connections 2024-05-27T09:29:30.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.663+0000 7f5b48c00640 1 --2- 172.21.15.114:0/347593653 >> v2:172.21.15.114:3300/0 conn(0x7f5b440cb740 0x7f5b440cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:30.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.663+0000 7f5b48c00640 1 --2- 172.21.15.114:0/347593653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b44156980 0x7f5b44160330 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:30.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.663+0000 7f5b48c00640 1 -- 172.21.15.114:0/347593653 >> 172.21.15.114:0/347593653 conn(0x7f5b44071460 msgr2=0x7f5b44068600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:30.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.663+0000 7f5b48c00640 1 -- 172.21.15.114:0/347593653 shutdown_connections 2024-05-27T09:29:30.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.663+0000 7f5b48c00640 1 -- 172.21.15.114:0/347593653 wait complete. 2024-05-27T09:29:30.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.665+0000 7f5b48c00640 1 Processor -- start 2024-05-27T09:29:30.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.665+0000 7f5b48c00640 1 -- start start 2024-05-27T09:29:30.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.667+0000 7f5b48c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b440cb740 0x7f5b440b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:30.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.667+0000 7f5b48c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b44156980 0x7f5b441f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:30.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.667+0000 7f5b48c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b44160870 0x7f5b440b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:30.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.667+0000 7f5b48c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5b440cc090 con 0x7f5b44160870 2024-05-27T09:29:30.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.667+0000 7f5b48c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4415fe10 con 0x7f5b440cb740 2024-05-27T09:29:30.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.667+0000 7f5b48c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f5b44156d90 con 0x7f5b44156980 2024-05-27T09:29:30.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.667+0000 7f5b43e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b44160870 0x7f5b440b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:30.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.667+0000 7f5b43e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b44160870 0x7f5b440b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:32870/0 (socket says 172.21.15.114:32870) 2024-05-27T09:29:30.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.667+0000 7f5b43e00640 1 -- 172.21.15.114:0/1410473570 learned_addr learned my addr 172.21.15.114:0/1410473570 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:29:30.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.667+0000 7f5b43400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b440cb740 0x7f5b440b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:30.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.667+0000 7f5b3be00640 1 --2- 172.21.15.114:0/1410473570 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b44156980 0x7f5b441f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:30.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.668+0000 7f5b43400640 1 -- 172.21.15.114:0/1410473570 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b44156980 msgr2=0x7f5b441f2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:30.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.668+0000 7f5b43400640 1 --2- 172.21.15.114:0/1410473570 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b44156980 0x7f5b441f2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:30.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.668+0000 7f5b43400640 1 -- 172.21.15.114:0/1410473570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b44160870 msgr2=0x7f5b440b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:30.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.668+0000 7f5b43400640 1 --2- 172.21.15.114:0/1410473570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b44160870 0x7f5b440b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:30.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.668+0000 7f5b43400640 1 -- 172.21.15.114:0/1410473570 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b441f3370 con 0x7f5b440cb740 2024-05-27T09:29:30.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.668+0000 7f5b43400640 1 --2- 172.21.15.114:0/1410473570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b440cb740 0x7f5b440b45d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5b34031000 tx=0x7f5b340046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:30.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.668+0000 7f5b3b400640 1 -- 172.21.15.114:0/1410473570 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34041020 con 0x7f5b440cb740 2024-05-27T09:29:30.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.668+0000 7f5b3b400640 1 -- 172.21.15.114:0/1410473570 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b34030720 con 0x7f5b440cb740 2024-05-27T09:29:30.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.668+0000 7f5b3b400640 1 -- 172.21.15.114:0/1410473570 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34037040 con 0x7f5b440cb740 2024-05-27T09:29:30.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.668+0000 7f5b48c00640 1 -- 172.21.15.114:0/1410473570 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b44159470 con 0x7f5b440cb740 2024-05-27T09:29:30.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.669+0000 7f5b48c00640 1 -- 172.21.15.114:0/1410473570 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b44159a30 con 0x7f5b440cb740 2024-05-27T09:29:30.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.671+0000 7f5b48c00640 1 -- 172.21.15.114:0/1410473570 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b4406d5d0 con 0x7f5b440cb740 2024-05-27T09:29:30.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.671+0000 7f5b3b400640 1 -- 172.21.15.114:0/1410473570 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5b34044020 con 0x7f5b440cb740 2024-05-27T09:29:30.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.672+0000 7f5b3b400640 1 --2- 172.21.15.114:0/1410473570 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5b20076300 0x7f5b20078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:30.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.672+0000 7f5b3b400640 1 -- 172.21.15.114:0/1410473570 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5794+0+0 (secure 0 0 0) 0x7f5b340b7d70 con 0x7f5b440cb740 2024-05-27T09:29:30.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.673+0000 7f5b3be00640 1 --2- 172.21.15.114:0/1410473570 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5b20076300 0x7f5b20078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:30.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.673+0000 7f5b3be00640 1 --2- 172.21.15.114:0/1410473570 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5b20076300 0x7f5b20078810 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c000f60 tx=0x7f5b2c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:30.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.677+0000 7f5b3b400640 1 -- 172.21.15.114:0/1410473570 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5b3404b030 con 0x7f5b440cb740 2024-05-27T09:29:30.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.966+0000 7f5b48c00640 1 -- 172.21.15.114:0/1410473570 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b44156450 con 0x7f5b440cb740 2024-05-27T09:29:30.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.968+0000 7f5b3b400640 1 -- 172.21.15.114:0/1410473570 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+6877 (secure 0 0 0) 0x7f5b44156450 con 0x7f5b440cb740 2024-05-27T09:29:30.968 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:29:30.968 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:27:33.841352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"f","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6839/444766663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":444766663},{"type":"v1","addr":"172.21.15.163:6839","nonce":444766663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4279,4297,4300]},"id":1}]} 2024-05-27T09:29:30.968 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 12 2024-05-27T09:29:30.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.970+0000 7f5b48c00640 1 -- 172.21.15.114:0/1410473570 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5b20076300 msgr2=0x7f5b20078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:30.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.970+0000 7f5b48c00640 1 --2- 172.21.15.114:0/1410473570 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5b20076300 0x7f5b20078810 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c000f60 tx=0x7f5b2c009040 comp rx=0 tx=0).stop 2024-05-27T09:29:30.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.970+0000 7f5b48c00640 1 -- 172.21.15.114:0/1410473570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b440cb740 msgr2=0x7f5b440b45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:30.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.970+0000 7f5b48c00640 1 --2- 172.21.15.114:0/1410473570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b440cb740 0x7f5b440b45d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5b34031000 tx=0x7f5b340046c0 comp rx=0 tx=0).stop 2024-05-27T09:29:30.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.970+0000 7f5b48c00640 1 -- 172.21.15.114:0/1410473570 shutdown_connections 2024-05-27T09:29:30.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.970+0000 7f5b48c00640 1 --2- 172.21.15.114:0/1410473570 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5b20076300 0x7f5b20078810 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:30.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.970+0000 7f5b48c00640 1 --2- 172.21.15.114:0/1410473570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5b44160870 0x7f5b440b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:30.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.970+0000 7f5b48c00640 1 --2- 172.21.15.114:0/1410473570 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b44156980 0x7f5b441f2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:30.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.970+0000 7f5b48c00640 1 --2- 172.21.15.114:0/1410473570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b440cb740 0x7f5b440b45d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:30.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.970+0000 7f5b48c00640 1 -- 172.21.15.114:0/1410473570 >> 172.21.15.114:0/1410473570 conn(0x7f5b44071460 msgr2=0x7f5b44150700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:30.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.970+0000 7f5b48c00640 1 -- 172.21.15.114:0/1410473570 shutdown_connections 2024-05-27T09:29:30.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:30.970+0000 7f5b48c00640 1 -- 172.21.15.114:0/1410473570 wait complete. 2024-05-27T09:29:31.000 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-05-27T09:29:31.021 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 183 2024-05-27T09:29:31.043 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 184 2024-05-27T09:29:31.066 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 185 2024-05-27T09:29:31.088 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 186 2024-05-27T09:29:31.113 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 187 2024-05-27T09:29:31.144 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 188 2024-05-27T09:29:31.172 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 189 2024-05-27T09:29:31.210 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 190 2024-05-27T09:29:31.249 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 191 2024-05-27T09:29:31.286 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 192 2024-05-27T09:29:31.325 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 193 2024-05-27T09:29:31.358 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 194 2024-05-27T09:29:31.391 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 195 2024-05-27T09:29:31.422 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 196 2024-05-27T09:29:31.478 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 197 2024-05-27T09:29:31.516 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 198 2024-05-27T09:29:31.542 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 199 2024-05-27T09:29:31.577 INFO:tasks.workunit.client.0.smithi114.stdout:Take first snapshot .snap/test1 2024-05-27T09:29:33.002 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:29:33.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.283+0000 7f16dca00640 1 Processor -- start 2024-05-27T09:29:33.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.283+0000 7f16dca00640 1 -- start start 2024-05-27T09:29:33.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.285+0000 7f16dca00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f16d80c2a00 0x7f16d8159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:33.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.285+0000 7f16dca00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f16d8159e40 0x7f16d8074140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:33.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.285+0000 7f16dca00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16d80c96d0 con 0x7f16d81534c0 2024-05-27T09:29:33.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.285+0000 7f16dca00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f16d8159190 con 0x7f16d80c2a00 2024-05-27T09:29:33.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.285+0000 7f16dca00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f16d81538d0 con 0x7f16d8159e40 2024-05-27T09:29:33.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.285+0000 7f16cde00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f16d80c2a00 0x7f16d8159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:33.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.286+0000 7f16cde00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f16d80c2a00 0x7f16d8159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:32876/0 (socket says 172.21.15.114:32876) 2024-05-27T09:29:33.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.286+0000 7f16cde00640 1 -- 172.21.15.114:0/787507281 learned_addr learned my addr 172.21.15.114:0/787507281 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:29:33.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.286+0000 7f16d7200640 1 --2- 172.21.15.114:0/787507281 >> v2:172.21.15.163:3300/0 conn(0x7f16d8159e40 0x7f16d8074140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:33.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.286+0000 7f16cde00640 1 -- 172.21.15.114:0/787507281 >> v2:172.21.15.163:3300/0 conn(0x7f16d8159e40 msgr2=0x7f16d8074140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:29:33.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.286+0000 7f16cde00640 1 --2- 172.21.15.114:0/787507281 >> v2:172.21.15.163:3300/0 conn(0x7f16d8159e40 0x7f16d8074140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:33.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.286+0000 7f16cde00640 1 -- 172.21.15.114:0/787507281 >> v1:172.21.15.163:6789/0 conn(0x7f16d81534c0 legacy=0x7f16d80c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:33.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.286+0000 7f16cde00640 1 -- 172.21.15.114:0/787507281 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d80c8930 con 0x7f16d80c2a00 2024-05-27T09:29:33.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.286+0000 7f16d5e00640 1 -- 172.21.15.114:0/787507281 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2165926682 0 0) 0x7f16d80c96d0 con 0x7f16d81534c0 2024-05-27T09:29:33.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.287+0000 7f16cde00640 1 --2- 172.21.15.114:0/787507281 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16d80c2a00 0x7f16d8159900 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f16d80cb6d0 tx=0x7f16c002c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba91a33774a1611b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:33.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.288+0000 7f16d5e00640 1 -- 172.21.15.114:0/787507281 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c000b2e0 con 0x7f16d80c2a00 2024-05-27T09:29:33.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.288+0000 7f16d5e00640 1 -- 172.21.15.114:0/787507281 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16c0037070 con 0x7f16d80c2a00 2024-05-27T09:29:33.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.288+0000 7f16d5e00640 1 -- 172.21.15.114:0/787507281 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c000b480 con 0x7f16d80c2a00 2024-05-27T09:29:33.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.288+0000 7f16dca00640 1 -- 172.21.15.114:0/787507281 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16d80c2a00 msgr2=0x7f16d8159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:33.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.288+0000 7f16dca00640 1 --2- 172.21.15.114:0/787507281 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16d80c2a00 0x7f16d8159900 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f16d80cb6d0 tx=0x7f16c002c940 comp rx=0 tx=0).stop 2024-05-27T09:29:33.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.288+0000 7f16dca00640 1 -- 172.21.15.114:0/787507281 shutdown_connections 2024-05-27T09:29:33.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.288+0000 7f16dca00640 1 --2- 172.21.15.114:0/787507281 >> v2:172.21.15.163:3300/0 conn(0x7f16d8159e40 0x7f16d8074140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:33.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.288+0000 7f16dca00640 1 --2- 172.21.15.114:0/787507281 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16d80c2a00 0x7f16d8159900 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:33.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.288+0000 7f16dca00640 1 -- 172.21.15.114:0/787507281 >> 172.21.15.114:0/787507281 conn(0x7f16d80ca320 msgr2=0x7f16d80ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:33.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.289+0000 7f16dca00640 1 -- 172.21.15.114:0/787507281 shutdown_connections 2024-05-27T09:29:33.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.289+0000 7f16dca00640 1 -- 172.21.15.114:0/787507281 wait complete. 2024-05-27T09:29:33.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.291+0000 7f16dca00640 1 Processor -- start 2024-05-27T09:29:33.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.291+0000 7f16dca00640 1 -- start start 2024-05-27T09:29:33.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.292+0000 7f16dca00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16d80c2a00 0x7f16d814dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:33.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.293+0000 7f16dca00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16d81534c0 0x7f16d8149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:33.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.293+0000 7f16dca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16d8159e40 0x7f16d814cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:33.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.293+0000 7f16dca00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f16d80c9f30 con 0x7f16d8159e40 2024-05-27T09:29:33.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.293+0000 7f16dca00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f16d80c9c50 con 0x7f16d80c2a00 2024-05-27T09:29:33.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.293+0000 7f16dca00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f16d81593e0 con 0x7f16d81534c0 2024-05-27T09:29:33.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.293+0000 7f16d7200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16d8159e40 0x7f16d814cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:33.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.293+0000 7f16d7200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16d8159e40 0x7f16d814cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:32892/0 (socket says 172.21.15.114:32892) 2024-05-27T09:29:33.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.293+0000 7f16d7200640 1 -- 172.21.15.114:0/600713295 learned_addr learned my addr 172.21.15.114:0/600713295 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:29:33.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.293+0000 7f16d6800640 1 --2- 172.21.15.114:0/600713295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16d80c2a00 0x7f16d814dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:33.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.293+0000 7f16cde00640 1 --2- 172.21.15.114:0/600713295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16d81534c0 0x7f16d8149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:33.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.293+0000 7f16d7200640 1 -- 172.21.15.114:0/600713295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16d81534c0 msgr2=0x7f16d8149840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:33.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.293+0000 7f16d7200640 1 --2- 172.21.15.114:0/600713295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16d81534c0 0x7f16d8149840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:33.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.293+0000 7f16d7200640 1 -- 172.21.15.114:0/600713295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16d80c2a00 msgr2=0x7f16d814dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:33.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.293+0000 7f16d7200640 1 --2- 172.21.15.114:0/600713295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16d80c2a00 0x7f16d814dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:33.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.293+0000 7f16d7200640 1 -- 172.21.15.114:0/600713295 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d81f5fb0 con 0x7f16d8159e40 2024-05-27T09:29:33.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.294+0000 7f16d7200640 1 --2- 172.21.15.114:0/600713295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16d8159e40 0x7f16d814cb30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f16d000e7f0 tx=0x7f16d000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:33.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.294+0000 7f16cf400640 1 -- 172.21.15.114:0/600713295 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0017070 con 0x7f16d8159e40 2024-05-27T09:29:33.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.294+0000 7f16cf400640 1 -- 172.21.15.114:0/600713295 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16d000c6d0 con 0x7f16d8159e40 2024-05-27T09:29:33.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.294+0000 7f16cf400640 1 -- 172.21.15.114:0/600713295 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d000cac0 con 0x7f16d8159e40 2024-05-27T09:29:33.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.294+0000 7f16dca00640 1 -- 172.21.15.114:0/600713295 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d8149d80 con 0x7f16d8159e40 2024-05-27T09:29:33.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.294+0000 7f16dca00640 1 -- 172.21.15.114:0/600713295 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d814b1e0 con 0x7f16d8159e40 2024-05-27T09:29:33.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.295+0000 7f16dca00640 1 -- 172.21.15.114:0/600713295 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16d80b3200 con 0x7f16d8159e40 2024-05-27T09:29:33.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.297+0000 7f16cf400640 1 -- 172.21.15.114:0/600713295 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f16d001a070 con 0x7f16d8159e40 2024-05-27T09:29:33.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.298+0000 7f16cf400640 1 --2- 172.21.15.114:0/600713295 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f16b40762d0 0x7f16b40787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:33.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.298+0000 7f16cf400640 1 -- 172.21.15.114:0/600713295 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5794+0+0 (secure 0 0 0) 0x7f16d0097640 con 0x7f16d8159e40 2024-05-27T09:29:33.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.298+0000 7f16d6800640 1 --2- 172.21.15.114:0/600713295 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f16b40762d0 0x7f16b40787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:33.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.299+0000 7f16d6800640 1 --2- 172.21.15.114:0/600713295 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f16b40762d0 0x7f16b40787e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f16c8000f60 tx=0x7f16c8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:33.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.303+0000 7f16cf400640 1 -- 172.21.15.114:0/600713295 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f16d80b3200 con 0x7f16d8159e40 2024-05-27T09:29:33.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.552+0000 7f16dca00640 1 -- 172.21.15.114:0/600713295 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16d8050860 con 0x7f16d8159e40 2024-05-27T09:29:33.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.553+0000 7f16cf400640 1 -- 172.21.15.114:0/600713295 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+6877 (secure 0 0 0) 0x7f16d0060960 con 0x7f16d8159e40 2024-05-27T09:29:33.555 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:29:33.555 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:27:33.841352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"f","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6839/444766663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":444766663},{"type":"v1","addr":"172.21.15.163:6839","nonce":444766663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4279,4297,4300]},"id":1}]} 2024-05-27T09:29:33.555 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 12 2024-05-27T09:29:33.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.556+0000 7f16dca00640 1 -- 172.21.15.114:0/600713295 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f16b40762d0 msgr2=0x7f16b40787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:33.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.556+0000 7f16dca00640 1 --2- 172.21.15.114:0/600713295 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f16b40762d0 0x7f16b40787e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f16c8000f60 tx=0x7f16c8009040 comp rx=0 tx=0).stop 2024-05-27T09:29:33.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.556+0000 7f16dca00640 1 -- 172.21.15.114:0/600713295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16d8159e40 msgr2=0x7f16d814cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:33.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.556+0000 7f16dca00640 1 --2- 172.21.15.114:0/600713295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16d8159e40 0x7f16d814cb30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f16d000e7f0 tx=0x7f16d000ecb0 comp rx=0 tx=0).stop 2024-05-27T09:29:33.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.557+0000 7f16dca00640 1 -- 172.21.15.114:0/600713295 shutdown_connections 2024-05-27T09:29:33.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.557+0000 7f16dca00640 1 --2- 172.21.15.114:0/600713295 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f16b40762d0 0x7f16b40787e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:33.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.557+0000 7f16dca00640 1 --2- 172.21.15.114:0/600713295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16d8159e40 0x7f16d814cb30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:33.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.557+0000 7f16dca00640 1 --2- 172.21.15.114:0/600713295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16d81534c0 0x7f16d8149840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:33.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.557+0000 7f16dca00640 1 --2- 172.21.15.114:0/600713295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16d80c2a00 0x7f16d814dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:33.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.557+0000 7f16dca00640 1 -- 172.21.15.114:0/600713295 >> 172.21.15.114:0/600713295 conn(0x7f16d80ca320 msgr2=0x7f16d8075da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:33.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.557+0000 7f16dca00640 1 -- 172.21.15.114:0/600713295 shutdown_connections 2024-05-27T09:29:33.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:33.557+0000 7f16dca00640 1 -- 172.21.15.114:0/600713295 wait complete. 2024-05-27T09:29:33.586 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-05-27T09:29:35.587 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:29:35.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.868+0000 7f0736800640 1 Processor -- start 2024-05-27T09:29:35.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.868+0000 7f0736800640 1 -- start start 2024-05-27T09:29:35.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.870+0000 7f0736800640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0730142720 0x7f0730073260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:35.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.870+0000 7f0736800640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07300739a0 con 0x7f07300c2990 2024-05-27T09:29:35.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.870+0000 7f0736800640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0730073c80 con 0x7f07301593b0 2024-05-27T09:29:35.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.870+0000 7f0736800640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f0730074d20 con 0x7f0730142720 2024-05-27T09:29:35.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.870+0000 7f072fe00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f07300c2990 0x7f0730163d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:52448/0 (socket says 172.21.15.114:52448) 2024-05-27T09:29:35.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.870+0000 7f072fe00640 1 -- 172.21.15.114:0/1555143438 learned_addr learned my addr 172.21.15.114:0/1555143438 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:29:35.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.870+0000 7f072ea00640 1 --2- 172.21.15.114:0/1555143438 >> v2:172.21.15.163:3300/0 conn(0x7f0730142720 0x7f0730073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:35.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.871+0000 7f072ea00640 1 -- 172.21.15.114:0/1555143438 >> v1:172.21.15.163:6789/0 conn(0x7f07301593b0 legacy=0x7f073015a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:35.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.871+0000 7f072ea00640 1 -- 172.21.15.114:0/1555143438 >> v1:172.21.15.114:6789/0 conn(0x7f07300c2990 legacy=0x7f0730163d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:35.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.871+0000 7f072ea00640 1 -- 172.21.15.114:0/1555143438 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0730073f60 con 0x7f0730142720 2024-05-27T09:29:35.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.872+0000 7f072ea00640 1 --2- 172.21.15.114:0/1555143438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0730142720 0x7f0730073260 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f071c0097a0 tx=0x7f071c02c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f790a2fc9381f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:35.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.872+0000 7f072e000640 1 -- 172.21.15.114:0/1555143438 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c038070 con 0x7f0730142720 2024-05-27T09:29:35.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.872+0000 7f072e000640 1 -- 172.21.15.114:0/1555143438 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f071c031710 con 0x7f0730142720 2024-05-27T09:29:35.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.872+0000 7f072e000640 1 -- 172.21.15.114:0/1555143438 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c031ae0 con 0x7f0730142720 2024-05-27T09:29:35.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.873+0000 7f0736800640 1 -- 172.21.15.114:0/1555143438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0730142720 msgr2=0x7f0730073260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:35.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.873+0000 7f0736800640 1 --2- 172.21.15.114:0/1555143438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0730142720 0x7f0730073260 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f071c0097a0 tx=0x7f071c02c4d0 comp rx=0 tx=0).stop 2024-05-27T09:29:35.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.873+0000 7f0736800640 1 -- 172.21.15.114:0/1555143438 shutdown_connections 2024-05-27T09:29:35.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.873+0000 7f0736800640 1 --2- 172.21.15.114:0/1555143438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0730142720 0x7f0730073260 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:35.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.873+0000 7f0736800640 1 -- 172.21.15.114:0/1555143438 >> 172.21.15.114:0/1555143438 conn(0x7f073006f6e0 msgr2=0x7f0730070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:35.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.873+0000 7f0736800640 1 -- 172.21.15.114:0/1555143438 shutdown_connections 2024-05-27T09:29:35.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.873+0000 7f0736800640 1 -- 172.21.15.114:0/1555143438 wait complete. 2024-05-27T09:29:35.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.875+0000 7f0736800640 1 Processor -- start 2024-05-27T09:29:35.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.876+0000 7f0736800640 1 -- start start 2024-05-27T09:29:35.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.877+0000 7f0736800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07300c2990 0x7f073015a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:35.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.877+0000 7f0736800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0730142720 0x7f073013f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:35.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.877+0000 7f0736800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07301593b0 0x7f07300c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:35.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.877+0000 7f0736800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0730074520 con 0x7f07301593b0 2024-05-27T09:29:35.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.877+0000 7f0736800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0730142b30 con 0x7f0730142720 2024-05-27T09:29:35.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.877+0000 7f0736800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f0730074800 con 0x7f07300c2990 2024-05-27T09:29:35.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.877+0000 7f072ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0730142720 0x7f073013f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:35.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.877+0000 7f072fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07301593b0 0x7f07300c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:35.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.877+0000 7f072ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0730142720 0x7f073013f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:60816/0 (socket says 172.21.15.114:60816) 2024-05-27T09:29:35.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.877+0000 7f072ea00640 1 -- 172.21.15.114:0/99470676 learned_addr learned my addr 172.21.15.114:0/99470676 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:29:35.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.877+0000 7f072f400640 1 --2- 172.21.15.114:0/99470676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07300c2990 0x7f073015a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:35.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.878+0000 7f072ea00640 1 -- 172.21.15.114:0/99470676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07300c2990 msgr2=0x7f073015a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:35.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.878+0000 7f072ea00640 1 --2- 172.21.15.114:0/99470676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07300c2990 0x7f073015a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:35.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.878+0000 7f072ea00640 1 -- 172.21.15.114:0/99470676 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07301593b0 msgr2=0x7f07300c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:35.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.878+0000 7f072ea00640 1 --2- 172.21.15.114:0/99470676 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07301593b0 0x7f07300c6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:35.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.878+0000 7f072ea00640 1 -- 172.21.15.114:0/99470676 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07301f2ad0 con 0x7f0730142720 2024-05-27T09:29:35.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.878+0000 7f072ea00640 1 --2- 172.21.15.114:0/99470676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0730142720 0x7f073013f5d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f071c036040 tx=0x7f071c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:35.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.879+0000 7f0727400640 1 -- 172.21.15.114:0/99470676 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c042020 con 0x7f0730142720 2024-05-27T09:29:35.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.879+0000 7f0727400640 1 -- 172.21.15.114:0/99470676 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f071c033830 con 0x7f0730142720 2024-05-27T09:29:35.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.879+0000 7f0736800640 1 -- 172.21.15.114:0/99470676 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07301f2d90 con 0x7f0730142720 2024-05-27T09:29:35.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.879+0000 7f0727400640 1 -- 172.21.15.114:0/99470676 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c038040 con 0x7f0730142720 2024-05-27T09:29:35.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.879+0000 7f0736800640 1 -- 172.21.15.114:0/99470676 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07301f3350 con 0x7f0730142720 2024-05-27T09:29:35.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.880+0000 7f0736800640 1 -- 172.21.15.114:0/99470676 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f073006a380 con 0x7f0730142720 2024-05-27T09:29:35.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.881+0000 7f0727400640 1 -- 172.21.15.114:0/99470676 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f071c033c40 con 0x7f0730142720 2024-05-27T09:29:35.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.882+0000 7f0727400640 1 --2- 172.21.15.114:0/99470676 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f07100762d0 0x7f07100787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:35.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.882+0000 7f0727400640 1 -- 172.21.15.114:0/99470676 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5766+0+0 (secure 0 0 0) 0x7f071c0b8900 con 0x7f0730142720 2024-05-27T09:29:35.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.883+0000 7f072f400640 1 --2- 172.21.15.114:0/99470676 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f07100762d0 0x7f07100787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:35.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.883+0000 7f072f400640 1 --2- 172.21.15.114:0/99470676 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f07100762d0 0x7f07100787e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0720000f60 tx=0x7f0720009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:35.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:35.886+0000 7f0727400640 1 -- 172.21.15.114:0/99470676 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f071c04c060 con 0x7f0730142720 2024-05-27T09:29:36.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:36.133+0000 7f0736800640 1 -- 172.21.15.114:0/99470676 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0730156d00 con 0x7f0730142720 2024-05-27T09:29:36.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:36.134+0000 7f0727400640 1 -- 172.21.15.114:0/99470676 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+6877 (secure 0 0 0) 0x7f071c08a280 con 0x7f0730142720 2024-05-27T09:29:36.136 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:29:36.136 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:27:33.841352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"f","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6839/444766663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":444766663},{"type":"v1","addr":"172.21.15.163:6839","nonce":444766663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4279,4297,4300]},"id":1}]} 2024-05-27T09:29:36.136 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 12 2024-05-27T09:29:36.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:36.137+0000 7f0736800640 1 -- 172.21.15.114:0/99470676 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f07100762d0 msgr2=0x7f07100787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:36.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:36.137+0000 7f0736800640 1 --2- 172.21.15.114:0/99470676 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f07100762d0 0x7f07100787e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0720000f60 tx=0x7f0720009040 comp rx=0 tx=0).stop 2024-05-27T09:29:36.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:36.137+0000 7f0736800640 1 -- 172.21.15.114:0/99470676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0730142720 msgr2=0x7f073013f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:36.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:36.137+0000 7f0736800640 1 --2- 172.21.15.114:0/99470676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0730142720 0x7f073013f5d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f071c036040 tx=0x7f071c0046c0 comp rx=0 tx=0).stop 2024-05-27T09:29:36.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:36.137+0000 7f0736800640 1 -- 172.21.15.114:0/99470676 shutdown_connections 2024-05-27T09:29:36.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:36.137+0000 7f0736800640 1 --2- 172.21.15.114:0/99470676 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f07100762d0 0x7f07100787e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:36.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:36.137+0000 7f0736800640 1 --2- 172.21.15.114:0/99470676 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07301593b0 0x7f07300c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:36.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:36.137+0000 7f0736800640 1 --2- 172.21.15.114:0/99470676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0730142720 0x7f073013f5d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:36.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:36.137+0000 7f0736800640 1 --2- 172.21.15.114:0/99470676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07300c2990 0x7f073015a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:36.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:36.137+0000 7f0736800640 1 -- 172.21.15.114:0/99470676 >> 172.21.15.114:0/99470676 conn(0x7f073006f6e0 msgr2=0x7f07300cb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:36.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:36.137+0000 7f0736800640 1 -- 172.21.15.114:0/99470676 shutdown_connections 2024-05-27T09:29:36.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:36.138+0000 7f0736800640 1 -- 172.21.15.114:0/99470676 wait complete. 2024-05-27T09:29:36.168 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-05-27T09:29:38.169 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:29:38.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.452+0000 7fd49d200640 1 Processor -- start 2024-05-27T09:29:38.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.453+0000 7fd49d200640 1 -- start start 2024-05-27T09:29:38.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.455+0000 7fd49d200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd4981593b0 0x7fd498142720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:38.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.455+0000 7fd49d200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4980c2990 0x7fd4980c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:38.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.455+0000 7fd49d200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4980739a0 con 0x7fd498073260 2024-05-27T09:29:38.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.455+0000 7fd49d200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fd498074d20 con 0x7fd4981593b0 2024-05-27T09:29:38.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.455+0000 7fd49d200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fd498073c80 con 0x7fd4980c2990 2024-05-27T09:29:38.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.455+0000 7fd495e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd4981593b0 0x7fd498142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:38.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.455+0000 7fd495e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd4981593b0 0x7fd498142720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34414/0 (socket says 172.21.15.114:34414) 2024-05-27T09:29:38.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.455+0000 7fd495e00640 1 -- 172.21.15.114:0/1922684522 learned_addr learned my addr 172.21.15.114:0/1922684522 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:29:38.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.455+0000 7fd48fe00640 1 --1- 172.21.15.114:0/1922684522 >> v1:172.21.15.114:6789/0 conn(0x7fd498073260 0x7fd49815a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:52464/0 (socket says 172.21.15.114:52464) 2024-05-27T09:29:38.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.455+0000 7fd496800640 1 --2- 172.21.15.114:0/1922684522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4980c2990 0x7fd4980c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:38.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.456+0000 7fd495e00640 1 -- 172.21.15.114:0/1922684522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4980c2990 msgr2=0x7fd4980c2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:29:38.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.456+0000 7fd495e00640 1 --2- 172.21.15.114:0/1922684522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4980c2990 0x7fd4980c2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:38.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.456+0000 7fd495e00640 1 -- 172.21.15.114:0/1922684522 >> v1:172.21.15.114:6789/0 conn(0x7fd498073260 legacy=0x7fd49815a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:38.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.456+0000 7fd495e00640 1 -- 172.21.15.114:0/1922684522 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd498073f60 con 0x7fd4981593b0 2024-05-27T09:29:38.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.456+0000 7fd495400640 1 -- 172.21.15.114:0/1922684522 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3109554509 0 0) 0x7fd4980739a0 con 0x7fd498073260 2024-05-27T09:29:38.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.457+0000 7fd495e00640 1 --2- 172.21.15.114:0/1922684522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4981593b0 0x7fd498142720 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd488009870 tx=0x7fd48802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805bb9488a55f7aa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:38.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.457+0000 7fd495400640 1 -- 172.21.15.114:0/1922684522 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd488037070 con 0x7fd4981593b0 2024-05-27T09:29:38.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.457+0000 7fd495400640 1 -- 172.21.15.114:0/1922684522 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd48802ae50 con 0x7fd4981593b0 2024-05-27T09:29:38.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.457+0000 7fd495400640 1 -- 172.21.15.114:0/1922684522 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd488032860 con 0x7fd4981593b0 2024-05-27T09:29:38.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.458+0000 7fd49d200640 1 -- 172.21.15.114:0/1922684522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4981593b0 msgr2=0x7fd498142720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:38.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.458+0000 7fd49d200640 1 --2- 172.21.15.114:0/1922684522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4981593b0 0x7fd498142720 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd488009870 tx=0x7fd48802a300 comp rx=0 tx=0).stop 2024-05-27T09:29:38.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.458+0000 7fd49d200640 1 -- 172.21.15.114:0/1922684522 shutdown_connections 2024-05-27T09:29:38.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.458+0000 7fd49d200640 1 --2- 172.21.15.114:0/1922684522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4980c2990 0x7fd4980c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:38.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.458+0000 7fd49d200640 1 --2- 172.21.15.114:0/1922684522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd4981593b0 0x7fd498142720 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:38.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.458+0000 7fd49d200640 1 -- 172.21.15.114:0/1922684522 >> 172.21.15.114:0/1922684522 conn(0x7fd49806f6e0 msgr2=0x7fd498070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:38.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.458+0000 7fd49d200640 1 -- 172.21.15.114:0/1922684522 shutdown_connections 2024-05-27T09:29:38.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.458+0000 7fd49d200640 1 -- 172.21.15.114:0/1922684522 wait complete. 2024-05-27T09:29:38.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.460+0000 7fd49d200640 1 Processor -- start 2024-05-27T09:29:38.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.461+0000 7fd49d200640 1 -- start start 2024-05-27T09:29:38.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.462+0000 7fd49d200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd498073260 0x7fd4981f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:38.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.462+0000 7fd49d200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4980c2990 0x7fd4981413f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:38.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.462+0000 7fd49d200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4981593b0 0x7fd49813d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:38.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.462+0000 7fd49d200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd498074800 con 0x7fd498073260 2024-05-27T09:29:38.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.462+0000 7fd49d200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd498074520 con 0x7fd4980c2990 2024-05-27T09:29:38.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.462+0000 7fd49d200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fd498073670 con 0x7fd4981593b0 2024-05-27T09:29:38.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.462+0000 7fd495e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd498073260 0x7fd4981f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:38.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.462+0000 7fd495e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd498073260 0x7fd4981f2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34420/0 (socket says 172.21.15.114:34420) 2024-05-27T09:29:38.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.462+0000 7fd495e00640 1 -- 172.21.15.114:0/363268839 learned_addr learned my addr 172.21.15.114:0/363268839 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:29:38.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.462+0000 7fd496800640 1 --2- 172.21.15.114:0/363268839 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4981593b0 0x7fd49813d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:38.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.463+0000 7fd495e00640 1 -- 172.21.15.114:0/363268839 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4981593b0 msgr2=0x7fd49813d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:38.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.463+0000 7fd495e00640 1 --2- 172.21.15.114:0/363268839 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4981593b0 0x7fd49813d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:38.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.463+0000 7fd495e00640 1 -- 172.21.15.114:0/363268839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4980c2990 msgr2=0x7fd4981413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:38.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.463+0000 7fd495e00640 1 --2- 172.21.15.114:0/363268839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4980c2990 0x7fd4981413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:38.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.463+0000 7fd495e00640 1 -- 172.21.15.114:0/363268839 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4981f2fd0 con 0x7fd498073260 2024-05-27T09:29:38.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.463+0000 7fd495e00640 1 --2- 172.21.15.114:0/363268839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd498073260 0x7fd4981f2a90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd488009870 tx=0x7fd4880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:38.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.463+0000 7fd48e000640 1 -- 172.21.15.114:0/363268839 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd488041070 con 0x7fd498073260 2024-05-27T09:29:38.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.463+0000 7fd48e000640 1 -- 172.21.15.114:0/363268839 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd48803c660 con 0x7fd498073260 2024-05-27T09:29:38.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.463+0000 7fd49d200640 1 -- 172.21.15.114:0/363268839 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4981ecbe0 con 0x7fd498073260 2024-05-27T09:29:38.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.463+0000 7fd48e000640 1 -- 172.21.15.114:0/363268839 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd488037040 con 0x7fd498073260 2024-05-27T09:29:38.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.464+0000 7fd49d200640 1 -- 172.21.15.114:0/363268839 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd49815abd0 con 0x7fd498073260 2024-05-27T09:29:38.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.465+0000 7fd49d200640 1 -- 172.21.15.114:0/363268839 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd49806a380 con 0x7fd498073260 2024-05-27T09:29:38.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.466+0000 7fd48e000640 1 -- 172.21.15.114:0/363268839 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd488044070 con 0x7fd498073260 2024-05-27T09:29:38.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.467+0000 7fd48e000640 1 --2- 172.21.15.114:0/363268839 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd4740762d0 0x7fd4740787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:38.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.467+0000 7fd48e000640 1 -- 172.21.15.114:0/363268839 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5766+0+0 (secure 0 0 0) 0x7fd4880b7c30 con 0x7fd498073260 2024-05-27T09:29:38.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.467+0000 7fd48fe00640 1 --2- 172.21.15.114:0/363268839 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd4740762d0 0x7fd4740787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:38.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.468+0000 7fd48fe00640 1 --2- 172.21.15.114:0/363268839 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd4740762d0 0x7fd4740787e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd480007fd0 tx=0x7fd480007960 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:38.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.472+0000 7fd48e000640 1 -- 172.21.15.114:0/363268839 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd488080e60 con 0x7fd498073260 2024-05-27T09:29:38.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.715+0000 7fd49d200640 1 -- 172.21.15.114:0/363268839 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4980c41f0 con 0x7fd498073260 2024-05-27T09:29:38.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.717+0000 7fd48e000640 1 -- 172.21.15.114:0/363268839 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+6877 (secure 0 0 0) 0x7fd488085160 con 0x7fd498073260 2024-05-27T09:29:38.718 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:29:38.718 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:27:33.841352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"f","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6839/444766663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":444766663},{"type":"v1","addr":"172.21.15.163:6839","nonce":444766663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4279,4297,4300]},"id":1}]} 2024-05-27T09:29:38.719 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 12 2024-05-27T09:29:38.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.719+0000 7fd49d200640 1 -- 172.21.15.114:0/363268839 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd4740762d0 msgr2=0x7fd4740787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:38.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.719+0000 7fd49d200640 1 --2- 172.21.15.114:0/363268839 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd4740762d0 0x7fd4740787e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd480007fd0 tx=0x7fd480007960 comp rx=0 tx=0).stop 2024-05-27T09:29:38.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.719+0000 7fd49d200640 1 -- 172.21.15.114:0/363268839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd498073260 msgr2=0x7fd4981f2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:38.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.719+0000 7fd49d200640 1 --2- 172.21.15.114:0/363268839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd498073260 0x7fd4981f2a90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd488009870 tx=0x7fd4880046c0 comp rx=0 tx=0).stop 2024-05-27T09:29:38.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.720+0000 7fd49d200640 1 -- 172.21.15.114:0/363268839 shutdown_connections 2024-05-27T09:29:38.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.720+0000 7fd49d200640 1 --2- 172.21.15.114:0/363268839 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd4740762d0 0x7fd4740787e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:38.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.720+0000 7fd49d200640 1 --2- 172.21.15.114:0/363268839 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4981593b0 0x7fd49813d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:38.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.720+0000 7fd49d200640 1 --2- 172.21.15.114:0/363268839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4980c2990 0x7fd4981413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:38.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.720+0000 7fd49d200640 1 --2- 172.21.15.114:0/363268839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd498073260 0x7fd4981f2a90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:38.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.720+0000 7fd49d200640 1 -- 172.21.15.114:0/363268839 >> 172.21.15.114:0/363268839 conn(0x7fd49806f6e0 msgr2=0x7fd49813f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:38.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.720+0000 7fd49d200640 1 -- 172.21.15.114:0/363268839 shutdown_connections 2024-05-27T09:29:38.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:38.720+0000 7fd49d200640 1 -- 172.21.15.114:0/363268839 wait complete. 2024-05-27T09:29:38.746 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-05-27T09:29:40.746 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:29:41.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.028+0000 7f5ea1800640 1 Processor -- start 2024-05-27T09:29:41.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.028+0000 7f5ea1800640 1 -- start start 2024-05-27T09:29:41.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.030+0000 7f5ea1800640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5e9c0b3640 0x7f5e9c070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:41.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.030+0000 7f5ea1800640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e9c0c93b0 con 0x7f5e9c0b3c00 2024-05-27T09:29:41.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.030+0000 7f5ea1800640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e9c0c9630 con 0x7f5e9c070350 2024-05-27T09:29:41.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.030+0000 7f5ea1800640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f5e9c0cbbb0 con 0x7f5e9c0b3640 2024-05-27T09:29:41.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.031+0000 7f5e93e00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5e9c0b3640 0x7f5e9c070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:41.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.031+0000 7f5e9be00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f5e9c0b3c00 0x7f5e9c0cb4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:52466/0 (socket says 172.21.15.114:52466) 2024-05-27T09:29:41.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.031+0000 7f5ea0e00640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5e9c070350 0x7f5e9c156410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:40316/0 (socket says 172.21.15.114:40316) 2024-05-27T09:29:41.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.031+0000 7f5e93e00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5e9c0b3640 0x7f5e9c070e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:60832/0 (socket says 172.21.15.114:60832) 2024-05-27T09:29:41.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.031+0000 7f5e9be00640 1 -- 172.21.15.114:0/1862755367 learned_addr learned my addr 172.21.15.114:0/1862755367 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:29:41.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.031+0000 7f5e93e00640 1 -- 172.21.15.114:0/1862755367 >> v1:172.21.15.163:6789/0 conn(0x7f5e9c070350 legacy=0x7f5e9c156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:41.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.031+0000 7f5e93e00640 1 -- 172.21.15.114:0/1862755367 >> v1:172.21.15.114:6789/0 conn(0x7f5e9c0b3c00 legacy=0x7f5e9c0cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:41.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.031+0000 7f5e93e00640 1 -- 172.21.15.114:0/1862755367 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9c072660 con 0x7f5e9c0b3640 2024-05-27T09:29:41.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.031+0000 7f5e9b400640 1 -- 172.21.15.114:0/1862755367 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3428543508 0 0) 0x7f5e8c001040 con 0x7f5e9c0b3c00 2024-05-27T09:29:41.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.032+0000 7f5e93e00640 1 --2- 172.21.15.114:0/1862755367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e9c0b3640 0x7f5e9c070e80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5e840097a0 tx=0x7f5e8402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5b2e600516c2956 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:41.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.032+0000 7f5e9b400640 1 -- 172.21.15.114:0/1862755367 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e84037070 con 0x7f5e9c0b3640 2024-05-27T09:29:41.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.033+0000 7f5e9b400640 1 -- 172.21.15.114:0/1862755367 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e84032440 con 0x7f5e9c0b3640 2024-05-27T09:29:41.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.033+0000 7f5e9b400640 1 -- 172.21.15.114:0/1862755367 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e84032810 con 0x7f5e9c0b3640 2024-05-27T09:29:41.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.033+0000 7f5ea1800640 1 -- 172.21.15.114:0/1862755367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e9c0b3640 msgr2=0x7f5e9c070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:41.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.033+0000 7f5ea1800640 1 --2- 172.21.15.114:0/1862755367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e9c0b3640 0x7f5e9c070e80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5e840097a0 tx=0x7f5e8402a230 comp rx=0 tx=0).stop 2024-05-27T09:29:41.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.033+0000 7f5ea1800640 1 -- 172.21.15.114:0/1862755367 shutdown_connections 2024-05-27T09:29:41.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.033+0000 7f5ea1800640 1 --2- 172.21.15.114:0/1862755367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e9c0b3640 0x7f5e9c070e80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:41.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.033+0000 7f5ea1800640 1 -- 172.21.15.114:0/1862755367 >> 172.21.15.114:0/1862755367 conn(0x7f5e9c06d7a0 msgr2=0x7f5e9c06dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:41.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.033+0000 7f5ea1800640 1 -- 172.21.15.114:0/1862755367 shutdown_connections 2024-05-27T09:29:41.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.033+0000 7f5ea1800640 1 -- 172.21.15.114:0/1862755367 wait complete. 2024-05-27T09:29:41.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.036+0000 7f5ea1800640 1 Processor -- start 2024-05-27T09:29:41.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.036+0000 7f5ea1800640 1 -- start start 2024-05-27T09:29:41.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.037+0000 7f5ea1800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e9c070350 0x7f5e9c0c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:41.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.037+0000 7f5ea1800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e9c0b3640 0x7f5e9c14c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:41.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.037+0000 7f5ea1800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e9c0b3c00 0x7f5e9c0cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:41.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.037+0000 7f5ea1800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c156b20 con 0x7f5e9c070350 2024-05-27T09:29:41.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.037+0000 7f5ea1800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c072410 con 0x7f5e9c0b3c00 2024-05-27T09:29:41.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.037+0000 7f5ea1800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f5e9c156ca0 con 0x7f5e9c0b3640 2024-05-27T09:29:41.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.037+0000 7f5e9be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e9c070350 0x7f5e9c0c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:41.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.038+0000 7f5e9be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e9c070350 0x7f5e9c0c7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34430/0 (socket says 172.21.15.114:34430) 2024-05-27T09:29:41.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.038+0000 7f5e9be00640 1 -- 172.21.15.114:0/4264357928 learned_addr learned my addr 172.21.15.114:0/4264357928 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:29:41.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.038+0000 7f5e93e00640 1 --2- 172.21.15.114:0/4264357928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e9c0b3640 0x7f5e9c14c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:41.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.038+0000 7f5ea0e00640 1 --2- 172.21.15.114:0/4264357928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e9c0b3c00 0x7f5e9c0cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:41.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.038+0000 7f5e9be00640 1 -- 172.21.15.114:0/4264357928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e9c0b3640 msgr2=0x7f5e9c14c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:41.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.038+0000 7f5e9be00640 1 --2- 172.21.15.114:0/4264357928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e9c0b3640 0x7f5e9c14c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:41.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.038+0000 7f5e9be00640 1 -- 172.21.15.114:0/4264357928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e9c0b3c00 msgr2=0x7f5e9c0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:41.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.038+0000 7f5e9be00640 1 --2- 172.21.15.114:0/4264357928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e9c0b3c00 0x7f5e9c0cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:41.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.038+0000 7f5e9be00640 1 -- 172.21.15.114:0/4264357928 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9c0c7e60 con 0x7f5e9c070350 2024-05-27T09:29:41.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.038+0000 7f5e9be00640 1 --2- 172.21.15.114:0/4264357928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e9c070350 0x7f5e9c0c7920 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c00c390 tx=0x7f5e8c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:41.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.039+0000 7f5e98c00640 1 -- 172.21.15.114:0/4264357928 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c0074e0 con 0x7f5e9c070350 2024-05-27T09:29:41.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.039+0000 7f5e98c00640 1 -- 172.21.15.114:0/4264357928 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e8c007680 con 0x7f5e9c070350 2024-05-27T09:29:41.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.039+0000 7f5ea1800640 1 -- 172.21.15.114:0/4264357928 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e9c154cb0 con 0x7f5e9c070350 2024-05-27T09:29:41.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.039+0000 7f5e98c00640 1 -- 172.21.15.114:0/4264357928 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c004a20 con 0x7f5e9c070350 2024-05-27T09:29:41.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.039+0000 7f5ea1800640 1 -- 172.21.15.114:0/4264357928 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e9c0c88b0 con 0x7f5e9c070350 2024-05-27T09:29:41.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.040+0000 7f5ea1800640 1 -- 172.21.15.114:0/4264357928 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e9c0ca0f0 con 0x7f5e9c070350 2024-05-27T09:29:41.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.041+0000 7f5e98c00640 1 -- 172.21.15.114:0/4264357928 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5e8c011070 con 0x7f5e9c070350 2024-05-27T09:29:41.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.042+0000 7f5e98c00640 1 --2- 172.21.15.114:0/4264357928 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5e780762d0 0x7f5e780787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:41.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.042+0000 7f5e98c00640 1 -- 172.21.15.114:0/4264357928 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5766+0+0 (secure 0 0 0) 0x7f5e8c097070 con 0x7f5e9c070350 2024-05-27T09:29:41.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.042+0000 7f5e93e00640 1 --2- 172.21.15.114:0/4264357928 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5e780762d0 0x7f5e780787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:41.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.043+0000 7f5e93e00640 1 --2- 172.21.15.114:0/4264357928 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5e780762d0 0x7f5e780787e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5e84034000 tx=0x7f5e8402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:41.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.047+0000 7f5e98c00640 1 -- 172.21.15.114:0/4264357928 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5e9c0ca0f0 con 0x7f5e9c070350 2024-05-27T09:29:41.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.296+0000 7f5ea1800640 1 -- 172.21.15.114:0/4264357928 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e9c069090 con 0x7f5e9c070350 2024-05-27T09:29:41.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.298+0000 7f5e98c00640 1 -- 172.21.15.114:0/4264357928 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+6877 (secure 0 0 0) 0x7f5e9c069090 con 0x7f5e9c070350 2024-05-27T09:29:41.299 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:29:41.299 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:27:33.841352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"f","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6839/444766663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":444766663},{"type":"v1","addr":"172.21.15.163:6839","nonce":444766663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4279,4297,4300]},"id":1}]} 2024-05-27T09:29:41.300 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 12 2024-05-27T09:29:41.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.300+0000 7f5ea1800640 1 -- 172.21.15.114:0/4264357928 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5e780762d0 msgr2=0x7f5e780787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:41.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.300+0000 7f5ea1800640 1 --2- 172.21.15.114:0/4264357928 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5e780762d0 0x7f5e780787e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5e84034000 tx=0x7f5e8402e000 comp rx=0 tx=0).stop 2024-05-27T09:29:41.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.300+0000 7f5ea1800640 1 -- 172.21.15.114:0/4264357928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e9c070350 msgr2=0x7f5e9c0c7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:41.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.300+0000 7f5ea1800640 1 --2- 172.21.15.114:0/4264357928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e9c070350 0x7f5e9c0c7920 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c00c390 tx=0x7f5e8c00c850 comp rx=0 tx=0).stop 2024-05-27T09:29:41.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.301+0000 7f5ea1800640 1 -- 172.21.15.114:0/4264357928 shutdown_connections 2024-05-27T09:29:41.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.301+0000 7f5ea1800640 1 --2- 172.21.15.114:0/4264357928 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5e780762d0 0x7f5e780787e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:41.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.301+0000 7f5ea1800640 1 --2- 172.21.15.114:0/4264357928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e9c0b3c00 0x7f5e9c0cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:41.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.301+0000 7f5ea1800640 1 --2- 172.21.15.114:0/4264357928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e9c0b3640 0x7f5e9c14c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:41.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.301+0000 7f5ea1800640 1 --2- 172.21.15.114:0/4264357928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e9c070350 0x7f5e9c0c7920 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:41.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.301+0000 7f5ea1800640 1 -- 172.21.15.114:0/4264357928 >> 172.21.15.114:0/4264357928 conn(0x7f5e9c06d7a0 msgr2=0x7f5e9c06dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:41.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.301+0000 7f5ea1800640 1 -- 172.21.15.114:0/4264357928 shutdown_connections 2024-05-27T09:29:41.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:41.301+0000 7f5ea1800640 1 -- 172.21.15.114:0/4264357928 wait complete. 2024-05-27T09:29:41.329 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-05-27T09:29:43.330 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:29:43.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.607+0000 7fa145400640 1 Processor -- start 2024-05-27T09:29:43.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.607+0000 7fa145400640 1 -- start start 2024-05-27T09:29:43.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.609+0000 7fa145400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa1400b2c60 0x7fa14015d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:43.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.609+0000 7fa145400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa1400ca140 0x7fa1400ca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:43.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.609+0000 7fa145400640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa140156f80 con 0x7fa140073310 2024-05-27T09:29:43.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.609+0000 7fa145400640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa14015d7c0 con 0x7fa1400ca140 2024-05-27T09:29:43.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.609+0000 7fa145400640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fa14015fd50 con 0x7fa1400b2c60 2024-05-27T09:29:43.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.609+0000 7fa135e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa1400ca140 0x7fa1400ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:43.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.609+0000 7fa135e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa1400ca140 0x7fa1400ca550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34432/0 (socket says 172.21.15.114:34432) 2024-05-27T09:29:43.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.609+0000 7fa135e00640 1 -- 172.21.15.114:0/3292383000 learned_addr learned my addr 172.21.15.114:0/3292383000 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:29:43.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.609+0000 7fa13e800640 1 --2- 172.21.15.114:0/3292383000 >> v2:172.21.15.163:3300/0 conn(0x7fa1400b2c60 0x7fa14015d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:43.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.610+0000 7fa135e00640 1 -- 172.21.15.114:0/3292383000 >> v2:172.21.15.163:3300/0 conn(0x7fa1400b2c60 msgr2=0x7fa14015d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:29:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.610+0000 7fa135e00640 1 --2- 172.21.15.114:0/3292383000 >> v2:172.21.15.163:3300/0 conn(0x7fa1400b2c60 0x7fa14015d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.610+0000 7fa135e00640 1 -- 172.21.15.114:0/3292383000 >> v1:172.21.15.163:6789/0 conn(0x7fa140073310 legacy=0x7fa1400c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.610+0000 7fa13de00640 1 -- 172.21.15.114:0/3292383000 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3753536704 0 0) 0x7fa140156f80 con 0x7fa140073310 2024-05-27T09:29:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.610+0000 7fa135e00640 1 -- 172.21.15.114:0/3292383000 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1400cac50 con 0x7fa1400ca140 2024-05-27T09:29:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.611+0000 7fa135e00640 1 --2- 172.21.15.114:0/3292383000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1400ca140 0x7fa1400ca550 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa14006d780 tx=0x7fa12802a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f75171882b498d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:43.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.611+0000 7fa13de00640 1 -- 172.21.15.114:0/3292383000 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa128036070 con 0x7fa1400ca140 2024-05-27T09:29:43.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.611+0000 7fa13de00640 1 -- 172.21.15.114:0/3292383000 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa128031600 con 0x7fa1400ca140 2024-05-27T09:29:43.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.611+0000 7fa13de00640 1 -- 172.21.15.114:0/3292383000 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1280319f0 con 0x7fa1400ca140 2024-05-27T09:29:43.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.612+0000 7fa145400640 1 -- 172.21.15.114:0/3292383000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1400ca140 msgr2=0x7fa1400ca550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:43.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.612+0000 7fa145400640 1 --2- 172.21.15.114:0/3292383000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1400ca140 0x7fa1400ca550 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa14006d780 tx=0x7fa12802a7c0 comp rx=0 tx=0).stop 2024-05-27T09:29:43.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.612+0000 7fa145400640 1 -- 172.21.15.114:0/3292383000 shutdown_connections 2024-05-27T09:29:43.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.612+0000 7fa145400640 1 --2- 172.21.15.114:0/3292383000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1400ca140 0x7fa1400ca550 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:43.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.612+0000 7fa145400640 1 --2- 172.21.15.114:0/3292383000 >> v2:172.21.15.163:3300/0 conn(0x7fa1400b2c60 0x7fa14015d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:43.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.612+0000 7fa145400640 1 -- 172.21.15.114:0/3292383000 >> 172.21.15.114:0/3292383000 conn(0x7fa14006f6e0 msgr2=0x7fa140070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:43.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.612+0000 7fa145400640 1 -- 172.21.15.114:0/3292383000 shutdown_connections 2024-05-27T09:29:43.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.612+0000 7fa145400640 1 -- 172.21.15.114:0/3292383000 wait complete. 2024-05-27T09:29:43.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.615+0000 7fa145400640 1 Processor -- start 2024-05-27T09:29:43.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.615+0000 7fa145400640 1 -- start start 2024-05-27T09:29:43.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.616+0000 7fa145400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa140073310 0x7fa1401e6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:43.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.616+0000 7fa145400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1400b2c60 0x7fa14015b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:43.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.616+0000 7fa145400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1400ca140 0x7fa1400b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:43.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.616+0000 7fa145400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa140160560 con 0x7fa140073310 2024-05-27T09:29:43.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.616+0000 7fa145400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa140160280 con 0x7fa1400ca140 2024-05-27T09:29:43.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.616+0000 7fa145400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa140160840 con 0x7fa1400b2c60 2024-05-27T09:29:43.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.617+0000 7fa13e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa140073310 0x7fa1401e6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:43.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.617+0000 7fa13e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa140073310 0x7fa1401e6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34444/0 (socket says 172.21.15.114:34444) 2024-05-27T09:29:43.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.617+0000 7fa13e800640 1 -- 172.21.15.114:0/2132965452 learned_addr learned my addr 172.21.15.114:0/2132965452 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:29:43.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.617+0000 7fa135e00640 1 --2- 172.21.15.114:0/2132965452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1400b2c60 0x7fa14015b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:43.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.617+0000 7fa13f200640 1 --2- 172.21.15.114:0/2132965452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1400ca140 0x7fa1400b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:43.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.617+0000 7fa13e800640 1 -- 172.21.15.114:0/2132965452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1400b2c60 msgr2=0x7fa14015b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:43.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.617+0000 7fa13e800640 1 --2- 172.21.15.114:0/2132965452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1400b2c60 0x7fa14015b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:43.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.617+0000 7fa13e800640 1 -- 172.21.15.114:0/2132965452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1400ca140 msgr2=0x7fa1400b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:43.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.617+0000 7fa13e800640 1 --2- 172.21.15.114:0/2132965452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1400ca140 0x7fa1400b17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:43.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.617+0000 7fa13e800640 1 -- 172.21.15.114:0/2132965452 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1401e67c0 con 0x7fa140073310 2024-05-27T09:29:43.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.617+0000 7fa13e800640 1 --2- 172.21.15.114:0/2132965452 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa140073310 0x7fa1401e6280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa13000e8a0 tx=0x7fa13000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:43.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.618+0000 7fa137400640 1 -- 172.21.15.114:0/2132965452 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa130017070 con 0x7fa140073310 2024-05-27T09:29:43.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.618+0000 7fa137400640 1 -- 172.21.15.114:0/2132965452 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa13000c940 con 0x7fa140073310 2024-05-27T09:29:43.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.618+0000 7fa137400640 1 -- 172.21.15.114:0/2132965452 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13000cd30 con 0x7fa140073310 2024-05-27T09:29:43.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.618+0000 7fa145400640 1 -- 172.21.15.114:0/2132965452 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1400afd50 con 0x7fa140073310 2024-05-27T09:29:43.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.618+0000 7fa145400640 1 -- 172.21.15.114:0/2132965452 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1401e5a80 con 0x7fa140073310 2024-05-27T09:29:43.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.619+0000 7fa145400640 1 -- 172.21.15.114:0/2132965452 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa14006a380 con 0x7fa140073310 2024-05-27T09:29:43.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.620+0000 7fa137400640 1 -- 172.21.15.114:0/2132965452 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa13001a070 con 0x7fa140073310 2024-05-27T09:29:43.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.621+0000 7fa137400640 1 --2- 172.21.15.114:0/2132965452 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa11c0762d0 0x7fa11c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:43.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.621+0000 7fa137400640 1 -- 172.21.15.114:0/2132965452 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5766+0+0 (secure 0 0 0) 0x7fa130097890 con 0x7fa140073310 2024-05-27T09:29:43.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.622+0000 7fa135e00640 1 --2- 172.21.15.114:0/2132965452 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa11c0762d0 0x7fa11c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:43.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.622+0000 7fa135e00640 1 --2- 172.21.15.114:0/2132965452 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa11c0762d0 0x7fa11c0787e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa128033000 tx=0x7fa128009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:43.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.627+0000 7fa137400640 1 -- 172.21.15.114:0/2132965452 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa130060ac0 con 0x7fa140073310 2024-05-27T09:29:43.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.870+0000 7fa145400640 1 -- 172.21.15.114:0/2132965452 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa140074660 con 0x7fa140073310 2024-05-27T09:29:43.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.871+0000 7fa137400640 1 -- 172.21.15.114:0/2132965452 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+6877 (secure 0 0 0) 0x7fa130064dc0 con 0x7fa140073310 2024-05-27T09:29:43.872 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:29:43.873 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4256,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:27:33.841352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"f","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6839/444766663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":444766663},{"type":"v1","addr":"172.21.15.163:6839","nonce":444766663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4279,4297,4300]},"id":1}]} 2024-05-27T09:29:43.873 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 12 2024-05-27T09:29:43.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.874+0000 7fa145400640 1 -- 172.21.15.114:0/2132965452 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa11c0762d0 msgr2=0x7fa11c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:43.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.874+0000 7fa145400640 1 --2- 172.21.15.114:0/2132965452 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa11c0762d0 0x7fa11c0787e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa128033000 tx=0x7fa128009000 comp rx=0 tx=0).stop 2024-05-27T09:29:43.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.874+0000 7fa145400640 1 -- 172.21.15.114:0/2132965452 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa140073310 msgr2=0x7fa1401e6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:43.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.874+0000 7fa145400640 1 --2- 172.21.15.114:0/2132965452 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa140073310 0x7fa1401e6280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa13000e8a0 tx=0x7fa13000ed60 comp rx=0 tx=0).stop 2024-05-27T09:29:43.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.874+0000 7fa145400640 1 -- 172.21.15.114:0/2132965452 shutdown_connections 2024-05-27T09:29:43.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.874+0000 7fa145400640 1 --2- 172.21.15.114:0/2132965452 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa11c0762d0 0x7fa11c0787e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:43.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.874+0000 7fa145400640 1 --2- 172.21.15.114:0/2132965452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1400ca140 0x7fa1400b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:43.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.874+0000 7fa145400640 1 --2- 172.21.15.114:0/2132965452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1400b2c60 0x7fa14015b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:43.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.874+0000 7fa145400640 1 --2- 172.21.15.114:0/2132965452 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa140073310 0x7fa1401e6280 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:43.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.874+0000 7fa145400640 1 -- 172.21.15.114:0/2132965452 >> 172.21.15.114:0/2132965452 conn(0x7fa14006f6e0 msgr2=0x7fa14015f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:43.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.874+0000 7fa145400640 1 -- 172.21.15.114:0/2132965452 shutdown_connections 2024-05-27T09:29:43.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:43.874+0000 7fa145400640 1 -- 172.21.15.114:0/2132965452 wait complete. 2024-05-27T09:29:43.903 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-05-27T09:29:45.904 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:29:46.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.171+0000 7f8acce00640 1 Processor -- start 2024-05-27T09:29:46.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.171+0000 7f8acce00640 1 -- start start 2024-05-27T09:29:46.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.173+0000 7f8acce00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8ac8154820 0x7f8ac80ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:46.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.173+0000 7f8acce00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f8ac80ca810 0x7f8ac815dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:46.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.173+0000 7f8acce00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ac80cac20 con 0x7f8ac8074a60 2024-05-27T09:29:46.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.173+0000 7f8acce00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f8ac8154c30 con 0x7f8ac80ca810 2024-05-27T09:29:46.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.173+0000 7f8acce00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f8ac815db60 con 0x7f8ac8154820 2024-05-27T09:29:46.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.173+0000 7f8ac5e00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f8ac8074a60 0x7f8ac8073260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:51800/0 (socket says 172.21.15.114:51800) 2024-05-27T09:29:46.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.173+0000 7f8ac5e00640 1 -- 172.21.15.114:0/2620371013 learned_addr learned my addr 172.21.15.114:0/2620371013 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:29:46.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.173+0000 7f8ac6800640 1 --2- 172.21.15.114:0/2620371013 >> v2:172.21.15.114:3300/0 conn(0x7f8ac80ca810 0x7f8ac815dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:46.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.174+0000 7f8ac6800640 1 -- 172.21.15.114:0/2620371013 >> v2:172.21.15.163:3300/0 conn(0x7f8ac8154820 msgr2=0x7f8ac80ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:29:46.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.174+0000 7f8ac6800640 1 --2- 172.21.15.114:0/2620371013 >> v2:172.21.15.163:3300/0 conn(0x7f8ac8154820 0x7f8ac80ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:46.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.174+0000 7f8ac6800640 1 -- 172.21.15.114:0/2620371013 >> v1:172.21.15.114:6789/0 conn(0x7f8ac8074a60 legacy=0x7f8ac8073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:46.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.174+0000 7f8ac6800640 1 -- 172.21.15.114:0/2620371013 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac815a810 con 0x7f8ac80ca810 2024-05-27T09:29:46.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.175+0000 7f8ac6800640 1 --2- 172.21.15.114:0/2620371013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ac80ca810 0x7f8ac815dff0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8ac806d7b0 tx=0x7f8ac002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7aa873e9abd343 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:46.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.175+0000 7f8ac5400640 1 -- 172.21.15.114:0/2620371013 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0007740 con 0x7f8ac80ca810 2024-05-27T09:29:46.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.175+0000 7f8ac5400640 1 -- 172.21.15.114:0/2620371013 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ac00078e0 con 0x7f8ac80ca810 2024-05-27T09:29:46.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.175+0000 7f8ac5400640 1 -- 172.21.15.114:0/2620371013 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0007cb0 con 0x7f8ac80ca810 2024-05-27T09:29:46.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.175+0000 7f8acce00640 1 -- 172.21.15.114:0/2620371013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ac80ca810 msgr2=0x7f8ac815dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:46.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.175+0000 7f8acce00640 1 --2- 172.21.15.114:0/2620371013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ac80ca810 0x7f8ac815dff0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8ac806d7b0 tx=0x7f8ac002a730 comp rx=0 tx=0).stop 2024-05-27T09:29:46.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.176+0000 7f8acce00640 1 -- 172.21.15.114:0/2620371013 shutdown_connections 2024-05-27T09:29:46.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.176+0000 7f8acce00640 1 --2- 172.21.15.114:0/2620371013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ac80ca810 0x7f8ac815dff0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:46.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.176+0000 7f8acce00640 1 --2- 172.21.15.114:0/2620371013 >> v2:172.21.15.163:3300/0 conn(0x7f8ac8154820 0x7f8ac80ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:46.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.176+0000 7f8acce00640 1 -- 172.21.15.114:0/2620371013 >> 172.21.15.114:0/2620371013 conn(0x7f8ac806f6e0 msgr2=0x7f8ac8070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:46.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.176+0000 7f8acce00640 1 -- 172.21.15.114:0/2620371013 shutdown_connections 2024-05-27T09:29:46.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.176+0000 7f8acce00640 1 -- 172.21.15.114:0/2620371013 wait complete. 2024-05-27T09:29:46.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.178+0000 7f8acce00640 1 Processor -- start 2024-05-27T09:29:46.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.178+0000 7f8acce00640 1 -- start start 2024-05-27T09:29:46.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.179+0000 7f8acce00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ac8074a60 0x7f8ac815d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:46.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.179+0000 7f8acce00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ac80ca810 0x7f8ac80c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:46.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.179+0000 7f8acce00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ac8154820 0x7f8ac80c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:46.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.179+0000 7f8acce00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac815b0b0 con 0x7f8ac8154820 2024-05-27T09:29:46.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.179+0000 7f8acce00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac815add0 con 0x7f8ac80ca810 2024-05-27T09:29:46.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.179+0000 7f8acce00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f8ac815b390 con 0x7f8ac8074a60 2024-05-27T09:29:46.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.180+0000 7f8ac6800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ac8154820 0x7f8ac80c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:46.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.180+0000 7f8ac5e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ac8074a60 0x7f8ac815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:46.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.180+0000 7f8abfe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ac80ca810 0x7f8ac80c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:46.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.180+0000 7f8ac5e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ac8074a60 0x7f8ac815d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:48926/0 (socket says 172.21.15.114:48926) 2024-05-27T09:29:46.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.180+0000 7f8ac5e00640 1 -- 172.21.15.114:0/460228871 learned_addr learned my addr 172.21.15.114:0/460228871 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:29:46.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.180+0000 7f8ac6800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ac8154820 0x7f8ac80c3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49426/0 (socket says 172.21.15.114:49426) 2024-05-27T09:29:46.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.180+0000 7f8ac6800640 1 -- 172.21.15.114:0/460228871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ac8074a60 msgr2=0x7f8ac815d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:46.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.180+0000 7f8ac6800640 1 --2- 172.21.15.114:0/460228871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ac8074a60 0x7f8ac815d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:46.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.180+0000 7f8ac6800640 1 -- 172.21.15.114:0/460228871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ac80ca810 msgr2=0x7f8ac80c39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:46.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.180+0000 7f8ac6800640 1 --2- 172.21.15.114:0/460228871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ac80ca810 0x7f8ac80c39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:46.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.180+0000 7f8ac6800640 1 -- 172.21.15.114:0/460228871 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac81f2de0 con 0x7f8ac8154820 2024-05-27T09:29:46.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.180+0000 7f8ac6800640 1 --2- 172.21.15.114:0/460228871 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ac8154820 0x7f8ac80c3f20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8ac806d830 tx=0x7f8ac0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:46.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.181+0000 7f8abe000640 1 -- 172.21.15.114:0/460228871 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0040070 con 0x7f8ac8154820 2024-05-27T09:29:46.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.181+0000 7f8abe000640 1 -- 172.21.15.114:0/460228871 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ac0031bc0 con 0x7f8ac8154820 2024-05-27T09:29:46.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.181+0000 7f8abe000640 1 -- 172.21.15.114:0/460228871 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0045460 con 0x7f8ac8154820 2024-05-27T09:29:46.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.181+0000 7f8acce00640 1 -- 172.21.15.114:0/460228871 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac8158b20 con 0x7f8ac8154820 2024-05-27T09:29:46.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.181+0000 7f8acce00640 1 -- 172.21.15.114:0/460228871 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac81f3380 con 0x7f8ac8154820 2024-05-27T09:29:46.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.182+0000 7f8acce00640 1 -- 172.21.15.114:0/460228871 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ac8158870 con 0x7f8ac8154820 2024-05-27T09:29:46.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.183+0000 7f8abe000640 1 -- 172.21.15.114:0/460228871 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8ac0039040 con 0x7f8ac8154820 2024-05-27T09:29:46.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.184+0000 7f8abe000640 1 --2- 172.21.15.114:0/460228871 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f8aa4076300 0x7f8aa4078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:46.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.184+0000 7f8abe000640 1 -- 172.21.15.114:0/460228871 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5824+0+0 (secure 0 0 0) 0x7f8ac00b8a50 con 0x7f8ac8154820 2024-05-27T09:29:46.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.185+0000 7f8ac5e00640 1 --2- 172.21.15.114:0/460228871 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f8aa4076300 0x7f8aa4078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:46.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.186+0000 7f8ac5e00640 1 --2- 172.21.15.114:0/460228871 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f8aa4076300 0x7f8aa4078810 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8000f60 tx=0x7f8ab8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:46.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.189+0000 7f8abe000640 1 -- 172.21.15.114:0/460228871 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8ac004c030 con 0x7f8ac8154820 2024-05-27T09:29:46.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.439+0000 7f8acce00640 1 -- 172.21.15.114:0/460228871 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ac806a380 con 0x7f8ac8154820 2024-05-27T09:29:46.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.440+0000 7f8abe000640 1 -- 172.21.15.114:0/460228871 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6049 (secure 0 0 0) 0x7f8ac0081d30 con 0x7f8ac8154820 2024-05-27T09:29:46.441 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:29:46.441 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:29:45.324170+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"a","rank":2,"incarnation":13,"state":"up:replay","state_seq":1,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4297,4300]},"id":1}]} 2024-05-27T09:29:46.442 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 13 2024-05-27T09:29:46.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.442+0000 7f8acce00640 1 -- 172.21.15.114:0/460228871 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f8aa4076300 msgr2=0x7f8aa4078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:46.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.442+0000 7f8acce00640 1 --2- 172.21.15.114:0/460228871 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f8aa4076300 0x7f8aa4078810 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8000f60 tx=0x7f8ab8009040 comp rx=0 tx=0).stop 2024-05-27T09:29:46.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.442+0000 7f8acce00640 1 -- 172.21.15.114:0/460228871 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ac8154820 msgr2=0x7f8ac80c3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:46.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.442+0000 7f8acce00640 1 --2- 172.21.15.114:0/460228871 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ac8154820 0x7f8ac80c3f20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8ac806d830 tx=0x7f8ac0004a40 comp rx=0 tx=0).stop 2024-05-27T09:29:46.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.442+0000 7f8acce00640 1 -- 172.21.15.114:0/460228871 shutdown_connections 2024-05-27T09:29:46.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.442+0000 7f8acce00640 1 --2- 172.21.15.114:0/460228871 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f8aa4076300 0x7f8aa4078810 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:46.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.442+0000 7f8acce00640 1 --2- 172.21.15.114:0/460228871 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8ac8154820 0x7f8ac80c3f20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:46.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.442+0000 7f8acce00640 1 --2- 172.21.15.114:0/460228871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ac80ca810 0x7f8ac80c39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:46.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.442+0000 7f8acce00640 1 --2- 172.21.15.114:0/460228871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ac8074a60 0x7f8ac815d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:46.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.442+0000 7f8acce00640 1 -- 172.21.15.114:0/460228871 >> 172.21.15.114:0/460228871 conn(0x7f8ac806f6e0 msgr2=0x7f8ac80cddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:46.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.443+0000 7f8acce00640 1 -- 172.21.15.114:0/460228871 shutdown_connections 2024-05-27T09:29:46.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.443+0000 7f8acce00640 1 -- 172.21.15.114:0/460228871 wait complete. 2024-05-27T09:29:46.468 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2024-05-27T09:29:46.468 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-05-27T09:29:46.468 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:29:46.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.720+0000 7f252d200640 1 Processor -- start 2024-05-27T09:29:46.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.721+0000 7f252d200640 1 -- start start 2024-05-27T09:29:46.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.723+0000 7f252d200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f252806d120 0x7f2528158930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:46.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.723+0000 7f252d200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f25280c5870 0x7f25280cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:46.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.723+0000 7f252d200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25280c5c80 con 0x7f25280c7d20 2024-05-27T09:29:46.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.723+0000 7f252d200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f25280c8840 con 0x7f25280c5870 2024-05-27T09:29:46.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.723+0000 7f252d200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f25280c9e50 con 0x7f252806d120 2024-05-27T09:29:46.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.723+0000 7f2527e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f25280c5870 0x7f25280cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:46.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.723+0000 7f2526a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f252806d120 0x7f2528158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:46.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.723+0000 7f2527e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f25280c5870 0x7f25280cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49442/0 (socket says 172.21.15.114:49442) 2024-05-27T09:29:46.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.723+0000 7f2527e00640 1 -- 172.21.15.114:0/1494134996 learned_addr learned my addr 172.21.15.114:0/1494134996 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:29:46.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.724+0000 7f2527e00640 1 -- 172.21.15.114:0/1494134996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f252806d120 msgr2=0x7f2528158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:29:46.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.724+0000 7f2527e00640 1 --2- 172.21.15.114:0/1494134996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f252806d120 0x7f2528158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:46.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.724+0000 7f2527e00640 1 -- 172.21.15.114:0/1494134996 >> v1:172.21.15.163:6789/0 conn(0x7f25280c7d20 legacy=0x7f2528152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:46.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.724+0000 7f2527e00640 1 -- 172.21.15.114:0/1494134996 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25280c8a90 con 0x7f25280c5870 2024-05-27T09:29:46.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.724+0000 7f2526000640 1 -- 172.21.15.114:0/1494134996 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3917387995 0 0) 0x7f25280c5c80 con 0x7f25280c7d20 2024-05-27T09:29:46.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.725+0000 7f2527e00640 1 --2- 172.21.15.114:0/1494134996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25280c5870 0x7f25280cd9e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2520009e50 tx=0x7f252002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=786911ba12731185 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:46.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.725+0000 7f2526000640 1 -- 172.21.15.114:0/1494134996 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2520036070 con 0x7f25280c5870 2024-05-27T09:29:46.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.725+0000 7f2526000640 1 -- 172.21.15.114:0/1494134996 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2520031600 con 0x7f25280c5870 2024-05-27T09:29:46.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.725+0000 7f2526000640 1 -- 172.21.15.114:0/1494134996 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25200319d0 con 0x7f25280c5870 2024-05-27T09:29:46.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.726+0000 7f252d200640 1 -- 172.21.15.114:0/1494134996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25280c5870 msgr2=0x7f25280cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:46.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.726+0000 7f252d200640 1 --2- 172.21.15.114:0/1494134996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25280c5870 0x7f25280cd9e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2520009e50 tx=0x7f252002a710 comp rx=0 tx=0).stop 2024-05-27T09:29:46.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.726+0000 7f252d200640 1 -- 172.21.15.114:0/1494134996 shutdown_connections 2024-05-27T09:29:46.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.726+0000 7f252d200640 1 --2- 172.21.15.114:0/1494134996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25280c5870 0x7f25280cd9e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:46.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.726+0000 7f252d200640 1 --2- 172.21.15.114:0/1494134996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f252806d120 0x7f2528158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:46.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.726+0000 7f252d200640 1 -- 172.21.15.114:0/1494134996 >> 172.21.15.114:0/1494134996 conn(0x7f252806f9c0 msgr2=0x7f25280734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:46.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.726+0000 7f252d200640 1 -- 172.21.15.114:0/1494134996 shutdown_connections 2024-05-27T09:29:46.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.726+0000 7f252d200640 1 -- 172.21.15.114:0/1494134996 wait complete. 2024-05-27T09:29:46.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.728+0000 7f252d200640 1 Processor -- start 2024-05-27T09:29:46.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.728+0000 7f252d200640 1 -- start start 2024-05-27T09:29:46.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.729+0000 7f252d200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f252806d120 0x7f252815c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:46.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.730+0000 7f252d200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25280c5870 0x7f2528154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:46.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.730+0000 7f252d200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25280c7d20 0x7f2528154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:46.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.730+0000 7f252d200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f25280c5350 con 0x7f25280c5870 2024-05-27T09:29:46.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.730+0000 7f252d200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f25280c5070 con 0x7f252806d120 2024-05-27T09:29:46.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.730+0000 7f252d200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f25280c6bd0 con 0x7f25280c7d20 2024-05-27T09:29:46.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.730+0000 7f2526a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25280c5870 0x7f2528154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:46.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.730+0000 7f2526a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25280c5870 0x7f2528154f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49458/0 (socket says 172.21.15.114:49458) 2024-05-27T09:29:46.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.730+0000 7f2527400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f252806d120 0x7f252815c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:46.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.730+0000 7f2526a00640 1 -- 172.21.15.114:0/318372934 learned_addr learned my addr 172.21.15.114:0/318372934 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:29:46.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.730+0000 7f2527e00640 1 --2- 172.21.15.114:0/318372934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25280c7d20 0x7f2528154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:46.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.730+0000 7f2526a00640 1 -- 172.21.15.114:0/318372934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25280c7d20 msgr2=0x7f2528154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:46.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.730+0000 7f2526a00640 1 --2- 172.21.15.114:0/318372934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25280c7d20 0x7f2528154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:46.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.730+0000 7f2526a00640 1 -- 172.21.15.114:0/318372934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f252806d120 msgr2=0x7f252815c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:46.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.730+0000 7f2526a00640 1 --2- 172.21.15.114:0/318372934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f252806d120 0x7f252815c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:46.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.730+0000 7f2526a00640 1 -- 172.21.15.114:0/318372934 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252815c820 con 0x7f25280c5870 2024-05-27T09:29:46.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.730+0000 7f2527400640 1 --2- 172.21.15.114:0/318372934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f252806d120 0x7f252815c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:29:46.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.731+0000 7f2526a00640 1 --2- 172.21.15.114:0/318372934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25280c5870 0x7f2528154f90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f251000e8a0 tx=0x7f251000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:46.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.731+0000 7f251f400640 1 -- 172.21.15.114:0/318372934 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510017070 con 0x7f25280c5870 2024-05-27T09:29:46.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.731+0000 7f251f400640 1 -- 172.21.15.114:0/318372934 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f251000c940 con 0x7f25280c5870 2024-05-27T09:29:46.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.731+0000 7f251f400640 1 -- 172.21.15.114:0/318372934 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251000cd30 con 0x7f25280c5870 2024-05-27T09:29:46.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.731+0000 7f252d200640 1 -- 172.21.15.114:0/318372934 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25281533e0 con 0x7f25280c5870 2024-05-27T09:29:46.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.731+0000 7f252d200640 1 -- 172.21.15.114:0/318372934 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25281562f0 con 0x7f25280c5870 2024-05-27T09:29:46.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.732+0000 7f252d200640 1 -- 172.21.15.114:0/318372934 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2528050860 con 0x7f25280c5870 2024-05-27T09:29:46.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.733+0000 7f251f400640 1 -- 172.21.15.114:0/318372934 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f251001a070 con 0x7f25280c5870 2024-05-27T09:29:46.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.734+0000 7f251f400640 1 --2- 172.21.15.114:0/318372934 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f25040762d0 0x7f25040787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:46.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.734+0000 7f251f400640 1 -- 172.21.15.114:0/318372934 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5824+0+0 (secure 0 0 0) 0x7f2510097520 con 0x7f25280c5870 2024-05-27T09:29:46.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.735+0000 7f2527400640 1 --2- 172.21.15.114:0/318372934 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f25040762d0 0x7f25040787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:46.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.735+0000 7f2527400640 1 --2- 172.21.15.114:0/318372934 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f25040762d0 0x7f25040787e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2518000f60 tx=0x7f2518009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:46.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:46.739+0000 7f251f400640 1 -- 172.21.15.114:0/318372934 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2510060740 con 0x7f25280c5870 2024-05-27T09:29:47.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:47.023+0000 7f252d200640 1 -- 172.21.15.114:0/318372934 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2528147ae0 con 0x7f25280c5870 2024-05-27T09:29:47.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:47.025+0000 7f251f400640 1 -- 172.21.15.114:0/318372934 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6049 (secure 0 0 0) 0x7f2510064a40 con 0x7f25280c5870 2024-05-27T09:29:47.026 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:29:47.026 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:29:45.324170+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"a","rank":2,"incarnation":13,"state":"up:replay","state_seq":1,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4297,4300]},"id":1}]} 2024-05-27T09:29:47.026 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 13 2024-05-27T09:29:47.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:47.027+0000 7f252d200640 1 -- 172.21.15.114:0/318372934 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f25040762d0 msgr2=0x7f25040787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:47.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:47.027+0000 7f252d200640 1 --2- 172.21.15.114:0/318372934 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f25040762d0 0x7f25040787e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2518000f60 tx=0x7f2518009040 comp rx=0 tx=0).stop 2024-05-27T09:29:47.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:47.027+0000 7f252d200640 1 -- 172.21.15.114:0/318372934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25280c5870 msgr2=0x7f2528154f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:47.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:47.027+0000 7f252d200640 1 --2- 172.21.15.114:0/318372934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25280c5870 0x7f2528154f90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f251000e8a0 tx=0x7f251000ed60 comp rx=0 tx=0).stop 2024-05-27T09:29:47.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:47.027+0000 7f252d200640 1 -- 172.21.15.114:0/318372934 shutdown_connections 2024-05-27T09:29:47.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:47.027+0000 7f252d200640 1 --2- 172.21.15.114:0/318372934 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f25040762d0 0x7f25040787e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:47.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:47.027+0000 7f252d200640 1 --2- 172.21.15.114:0/318372934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25280c7d20 0x7f2528154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:47.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:47.027+0000 7f252d200640 1 --2- 172.21.15.114:0/318372934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25280c5870 0x7f2528154f90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:47.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:47.027+0000 7f252d200640 1 --2- 172.21.15.114:0/318372934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f252806d120 0x7f252815c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:47.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:47.027+0000 7f252d200640 1 -- 172.21.15.114:0/318372934 >> 172.21.15.114:0/318372934 conn(0x7f252806f9c0 msgr2=0x7f252814c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:47.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:47.027+0000 7f252d200640 1 -- 172.21.15.114:0/318372934 shutdown_connections 2024-05-27T09:29:47.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:47.027+0000 7f252d200640 1 -- 172.21.15.114:0/318372934 wait complete. 2024-05-27T09:29:47.056 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 13, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-05-27T09:27:18.079214+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 13, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4256": { "addr": "172.21.15.114:6835/1325619883", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6833", "nonce": 1325619883, "type": "v2" }, { "addr": "172.21.15.114:6835", "nonce": 1325619883, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4256, "incarnation": 13, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:replay", "state_seq": 1 }, "gid_4297": { "addr": "172.21.15.114:6834/657027519", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6832", "nonce": 657027519, "type": "v2" }, { "addr": "172.21.15.114:6834", "nonce": 657027519, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4297, "incarnation": 9, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4300": { "addr": "172.21.15.114:6837/1448950017", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6836", "nonce": 1448950017, "type": "v2" }, { "addr": "172.21.15.114:6837", "nonce": 1448950017, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4300, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 26, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-05-27T09:29:45.324170+0000", "qdb_cluster": [ 4297, 4300 ], "qdb_leader": 4300, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4300, "mds_1": 4297, "mds_2": 4256 } } } ], "standbys": [ { "addr": "172.21.15.163:6835/3513522318", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 3513522318, "type": "v2" }, { "addr": "172.21.15.163:6835", "nonce": 3513522318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6838/1753596112", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6836", "nonce": 1753596112, "type": "v2" }, { "addr": "172.21.15.163:6838", "nonce": 1753596112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-05-27T09:29:48.052 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:29:48.059 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:29:49.056 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:29:49.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.340+0000 7f0d14a00640 1 Processor -- start 2024-05-27T09:29:49.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.340+0000 7f0d14a00640 1 -- start start 2024-05-27T09:29:49.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.343+0000 7f0d14a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0d10142720 0x7f0d10073260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:49.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.343+0000 7f0d14a00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d100739a0 con 0x7f0d101593b0 2024-05-27T09:29:49.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.343+0000 7f0d14a00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d10073c80 con 0x7f0d100c2990 2024-05-27T09:29:49.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.343+0000 7f0d14a00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f0d10074d20 con 0x7f0d10142720 2024-05-27T09:29:49.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.343+0000 7f0d05400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0d10142720 0x7f0d10073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:49.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.343+0000 7f0d0de00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f0d101593b0 0x7f0d1015a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:51804/0 (socket says 172.21.15.114:51804) 2024-05-27T09:29:49.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.343+0000 7f0d0de00640 1 -- 172.21.15.114:0/3578196301 learned_addr learned my addr 172.21.15.114:0/3578196301 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:29:49.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.344+0000 7f0d05400640 1 -- 172.21.15.114:0/3578196301 >> v1:172.21.15.163:6789/0 conn(0x7f0d100c2990 legacy=0x7f0d10163d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:49.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.344+0000 7f0d05400640 1 -- 172.21.15.114:0/3578196301 >> v1:172.21.15.114:6789/0 conn(0x7f0d101593b0 legacy=0x7f0d1015a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:49.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.344+0000 7f0d05400640 1 -- 172.21.15.114:0/3578196301 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d10073f60 con 0x7f0d10142720 2024-05-27T09:29:49.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.344+0000 7f0d0d400640 1 -- 172.21.15.114:0/3578196301 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3432567474 0 0) 0x7f0d100739a0 con 0x7f0d101593b0 2024-05-27T09:29:49.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.345+0000 7f0d05400640 1 --2- 172.21.15.114:0/3578196301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d10142720 0x7f0d10073260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800ba40 tx=0x7f0cf802c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d07f2e2d597b43b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:49.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.345+0000 7f0d0d400640 1 -- 172.21.15.114:0/3578196301 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8036020 con 0x7f0d10142720 2024-05-27T09:29:49.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.345+0000 7f0d0d400640 1 -- 172.21.15.114:0/3578196301 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf8032870 con 0x7f0d10142720 2024-05-27T09:29:49.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.345+0000 7f0d0d400640 1 -- 172.21.15.114:0/3578196301 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8032c40 con 0x7f0d10142720 2024-05-27T09:29:49.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.346+0000 7f0d14a00640 1 -- 172.21.15.114:0/3578196301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d10142720 msgr2=0x7f0d10073260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:49.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.346+0000 7f0d14a00640 1 --2- 172.21.15.114:0/3578196301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d10142720 0x7f0d10073260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800ba40 tx=0x7f0cf802c970 comp rx=0 tx=0).stop 2024-05-27T09:29:49.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.346+0000 7f0d14a00640 1 -- 172.21.15.114:0/3578196301 shutdown_connections 2024-05-27T09:29:49.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.346+0000 7f0d14a00640 1 --2- 172.21.15.114:0/3578196301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d10142720 0x7f0d10073260 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:49.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.346+0000 7f0d14a00640 1 -- 172.21.15.114:0/3578196301 >> 172.21.15.114:0/3578196301 conn(0x7f0d1006f6e0 msgr2=0x7f0d10070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:49.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.346+0000 7f0d14a00640 1 -- 172.21.15.114:0/3578196301 shutdown_connections 2024-05-27T09:29:49.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.346+0000 7f0d14a00640 1 -- 172.21.15.114:0/3578196301 wait complete. 2024-05-27T09:29:49.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.348+0000 7f0d14a00640 1 Processor -- start 2024-05-27T09:29:49.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.348+0000 7f0d14a00640 1 -- start start 2024-05-27T09:29:49.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.350+0000 7f0d14a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d100c2990 0x7f0d10140ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:49.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.350+0000 7f0d14a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d10142720 0x7f0d1006c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:49.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.350+0000 7f0d14a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d101593b0 0x7f0d1006cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:49.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.350+0000 7f0d14a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0d10074520 con 0x7f0d101593b0 2024-05-27T09:29:49.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.350+0000 7f0d14a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0d10142b30 con 0x7f0d10142720 2024-05-27T09:29:49.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.350+0000 7f0d14a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f0d10074800 con 0x7f0d100c2990 2024-05-27T09:29:49.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.350+0000 7f0d0e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d101593b0 0x7f0d1006cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:49.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.350+0000 7f0d0e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d101593b0 0x7f0d1006cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49474/0 (socket says 172.21.15.114:49474) 2024-05-27T09:29:49.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.350+0000 7f0d0e800640 1 -- 172.21.15.114:0/3446590650 learned_addr learned my addr 172.21.15.114:0/3446590650 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:29:49.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.350+0000 7f0d05400640 1 --2- 172.21.15.114:0/3446590650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d10142720 0x7f0d1006c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:49.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.350+0000 7f0d0de00640 1 --2- 172.21.15.114:0/3446590650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d100c2990 0x7f0d10140ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:49.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.350+0000 7f0d0e800640 1 -- 172.21.15.114:0/3446590650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d100c2990 msgr2=0x7f0d10140ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:49.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.350+0000 7f0d0e800640 1 --2- 172.21.15.114:0/3446590650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d100c2990 0x7f0d10140ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:49.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.350+0000 7f0d0e800640 1 -- 172.21.15.114:0/3446590650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d10142720 msgr2=0x7f0d1006c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:49.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.350+0000 7f0d0e800640 1 --2- 172.21.15.114:0/3446590650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d10142720 0x7f0d1006c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:49.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.350+0000 7f0d0e800640 1 -- 172.21.15.114:0/3446590650 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1015a570 con 0x7f0d101593b0 2024-05-27T09:29:49.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.351+0000 7f0d0e800640 1 --2- 172.21.15.114:0/3446590650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d101593b0 0x7f0d1006cde0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800c3f0 tx=0x7f0d0800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:49.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.351+0000 7f0d06a00640 1 -- 172.21.15.114:0/3446590650 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d08017070 con 0x7f0d101593b0 2024-05-27T09:29:49.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.351+0000 7f0d06a00640 1 -- 172.21.15.114:0/3446590650 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d080052d0 con 0x7f0d101593b0 2024-05-27T09:29:49.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.351+0000 7f0d06a00640 1 -- 172.21.15.114:0/3446590650 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d08003810 con 0x7f0d101593b0 2024-05-27T09:29:49.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.351+0000 7f0d14a00640 1 -- 172.21.15.114:0/3446590650 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d1013c150 con 0x7f0d101593b0 2024-05-27T09:29:49.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.351+0000 7f0d14a00640 1 -- 172.21.15.114:0/3446590650 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d1015b7f0 con 0x7f0d101593b0 2024-05-27T09:29:49.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.352+0000 7f0d14a00640 1 -- 172.21.15.114:0/3446590650 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d1006a380 con 0x7f0d101593b0 2024-05-27T09:29:49.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.354+0000 7f0d06a00640 1 -- 172.21.15.114:0/3446590650 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0d0801a070 con 0x7f0d101593b0 2024-05-27T09:29:49.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.355+0000 7f0d06a00640 1 --2- 172.21.15.114:0/3446590650 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0cec0762d0 0x7f0cec0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:49.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.355+0000 7f0d06a00640 1 -- 172.21.15.114:0/3446590650 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5824+0+0 (secure 0 0 0) 0x7f0d080969b0 con 0x7f0d101593b0 2024-05-27T09:29:49.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.355+0000 7f0d0de00640 1 --2- 172.21.15.114:0/3446590650 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0cec0762d0 0x7f0cec0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:49.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.356+0000 7f0d0de00640 1 --2- 172.21.15.114:0/3446590650 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0cec0762d0 0x7f0cec0787e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0d000012e0 tx=0x7f0d00004d40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:49.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.360+0000 7f0d06a00640 1 -- 172.21.15.114:0/3446590650 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0d0805fc80 con 0x7f0d101593b0 2024-05-27T09:29:49.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.606+0000 7f0d14a00640 1 -- 172.21.15.114:0/3446590650 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d100c41f0 con 0x7f0d101593b0 2024-05-27T09:29:49.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.607+0000 7f0d06a00640 1 -- 172.21.15.114:0/3446590650 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+6049 (secure 0 0 0) 0x7f0d08068350 con 0x7f0d101593b0 2024-05-27T09:29:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:29:49.609 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:29:45.324170+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"a","rank":2,"incarnation":13,"state":"up:replay","state_seq":1,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4297,4300]},"id":1}]} 2024-05-27T09:29:49.609 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 13 2024-05-27T09:29:49.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.610+0000 7f0d14a00640 1 -- 172.21.15.114:0/3446590650 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0cec0762d0 msgr2=0x7f0cec0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:49.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.610+0000 7f0d14a00640 1 --2- 172.21.15.114:0/3446590650 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0cec0762d0 0x7f0cec0787e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0d000012e0 tx=0x7f0d00004d40 comp rx=0 tx=0).stop 2024-05-27T09:29:49.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.610+0000 7f0d14a00640 1 -- 172.21.15.114:0/3446590650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d101593b0 msgr2=0x7f0d1006cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:49.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.610+0000 7f0d14a00640 1 --2- 172.21.15.114:0/3446590650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d101593b0 0x7f0d1006cde0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800c3f0 tx=0x7f0d0800c8b0 comp rx=0 tx=0).stop 2024-05-27T09:29:49.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.610+0000 7f0d14a00640 1 -- 172.21.15.114:0/3446590650 shutdown_connections 2024-05-27T09:29:49.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.610+0000 7f0d14a00640 1 --2- 172.21.15.114:0/3446590650 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0cec0762d0 0x7f0cec0787e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:49.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.610+0000 7f0d14a00640 1 --2- 172.21.15.114:0/3446590650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d101593b0 0x7f0d1006cde0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:49.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.610+0000 7f0d14a00640 1 --2- 172.21.15.114:0/3446590650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d10142720 0x7f0d1006c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:49.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.610+0000 7f0d14a00640 1 --2- 172.21.15.114:0/3446590650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d100c2990 0x7f0d10140ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:49.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.610+0000 7f0d14a00640 1 -- 172.21.15.114:0/3446590650 >> 172.21.15.114:0/3446590650 conn(0x7f0d1006f6e0 msgr2=0x7f0d100cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:49.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.610+0000 7f0d14a00640 1 -- 172.21.15.114:0/3446590650 shutdown_connections 2024-05-27T09:29:49.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:49.611+0000 7f0d14a00640 1 -- 172.21.15.114:0/3446590650 wait complete. 2024-05-27T09:29:49.641 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:29:51.641 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:29:51.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.923+0000 7f3fb1200640 1 Processor -- start 2024-05-27T09:29:51.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.923+0000 7f3fb1200640 1 -- start start 2024-05-27T09:29:51.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.925+0000 7f3fb1200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3fac0cda10 0x7f3fac160760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:51.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.925+0000 7f3fb1200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fac15d130 0x7f3fac08d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:51.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.925+0000 7f3fb1200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fac0cca00 con 0x7f3fac08dea0 2024-05-27T09:29:51.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.925+0000 7f3fb1200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f3fac1602d0 con 0x7f3fac0cda10 2024-05-27T09:29:51.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.925+0000 7f3fb1200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f3fac0ccce0 con 0x7f3fac15d130 2024-05-27T09:29:51.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.925+0000 7f3faa800640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f3fac08dea0 0x7f3fac0c7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:51808/0 (socket says 172.21.15.114:51808) 2024-05-27T09:29:51.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.925+0000 7f3faa800640 1 -- 172.21.15.114:0/17467652 learned_addr learned my addr 172.21.15.114:0/17467652 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:29:51.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.925+0000 7f3fa9400640 1 --2- 172.21.15.114:0/17467652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fac15d130 0x7f3fac08d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:51.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.925+0000 7f3fa9e00640 1 --2- 172.21.15.114:0/17467652 >> v2:172.21.15.163:3300/0 conn(0x7f3fac0cda10 0x7f3fac160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:51.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.926+0000 7f3fa9400640 1 -- 172.21.15.114:0/17467652 >> v2:172.21.15.163:3300/0 conn(0x7f3fac0cda10 msgr2=0x7f3fac160760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:29:51.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.926+0000 7f3fa9400640 1 --2- 172.21.15.114:0/17467652 >> v2:172.21.15.163:3300/0 conn(0x7f3fac0cda10 0x7f3fac160760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:51.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.926+0000 7f3fa9400640 1 -- 172.21.15.114:0/17467652 >> v1:172.21.15.114:6789/0 conn(0x7f3fac08dea0 legacy=0x7f3fac0c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:51.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.926+0000 7f3fa8a00640 1 -- 172.21.15.114:0/17467652 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 11525830 0 0) 0x7f3fac0cca00 con 0x7f3fac08dea0 2024-05-27T09:29:51.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.926+0000 7f3fa9400640 1 -- 172.21.15.114:0/17467652 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fac0ccfc0 con 0x7f3fac15d130 2024-05-27T09:29:51.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.926+0000 7f3fa9e00640 1 --2- 172.21.15.114:0/17467652 >> v2:172.21.15.163:3300/0 conn(0x7f3fac0cda10 0x7f3fac160760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:29:51.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.927+0000 7f3fa9400640 1 --2- 172.21.15.114:0/17467652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fac15d130 0x7f3fac08d960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3f940097a0 tx=0x7f3f9402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8575d88d61e31432 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:51.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.927+0000 7f3fa8a00640 1 -- 172.21.15.114:0/17467652 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94037070 con 0x7f3fac15d130 2024-05-27T09:29:51.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.927+0000 7f3fa8a00640 1 -- 172.21.15.114:0/17467652 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f94032440 con 0x7f3fac15d130 2024-05-27T09:29:51.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.927+0000 7f3fa8a00640 1 -- 172.21.15.114:0/17467652 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94032810 con 0x7f3fac15d130 2024-05-27T09:29:51.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.928+0000 7f3fb1200640 1 -- 172.21.15.114:0/17467652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fac15d130 msgr2=0x7f3fac08d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:51.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.928+0000 7f3fb1200640 1 --2- 172.21.15.114:0/17467652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fac15d130 0x7f3fac08d960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3f940097a0 tx=0x7f3f9402a210 comp rx=0 tx=0).stop 2024-05-27T09:29:51.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.928+0000 7f3fb1200640 1 -- 172.21.15.114:0/17467652 shutdown_connections 2024-05-27T09:29:51.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.928+0000 7f3fb1200640 1 --2- 172.21.15.114:0/17467652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fac15d130 0x7f3fac08d960 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:51.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.928+0000 7f3fb1200640 1 --2- 172.21.15.114:0/17467652 >> v2:172.21.15.163:3300/0 conn(0x7f3fac0cda10 0x7f3fac160760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:51.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.928+0000 7f3fb1200640 1 -- 172.21.15.114:0/17467652 >> 172.21.15.114:0/17467652 conn(0x7f3fac06f2e0 msgr2=0x7f3fac068600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:51.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.928+0000 7f3fb1200640 1 -- 172.21.15.114:0/17467652 shutdown_connections 2024-05-27T09:29:51.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.928+0000 7f3fb1200640 1 -- 172.21.15.114:0/17467652 wait complete. 2024-05-27T09:29:51.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.930+0000 7f3fb1200640 1 Processor -- start 2024-05-27T09:29:51.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.931+0000 7f3fb1200640 1 -- start start 2024-05-27T09:29:51.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.932+0000 7f3fb1200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fac08dea0 0x7f3fac1f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:51.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.932+0000 7f3fb1200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fac0cda10 0x7f3fac0b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:51.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.932+0000 7f3fb1200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fac15d130 0x7f3fac151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:51.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.932+0000 7f3fb1200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac15d540 con 0x7f3fac15d130 2024-05-27T09:29:51.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.932+0000 7f3fb1200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac15ff60 con 0x7f3fac0cda10 2024-05-27T09:29:51.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.932+0000 7f3fb1200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f3fac1f6f20 con 0x7f3fac08dea0 2024-05-27T09:29:51.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.932+0000 7f3faa800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fac15d130 0x7f3fac151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:51.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.932+0000 7f3fa9400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fac0cda10 0x7f3fac0b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:51.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.932+0000 7f3fa9e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fac08dea0 0x7f3fac1f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:51.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.932+0000 7f3faa800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fac15d130 0x7f3fac151420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49480/0 (socket says 172.21.15.114:49480) 2024-05-27T09:29:51.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.932+0000 7f3faa800640 1 -- 172.21.15.114:0/1912160185 learned_addr learned my addr 172.21.15.114:0/1912160185 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:29:51.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.933+0000 7f3fa9400640 1 -- 172.21.15.114:0/1912160185 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fac08dea0 msgr2=0x7f3fac1f3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:51.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.933+0000 7f3fa9400640 1 --2- 172.21.15.114:0/1912160185 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fac08dea0 0x7f3fac1f3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:51.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.933+0000 7f3fa9400640 1 -- 172.21.15.114:0/1912160185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fac15d130 msgr2=0x7f3fac151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:51.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.933+0000 7f3fa9400640 1 --2- 172.21.15.114:0/1912160185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fac15d130 0x7f3fac151420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:51.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.933+0000 7f3fa9400640 1 -- 172.21.15.114:0/1912160185 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fac1f4170 con 0x7f3fac0cda10 2024-05-27T09:29:51.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.933+0000 7f3fa9400640 1 --2- 172.21.15.114:0/1912160185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fac0cda10 0x7f3fac0b5ee0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3f94031000 tx=0x7f3f940046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:51.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.934+0000 7f3fa2000640 1 -- 172.21.15.114:0/1912160185 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94041020 con 0x7f3fac0cda10 2024-05-27T09:29:51.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.934+0000 7f3fa2000640 1 -- 172.21.15.114:0/1912160185 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f94030700 con 0x7f3fac0cda10 2024-05-27T09:29:51.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.934+0000 7f3fa2000640 1 -- 172.21.15.114:0/1912160185 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94037040 con 0x7f3fac0cda10 2024-05-27T09:29:51.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.934+0000 7f3fb1200640 1 -- 172.21.15.114:0/1912160185 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fac1f7170 con 0x7f3fac0cda10 2024-05-27T09:29:51.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.934+0000 7f3fb1200640 1 -- 172.21.15.114:0/1912160185 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fac1f7450 con 0x7f3fac0cda10 2024-05-27T09:29:51.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.935+0000 7f3fb1200640 1 -- 172.21.15.114:0/1912160185 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fac0b7d60 con 0x7f3fac0cda10 2024-05-27T09:29:51.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.936+0000 7f3fa2000640 1 -- 172.21.15.114:0/1912160185 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3f94044020 con 0x7f3fac0cda10 2024-05-27T09:29:51.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.937+0000 7f3fa2000640 1 --2- 172.21.15.114:0/1912160185 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f3f88076300 0x7f3f88078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:51.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.937+0000 7f3fa2000640 1 -- 172.21.15.114:0/1912160185 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5824+0+0 (secure 0 0 0) 0x7f3f940b7d40 con 0x7f3fac0cda10 2024-05-27T09:29:51.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.938+0000 7f3fa9e00640 1 --2- 172.21.15.114:0/1912160185 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f3f88076300 0x7f3f88078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:51.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.938+0000 7f3fa9e00640 1 --2- 172.21.15.114:0/1912160185 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f3f88076300 0x7f3f88078810 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009560 tx=0x7f3f9c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:51.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:51.943+0000 7f3fa2000640 1 -- 172.21.15.114:0/1912160185 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3fac0b7d60 con 0x7f3fac0cda10 2024-05-27T09:29:52.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:52.187+0000 7f3fb1200640 1 -- 172.21.15.114:0/1912160185 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fac06a380 con 0x7f3fac0cda10 2024-05-27T09:29:52.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:52.188+0000 7f3fa2000640 1 -- 172.21.15.114:0/1912160185 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+6051 (secure 0 0 0) 0x7f3f9404b030 con 0x7f3fac0cda10 2024-05-27T09:29:52.189 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:29:52.189 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:29:50.348713+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"a","rank":2,"incarnation":13,"state":"up:resolve","state_seq":41,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4297]},"id":1}]} 2024-05-27T09:29:52.190 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-05-27T09:29:52.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:52.191+0000 7f3fb1200640 1 -- 172.21.15.114:0/1912160185 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f3f88076300 msgr2=0x7f3f88078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:52.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:52.191+0000 7f3fb1200640 1 --2- 172.21.15.114:0/1912160185 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f3f88076300 0x7f3f88078810 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009560 tx=0x7f3f9c009190 comp rx=0 tx=0).stop 2024-05-27T09:29:52.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:52.191+0000 7f3fb1200640 1 -- 172.21.15.114:0/1912160185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fac0cda10 msgr2=0x7f3fac0b5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:52.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:52.191+0000 7f3fb1200640 1 --2- 172.21.15.114:0/1912160185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fac0cda10 0x7f3fac0b5ee0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3f94031000 tx=0x7f3f940046c0 comp rx=0 tx=0).stop 2024-05-27T09:29:52.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:52.191+0000 7f3fb1200640 1 -- 172.21.15.114:0/1912160185 shutdown_connections 2024-05-27T09:29:52.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:52.191+0000 7f3fb1200640 1 --2- 172.21.15.114:0/1912160185 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f3f88076300 0x7f3f88078810 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:52.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:52.191+0000 7f3fb1200640 1 --2- 172.21.15.114:0/1912160185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3fac15d130 0x7f3fac151420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:52.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:52.191+0000 7f3fb1200640 1 --2- 172.21.15.114:0/1912160185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fac0cda10 0x7f3fac0b5ee0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:52.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:52.191+0000 7f3fb1200640 1 --2- 172.21.15.114:0/1912160185 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fac08dea0 0x7f3fac1f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:52.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:52.191+0000 7f3fb1200640 1 -- 172.21.15.114:0/1912160185 >> 172.21.15.114:0/1912160185 conn(0x7f3fac06f2e0 msgr2=0x7f3fac1535c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:52.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:52.191+0000 7f3fb1200640 1 -- 172.21.15.114:0/1912160185 shutdown_connections 2024-05-27T09:29:52.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:52.191+0000 7f3fb1200640 1 -- 172.21.15.114:0/1912160185 wait complete. 2024-05-27T09:29:52.217 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:29:54.218 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:29:54.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.500+0000 7f7c1ee00640 1 Processor -- start 2024-05-27T09:29:54.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.500+0000 7f7c1ee00640 1 -- start start 2024-05-27T09:29:54.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.502+0000 7f7c1ee00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7c180b2c60 0x7f7c1815d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:54.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.502+0000 7f7c1ee00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c180ca140 0x7f7c180ca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:54.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.502+0000 7f7c1ee00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c18156f80 con 0x7f7c18073310 2024-05-27T09:29:54.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.502+0000 7f7c1ee00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f7c1815d7c0 con 0x7f7c180b2c60 2024-05-27T09:29:54.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.502+0000 7f7c1ee00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f7c1815fd50 con 0x7f7c180ca140 2024-05-27T09:29:54.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.502+0000 7f7c17e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7c180b2c60 0x7f7c1815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:54.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.502+0000 7f7c1ce00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f7c18073310 0x7f7c180c3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:51818/0 (socket says 172.21.15.114:51818) 2024-05-27T09:29:54.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.502+0000 7f7c1ce00640 1 -- 172.21.15.114:0/1689961196 learned_addr learned my addr 172.21.15.114:0/1689961196 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:29:54.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.502+0000 7f7c17e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7c180b2c60 0x7f7c1815d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49496/0 (socket says 172.21.15.114:49496) 2024-05-27T09:29:54.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.503+0000 7f7c0fe00640 1 --2- 172.21.15.114:0/1689961196 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c180ca140 0x7f7c180ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:54.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.503+0000 7f7c17e00640 1 -- 172.21.15.114:0/1689961196 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c180ca140 msgr2=0x7f7c180ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:29:54.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.503+0000 7f7c17e00640 1 --2- 172.21.15.114:0/1689961196 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c180ca140 0x7f7c180ca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:54.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.503+0000 7f7c17e00640 1 -- 172.21.15.114:0/1689961196 >> v1:172.21.15.114:6789/0 conn(0x7f7c18073310 legacy=0x7f7c180c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:54.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.503+0000 7f7c17e00640 1 -- 172.21.15.114:0/1689961196 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c180cac50 con 0x7f7c180b2c60 2024-05-27T09:29:54.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.503+0000 7f7c17400640 1 -- 172.21.15.114:0/1689961196 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1619400081 0 0) 0x7f7c18156f80 con 0x7f7c18073310 2024-05-27T09:29:54.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.504+0000 7f7c17e00640 1 --2- 172.21.15.114:0/1689961196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c180b2c60 0x7f7c1815d280 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7c080097a0 tx=0x7f7c0802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967ed61ee83e27e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:54.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.504+0000 7f7c17400640 1 -- 172.21.15.114:0/1689961196 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08037070 con 0x7f7c180b2c60 2024-05-27T09:29:54.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.504+0000 7f7c17400640 1 -- 172.21.15.114:0/1689961196 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c08032440 con 0x7f7c180b2c60 2024-05-27T09:29:54.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.504+0000 7f7c17400640 1 -- 172.21.15.114:0/1689961196 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08032810 con 0x7f7c180b2c60 2024-05-27T09:29:54.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.505+0000 7f7c1ee00640 1 -- 172.21.15.114:0/1689961196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c180b2c60 msgr2=0x7f7c1815d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:54.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.505+0000 7f7c1ee00640 1 --2- 172.21.15.114:0/1689961196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c180b2c60 0x7f7c1815d280 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7c080097a0 tx=0x7f7c0802a210 comp rx=0 tx=0).stop 2024-05-27T09:29:54.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.505+0000 7f7c1ee00640 1 -- 172.21.15.114:0/1689961196 shutdown_connections 2024-05-27T09:29:54.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.505+0000 7f7c1ee00640 1 --2- 172.21.15.114:0/1689961196 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c180ca140 0x7f7c180ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:54.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.505+0000 7f7c1ee00640 1 --2- 172.21.15.114:0/1689961196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c180b2c60 0x7f7c1815d280 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:54.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.505+0000 7f7c1ee00640 1 -- 172.21.15.114:0/1689961196 >> 172.21.15.114:0/1689961196 conn(0x7f7c1806f6e0 msgr2=0x7f7c18070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:54.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.505+0000 7f7c1ee00640 1 -- 172.21.15.114:0/1689961196 shutdown_connections 2024-05-27T09:29:54.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.505+0000 7f7c1ee00640 1 -- 172.21.15.114:0/1689961196 wait complete. 2024-05-27T09:29:54.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.507+0000 7f7c1ee00640 1 Processor -- start 2024-05-27T09:29:54.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.508+0000 7f7c1ee00640 1 -- start start 2024-05-27T09:29:54.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.509+0000 7f7c1ee00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c18073310 0x7f7c180c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:54.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.509+0000 7f7c1ee00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c180b2c60 0x7f7c180c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:54.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.509+0000 7f7c1ee00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c180ca140 0x7f7c181f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:54.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.509+0000 7f7c1ee00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7c18160560 con 0x7f7c180b2c60 2024-05-27T09:29:54.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.509+0000 7f7c1ee00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7c18160280 con 0x7f7c18073310 2024-05-27T09:29:54.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.509+0000 7f7c1ee00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f7c18160840 con 0x7f7c180ca140 2024-05-27T09:29:54.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.509+0000 7f7c0fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c180b2c60 0x7f7c180c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:54.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.509+0000 7f7c0fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c180b2c60 0x7f7c180c59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49500/0 (socket says 172.21.15.114:49500) 2024-05-27T09:29:54.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.509+0000 7f7c0fe00640 1 -- 172.21.15.114:0/381853752 learned_addr learned my addr 172.21.15.114:0/381853752 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:29:54.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.509+0000 7f7c17e00640 1 --2- 172.21.15.114:0/381853752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c18073310 0x7f7c180c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:54.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.509+0000 7f7c1ce00640 1 --2- 172.21.15.114:0/381853752 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c180ca140 0x7f7c181f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:54.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.510+0000 7f7c0fe00640 1 -- 172.21.15.114:0/381853752 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c180ca140 msgr2=0x7f7c181f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:54.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.510+0000 7f7c0fe00640 1 --2- 172.21.15.114:0/381853752 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c180ca140 0x7f7c181f6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:54.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.510+0000 7f7c0fe00640 1 -- 172.21.15.114:0/381853752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c18073310 msgr2=0x7f7c180c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:54.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.510+0000 7f7c0fe00640 1 --2- 172.21.15.114:0/381853752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c18073310 0x7f7c180c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:54.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.510+0000 7f7c0fe00640 1 -- 172.21.15.114:0/381853752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c180c5ef0 con 0x7f7c180b2c60 2024-05-27T09:29:54.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.510+0000 7f7c17e00640 1 --2- 172.21.15.114:0/381853752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c18073310 0x7f7c180c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:29:54.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.510+0000 7f7c0fe00640 1 --2- 172.21.15.114:0/381853752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c180b2c60 0x7f7c180c59b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000e8a0 tx=0x7f7c0000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:54.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.510+0000 7f7c14c00640 1 -- 172.21.15.114:0/381853752 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00017070 con 0x7f7c180b2c60 2024-05-27T09:29:54.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.510+0000 7f7c14c00640 1 -- 172.21.15.114:0/381853752 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0000c940 con 0x7f7c180b2c60 2024-05-27T09:29:54.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.510+0000 7f7c14c00640 1 -- 172.21.15.114:0/381853752 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0000cd30 con 0x7f7c180b2c60 2024-05-27T09:29:54.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.510+0000 7f7c1ee00640 1 -- 172.21.15.114:0/381853752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c181f69f0 con 0x7f7c180b2c60 2024-05-27T09:29:54.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.511+0000 7f7c1ee00640 1 -- 172.21.15.114:0/381853752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c181f6cb0 con 0x7f7c180b2c60 2024-05-27T09:29:54.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.512+0000 7f7c1ee00640 1 -- 172.21.15.114:0/381853752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c1806a380 con 0x7f7c180b2c60 2024-05-27T09:29:54.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.513+0000 7f7c14c00640 1 -- 172.21.15.114:0/381853752 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7c0001a070 con 0x7f7c180b2c60 2024-05-27T09:29:54.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.514+0000 7f7c14c00640 1 --2- 172.21.15.114:0/381853752 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7bf4076300 0x7f7bf4078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:54.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.514+0000 7f7c14c00640 1 -- 172.21.15.114:0/381853752 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5824+0+0 (secure 0 0 0) 0x7f7c000974e0 con 0x7f7c180b2c60 2024-05-27T09:29:54.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.514+0000 7f7c17e00640 1 --2- 172.21.15.114:0/381853752 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7bf4076300 0x7f7bf4078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:54.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.515+0000 7f7c17e00640 1 --2- 172.21.15.114:0/381853752 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7bf4076300 0x7f7bf4078810 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7c08034000 tx=0x7f7c0802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:54.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.519+0000 7f7c14c00640 1 -- 172.21.15.114:0/381853752 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7c00060700 con 0x7f7c180b2c60 2024-05-27T09:29:54.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.756+0000 7f7c1ee00640 1 -- 172.21.15.114:0/381853752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c1815a250 con 0x7f7c180b2c60 2024-05-27T09:29:54.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.757+0000 7f7c14c00640 1 -- 172.21.15.114:0/381853752 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+6051 (secure 0 0 0) 0x7f7c00064a00 con 0x7f7c180b2c60 2024-05-27T09:29:54.758 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:29:54.758 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:29:50.348713+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"a","rank":2,"incarnation":13,"state":"up:resolve","state_seq":41,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4297]},"id":1}]} 2024-05-27T09:29:54.759 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 14 2024-05-27T09:29:54.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.759+0000 7f7c1ee00640 1 -- 172.21.15.114:0/381853752 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7bf4076300 msgr2=0x7f7bf4078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:54.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.759+0000 7f7c1ee00640 1 --2- 172.21.15.114:0/381853752 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7bf4076300 0x7f7bf4078810 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7c08034000 tx=0x7f7c0802e000 comp rx=0 tx=0).stop 2024-05-27T09:29:54.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.759+0000 7f7c1ee00640 1 -- 172.21.15.114:0/381853752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c180b2c60 msgr2=0x7f7c180c59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:54.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.759+0000 7f7c1ee00640 1 --2- 172.21.15.114:0/381853752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c180b2c60 0x7f7c180c59b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000e8a0 tx=0x7f7c0000ed60 comp rx=0 tx=0).stop 2024-05-27T09:29:54.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.759+0000 7f7c1ee00640 1 -- 172.21.15.114:0/381853752 shutdown_connections 2024-05-27T09:29:54.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.759+0000 7f7c1ee00640 1 --2- 172.21.15.114:0/381853752 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7bf4076300 0x7f7bf4078810 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:54.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.760+0000 7f7c1ee00640 1 --2- 172.21.15.114:0/381853752 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c180ca140 0x7f7c181f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:54.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.760+0000 7f7c1ee00640 1 --2- 172.21.15.114:0/381853752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c180b2c60 0x7f7c180c59b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:54.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.760+0000 7f7c1ee00640 1 --2- 172.21.15.114:0/381853752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c18073310 0x7f7c180c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:54.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.760+0000 7f7c1ee00640 1 -- 172.21.15.114:0/381853752 >> 172.21.15.114:0/381853752 conn(0x7f7c1806f6e0 msgr2=0x7f7c180cd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:54.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.760+0000 7f7c1ee00640 1 -- 172.21.15.114:0/381853752 shutdown_connections 2024-05-27T09:29:54.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:54.760+0000 7f7c1ee00640 1 -- 172.21.15.114:0/381853752 wait complete. 2024-05-27T09:29:54.785 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:29:56.786 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:29:57.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.065+0000 7fbf1f200640 1 Processor -- start 2024-05-27T09:29:57.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.065+0000 7fbf1f200640 1 -- start start 2024-05-27T09:29:57.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.067+0000 7fbf1f200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf200741f0 0x7fbf200c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:57.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.067+0000 7fbf1f200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbf201547d0 0x7fbf2015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:57.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.068+0000 7fbf1f200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf2006fe50 con 0x7fbf20074840 2024-05-27T09:29:57.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.068+0000 7fbf1f200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fbf200703d0 con 0x7fbf201547d0 2024-05-27T09:29:57.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.068+0000 7fbf1f200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fbf2015b260 con 0x7fbf200741f0 2024-05-27T09:29:57.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.068+0000 7fbf1e800640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fbf20074840 0x7fbf2006edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:44754/0 (socket says 172.21.15.114:44754) 2024-05-27T09:29:57.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.068+0000 7fbf1e800640 1 -- 172.21.15.114:0/2739064637 learned_addr learned my addr 172.21.15.114:0/2739064637 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:29:57.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.068+0000 7fbf15400640 1 --2- 172.21.15.114:0/2739064637 >> v2:172.21.15.114:3300/0 conn(0x7fbf201547d0 0x7fbf2015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:57.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.068+0000 7fbf1de00640 1 --2- 172.21.15.114:0/2739064637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf200741f0 0x7fbf200c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:57.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.068+0000 7fbf1d400640 1 -- 172.21.15.114:0/2739064637 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3809372759 0 0) 0x7fbf2006fe50 con 0x7fbf20074840 2024-05-27T09:29:57.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.069+0000 7fbf1d400640 1 -- 172.21.15.114:0/2739064637 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf04003520 con 0x7fbf20074840 2024-05-27T09:29:57.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.069+0000 7fbf15400640 1 -- 172.21.15.114:0/2739064637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf200741f0 msgr2=0x7fbf200c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:29:57.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.069+0000 7fbf15400640 1 --2- 172.21.15.114:0/2739064637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf200741f0 0x7fbf200c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:57.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.069+0000 7fbf15400640 1 -- 172.21.15.114:0/2739064637 >> v1:172.21.15.114:6789/0 conn(0x7fbf20074840 legacy=0x7fbf2006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:57.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.069+0000 7fbf15400640 1 -- 172.21.15.114:0/2739064637 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf200700f0 con 0x7fbf201547d0 2024-05-27T09:29:57.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.070+0000 7fbf15400640 1 --2- 172.21.15.114:0/2739064637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf201547d0 0x7fbf2015ad20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbf08009d90 tx=0x7fbf0802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ecd54cf19cba7c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:57.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.070+0000 7fbf1d400640 1 -- 172.21.15.114:0/2739064637 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08007740 con 0x7fbf201547d0 2024-05-27T09:29:57.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.070+0000 7fbf1d400640 1 -- 172.21.15.114:0/2739064637 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf080078e0 con 0x7fbf201547d0 2024-05-27T09:29:57.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.070+0000 7fbf1d400640 1 -- 172.21.15.114:0/2739064637 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08007cb0 con 0x7fbf201547d0 2024-05-27T09:29:57.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.071+0000 7fbf1f200640 1 -- 172.21.15.114:0/2739064637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf201547d0 msgr2=0x7fbf2015ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:57.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.071+0000 7fbf1f200640 1 --2- 172.21.15.114:0/2739064637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf201547d0 0x7fbf2015ad20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbf08009d90 tx=0x7fbf0802a6f0 comp rx=0 tx=0).stop 2024-05-27T09:29:57.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.071+0000 7fbf1f200640 1 -- 172.21.15.114:0/2739064637 shutdown_connections 2024-05-27T09:29:57.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.071+0000 7fbf1f200640 1 --2- 172.21.15.114:0/2739064637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf201547d0 0x7fbf2015ad20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:57.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.071+0000 7fbf1f200640 1 --2- 172.21.15.114:0/2739064637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf200741f0 0x7fbf200c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:57.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.071+0000 7fbf1f200640 1 -- 172.21.15.114:0/2739064637 >> 172.21.15.114:0/2739064637 conn(0x7fbf200ca320 msgr2=0x7fbf200ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:57.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.071+0000 7fbf1f200640 1 -- 172.21.15.114:0/2739064637 shutdown_connections 2024-05-27T09:29:57.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.071+0000 7fbf1f200640 1 -- 172.21.15.114:0/2739064637 wait complete. 2024-05-27T09:29:57.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.074+0000 7fbf1f200640 1 Processor -- start 2024-05-27T09:29:57.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.074+0000 7fbf1f200640 1 -- start start 2024-05-27T09:29:57.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.075+0000 7fbf1f200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf200741f0 0x7fbf201f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:57.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.075+0000 7fbf1f200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf20074840 0x7fbf2014f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:57.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.075+0000 7fbf1f200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf201547d0 0x7fbf20147060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:57.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.075+0000 7fbf1f200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbf20073790 con 0x7fbf201547d0 2024-05-27T09:29:57.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.075+0000 7fbf1f200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbf200734b0 con 0x7fbf200741f0 2024-05-27T09:29:57.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.075+0000 7fbf1f200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fbf2006f500 con 0x7fbf20074840 2024-05-27T09:29:57.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.076+0000 7fbf1e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf201547d0 0x7fbf20147060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:57.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.076+0000 7fbf1de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf200741f0 0x7fbf201f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:57.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.076+0000 7fbf1e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf201547d0 0x7fbf20147060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42808/0 (socket says 172.21.15.114:42808) 2024-05-27T09:29:57.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.076+0000 7fbf1e800640 1 -- 172.21.15.114:0/4053196071 learned_addr learned my addr 172.21.15.114:0/4053196071 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:29:57.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.076+0000 7fbf15400640 1 --2- 172.21.15.114:0/4053196071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf20074840 0x7fbf2014f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:57.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.076+0000 7fbf1e800640 1 -- 172.21.15.114:0/4053196071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf20074840 msgr2=0x7fbf2014f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:57.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.076+0000 7fbf1e800640 1 --2- 172.21.15.114:0/4053196071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf20074840 0x7fbf2014f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:57.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.076+0000 7fbf1e800640 1 -- 172.21.15.114:0/4053196071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf200741f0 msgr2=0x7fbf201f1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:57.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.076+0000 7fbf1e800640 1 --2- 172.21.15.114:0/4053196071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf200741f0 0x7fbf201f1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:57.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.076+0000 7fbf1e800640 1 -- 172.21.15.114:0/4053196071 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf201e6880 con 0x7fbf201547d0 2024-05-27T09:29:57.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.077+0000 7fbf1e800640 1 --2- 172.21.15.114:0/4053196071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf201547d0 0x7fbf20147060 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800c3c0 tx=0x7fbf1800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:57.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.077+0000 7fbf16a00640 1 -- 172.21.15.114:0/4053196071 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf18017070 con 0x7fbf201547d0 2024-05-27T09:29:57.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.077+0000 7fbf16a00640 1 -- 172.21.15.114:0/4053196071 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf18005ce0 con 0x7fbf201547d0 2024-05-27T09:29:57.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.077+0000 7fbf1f200640 1 -- 172.21.15.114:0/4053196071 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf201e6e00 con 0x7fbf201547d0 2024-05-27T09:29:57.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.077+0000 7fbf16a00640 1 -- 172.21.15.114:0/4053196071 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf180072d0 con 0x7fbf201547d0 2024-05-27T09:29:57.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.077+0000 7fbf1f200640 1 -- 172.21.15.114:0/4053196071 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf20158880 con 0x7fbf201547d0 2024-05-27T09:29:57.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.078+0000 7fbf1f200640 1 -- 172.21.15.114:0/4053196071 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf201e6b20 con 0x7fbf201547d0 2024-05-27T09:29:57.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.079+0000 7fbf16a00640 1 -- 172.21.15.114:0/4053196071 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbf1801a070 con 0x7fbf201547d0 2024-05-27T09:29:57.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.080+0000 7fbf16a00640 1 --2- 172.21.15.114:0/4053196071 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbefc0762d0 0x7fbefc0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:57.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.081+0000 7fbf16a00640 1 -- 172.21.15.114:0/4053196071 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5824+0+0 (secure 0 0 0) 0x7fbf18096850 con 0x7fbf201547d0 2024-05-27T09:29:57.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.081+0000 7fbf1de00640 1 --2- 172.21.15.114:0/4053196071 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbefc0762d0 0x7fbefc0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:57.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.081+0000 7fbf1de00640 1 --2- 172.21.15.114:0/4053196071 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbefc0762d0 0x7fbefc0787e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009aa0 tx=0x7fbf10009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:57.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.085+0000 7fbf16a00640 1 -- 172.21.15.114:0/4053196071 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbf18007e20 con 0x7fbf201547d0 2024-05-27T09:29:57.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.335+0000 7fbf1f200640 1 -- 172.21.15.114:0/4053196071 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf20074600 con 0x7fbf201547d0 2024-05-27T09:29:57.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.337+0000 7fbf16a00640 1 -- 172.21.15.114:0/4053196071 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+6053 (secure 0 0 0) 0x7fbf1805fb00 con 0x7fbf201547d0 2024-05-27T09:29:57.338 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:29:57.338 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:29:55.401852+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"a","rank":2,"incarnation":13,"state":"up:reconnect","state_seq":43,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4297,4300]},"id":1}]} 2024-05-27T09:29:57.338 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 15 2024-05-27T09:29:57.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.339+0000 7fbf1f200640 1 -- 172.21.15.114:0/4053196071 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbefc0762d0 msgr2=0x7fbefc0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:57.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.339+0000 7fbf1f200640 1 --2- 172.21.15.114:0/4053196071 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbefc0762d0 0x7fbefc0787e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009aa0 tx=0x7fbf10009190 comp rx=0 tx=0).stop 2024-05-27T09:29:57.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.339+0000 7fbf1f200640 1 -- 172.21.15.114:0/4053196071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf201547d0 msgr2=0x7fbf20147060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:57.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.339+0000 7fbf1f200640 1 --2- 172.21.15.114:0/4053196071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf201547d0 0x7fbf20147060 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800c3c0 tx=0x7fbf1800c880 comp rx=0 tx=0).stop 2024-05-27T09:29:57.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.339+0000 7fbf1f200640 1 -- 172.21.15.114:0/4053196071 shutdown_connections 2024-05-27T09:29:57.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.339+0000 7fbf1f200640 1 --2- 172.21.15.114:0/4053196071 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbefc0762d0 0x7fbefc0787e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:57.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.339+0000 7fbf1f200640 1 --2- 172.21.15.114:0/4053196071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf201547d0 0x7fbf20147060 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:57.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.339+0000 7fbf1f200640 1 --2- 172.21.15.114:0/4053196071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf20074840 0x7fbf2014f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:57.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.339+0000 7fbf1f200640 1 --2- 172.21.15.114:0/4053196071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf200741f0 0x7fbf201f1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:57.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.339+0000 7fbf1f200640 1 -- 172.21.15.114:0/4053196071 >> 172.21.15.114:0/4053196071 conn(0x7fbf200ca320 msgr2=0x7fbf201539d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:57.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.340+0000 7fbf1f200640 1 -- 172.21.15.114:0/4053196071 shutdown_connections 2024-05-27T09:29:57.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:57.340+0000 7fbf1f200640 1 -- 172.21.15.114:0/4053196071 wait complete. 2024-05-27T09:29:57.366 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:29:59.367 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:29:59.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.649+0000 7fb426600640 1 Processor -- start 2024-05-27T09:29:59.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.650+0000 7fb426600640 1 -- start start 2024-05-27T09:29:59.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.652+0000 7fb426600640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb4200c5310 0x7fb4200c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:59.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.652+0000 7fb426600640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb42015c7b0 con 0x7fb420068a00 2024-05-27T09:29:59.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.652+0000 7fb426600640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4200c8b50 con 0x7fb4200c9610 2024-05-27T09:29:59.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.652+0000 7fb426600640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fb42015d530 con 0x7fb4200c5310 2024-05-27T09:29:59.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.652+0000 7fb425200640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fb420068a00 0x7fb420156720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:44768/0 (socket says 172.21.15.114:44768) 2024-05-27T09:29:59.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.652+0000 7fb425200640 1 -- 172.21.15.114:0/586484852 learned_addr learned my addr 172.21.15.114:0/586484852 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:29:59.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.652+0000 7fb41fe00640 1 --2- 172.21.15.114:0/586484852 >> v2:172.21.15.114:3300/0 conn(0x7fb4200c5310 0x7fb4200c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:59.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.653+0000 7fb41f400640 1 -- 172.21.15.114:0/586484852 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2901339143 0 0) 0x7fb4200c8b50 con 0x7fb4200c9610 2024-05-27T09:29:59.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.653+0000 7fb41f400640 1 -- 172.21.15.114:0/586484852 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb408003520 con 0x7fb4200c9610 2024-05-27T09:29:59.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.653+0000 7fb41f400640 1 -- 172.21.15.114:0/586484852 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4141343976 0 0) 0x7fb42015c7b0 con 0x7fb420068a00 2024-05-27T09:29:59.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.653+0000 7fb41f400640 1 -- 172.21.15.114:0/586484852 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4200c8b50 con 0x7fb420068a00 2024-05-27T09:29:59.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.653+0000 7fb41fe00640 1 -- 172.21.15.114:0/586484852 >> v1:172.21.15.163:6789/0 conn(0x7fb4200c9610 legacy=0x7fb4200743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:59.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.653+0000 7fb41fe00640 1 -- 172.21.15.114:0/586484852 >> v1:172.21.15.114:6789/0 conn(0x7fb420068a00 legacy=0x7fb420156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:59.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.653+0000 7fb41fe00640 1 -- 172.21.15.114:0/586484852 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42015ca50 con 0x7fb4200c5310 2024-05-27T09:29:59.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.654+0000 7fb41fe00640 1 --2- 172.21.15.114:0/586484852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4200c5310 0x7fb4200c90d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0097a0 tx=0x7fb40c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83bfff1bd50d2322 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:59.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.654+0000 7fb41f400640 1 -- 172.21.15.114:0/586484852 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c037070 con 0x7fb4200c5310 2024-05-27T09:29:59.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.654+0000 7fb41f400640 1 -- 172.21.15.114:0/586484852 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb40c032640 con 0x7fb4200c5310 2024-05-27T09:29:59.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.654+0000 7fb41f400640 1 -- 172.21.15.114:0/586484852 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c032a10 con 0x7fb4200c5310 2024-05-27T09:29:59.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.655+0000 7fb426600640 1 -- 172.21.15.114:0/586484852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4200c5310 msgr2=0x7fb4200c90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:59.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.655+0000 7fb426600640 1 --2- 172.21.15.114:0/586484852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4200c5310 0x7fb4200c90d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0097a0 tx=0x7fb40c02a2c0 comp rx=0 tx=0).stop 2024-05-27T09:29:59.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.655+0000 7fb426600640 1 -- 172.21.15.114:0/586484852 shutdown_connections 2024-05-27T09:29:59.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.655+0000 7fb426600640 1 --2- 172.21.15.114:0/586484852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4200c5310 0x7fb4200c90d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:59.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.655+0000 7fb426600640 1 -- 172.21.15.114:0/586484852 >> 172.21.15.114:0/586484852 conn(0x7fb420073890 msgr2=0x7fb420073ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:59.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.655+0000 7fb426600640 1 -- 172.21.15.114:0/586484852 shutdown_connections 2024-05-27T09:29:59.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.656+0000 7fb426600640 1 -- 172.21.15.114:0/586484852 wait complete. 2024-05-27T09:29:59.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.658+0000 7fb426600640 1 Processor -- start 2024-05-27T09:29:59.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.658+0000 7fb426600640 1 -- start start 2024-05-27T09:29:59.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.659+0000 7fb426600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb420068a00 0x7fb42006c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:59.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.659+0000 7fb426600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4200c5310 0x7fb4201f2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:59.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.659+0000 7fb426600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4200c9610 0x7fb42014b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:59.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.659+0000 7fb426600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb42015d0a0 con 0x7fb420068a00 2024-05-27T09:29:59.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.659+0000 7fb426600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb4200c8e30 con 0x7fb4200c5310 2024-05-27T09:29:59.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.659+0000 7fb426600640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fb4200c6690 con 0x7fb4200c9610 2024-05-27T09:29:59.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.660+0000 7fb425200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb420068a00 0x7fb42006c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:59.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.660+0000 7fb425200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb420068a00 0x7fb42006c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42830/0 (socket says 172.21.15.114:42830) 2024-05-27T09:29:59.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.660+0000 7fb425200640 1 -- 172.21.15.114:0/1674907909 learned_addr learned my addr 172.21.15.114:0/1674907909 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:29:59.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.660+0000 7fb41fe00640 1 --2- 172.21.15.114:0/1674907909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4200c5310 0x7fb4201f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:59.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.660+0000 7fb425c00640 1 --2- 172.21.15.114:0/1674907909 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4200c9610 0x7fb42014b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:59.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.660+0000 7fb425200640 1 -- 172.21.15.114:0/1674907909 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4200c9610 msgr2=0x7fb42014b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:59.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.660+0000 7fb425200640 1 --2- 172.21.15.114:0/1674907909 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4200c9610 0x7fb42014b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:59.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.660+0000 7fb425200640 1 -- 172.21.15.114:0/1674907909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4200c5310 msgr2=0x7fb4201f2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:59.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.660+0000 7fb425200640 1 --2- 172.21.15.114:0/1674907909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4200c5310 0x7fb4201f2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:59.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.660+0000 7fb425200640 1 -- 172.21.15.114:0/1674907909 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb420151790 con 0x7fb420068a00 2024-05-27T09:29:59.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.660+0000 7fb425200640 1 --2- 172.21.15.114:0/1674907909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb420068a00 0x7fb42006c9e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb41400c410 tx=0x7fb41400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:59.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.661+0000 7fb41cc00640 1 -- 172.21.15.114:0/1674907909 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414017070 con 0x7fb420068a00 2024-05-27T09:29:59.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.661+0000 7fb41cc00640 1 -- 172.21.15.114:0/1674907909 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb414005ce0 con 0x7fb420068a00 2024-05-27T09:29:59.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.661+0000 7fb41cc00640 1 -- 172.21.15.114:0/1674907909 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4140072d0 con 0x7fb420068a00 2024-05-27T09:29:59.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.661+0000 7fb426600640 1 -- 172.21.15.114:0/1674907909 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4201584f0 con 0x7fb420068a00 2024-05-27T09:29:59.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.661+0000 7fb426600640 1 -- 172.21.15.114:0/1674907909 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb420158ab0 con 0x7fb420068a00 2024-05-27T09:29:59.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.662+0000 7fb426600640 1 -- 172.21.15.114:0/1674907909 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb420050860 con 0x7fb420068a00 2024-05-27T09:29:59.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.663+0000 7fb41cc00640 1 -- 172.21.15.114:0/1674907909 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb41401a070 con 0x7fb420068a00 2024-05-27T09:29:59.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.664+0000 7fb41cc00640 1 --2- 172.21.15.114:0/1674907909 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb4000762d0 0x7fb4000787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:59.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.664+0000 7fb41cc00640 1 -- 172.21.15.114:0/1674907909 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5824+0+0 (secure 0 0 0) 0x7fb414096990 con 0x7fb420068a00 2024-05-27T09:29:59.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.664+0000 7fb41fe00640 1 --2- 172.21.15.114:0/1674907909 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb4000762d0 0x7fb4000787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:59.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.665+0000 7fb41fe00640 1 --2- 172.21.15.114:0/1674907909 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb4000762d0 0x7fb4000787e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb40c034000 tx=0x7fb40c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:59.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.670+0000 7fb41cc00640 1 -- 172.21.15.114:0/1674907909 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb41405fbb0 con 0x7fb420068a00 2024-05-27T09:29:59.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.917+0000 7fb426600640 1 -- 172.21.15.114:0/1674907909 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4200c7ef0 con 0x7fb420068a00 2024-05-27T09:29:59.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.919+0000 7fb41cc00640 1 -- 172.21.15.114:0/1674907909 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+6053 (secure 0 0 0) 0x7fb414063eb0 con 0x7fb420068a00 2024-05-27T09:29:59.920 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:29:59.920 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:29:55.401852+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"a","rank":2,"incarnation":13,"state":"up:reconnect","state_seq":43,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4297,4300]},"id":1}]} 2024-05-27T09:29:59.921 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 15 2024-05-27T09:29:59.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.923+0000 7fb426600640 1 -- 172.21.15.114:0/1674907909 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb4000762d0 msgr2=0x7fb4000787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:59.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.923+0000 7fb426600640 1 --2- 172.21.15.114:0/1674907909 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb4000762d0 0x7fb4000787e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb40c034000 tx=0x7fb40c02e000 comp rx=0 tx=0).stop 2024-05-27T09:29:59.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.923+0000 7fb426600640 1 -- 172.21.15.114:0/1674907909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb420068a00 msgr2=0x7fb42006c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:59.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.923+0000 7fb426600640 1 --2- 172.21.15.114:0/1674907909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb420068a00 0x7fb42006c9e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb41400c410 tx=0x7fb41400c8d0 comp rx=0 tx=0).stop 2024-05-27T09:29:59.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.923+0000 7fb426600640 1 -- 172.21.15.114:0/1674907909 shutdown_connections 2024-05-27T09:29:59.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.923+0000 7fb426600640 1 --2- 172.21.15.114:0/1674907909 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb4000762d0 0x7fb4000787e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:59.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.923+0000 7fb426600640 1 --2- 172.21.15.114:0/1674907909 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4200c9610 0x7fb42014b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:59.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.923+0000 7fb426600640 1 --2- 172.21.15.114:0/1674907909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4200c5310 0x7fb4201f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:59.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.923+0000 7fb426600640 1 --2- 172.21.15.114:0/1674907909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb420068a00 0x7fb42006c9e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:59.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.923+0000 7fb426600640 1 -- 172.21.15.114:0/1674907909 >> 172.21.15.114:0/1674907909 conn(0x7fb420073890 msgr2=0x7fb4200c8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:59.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.923+0000 7fb426600640 1 -- 172.21.15.114:0/1674907909 shutdown_connections 2024-05-27T09:29:59.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:29:59.923+0000 7fb426600640 1 -- 172.21.15.114:0/1674907909 wait complete. 2024-05-27T09:29:59.952 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 15, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-05-27T09:27:18.079214+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 15, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4256": { "addr": "172.21.15.114:6835/1325619883", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6833", "nonce": 1325619883, "type": "v2" }, { "addr": "172.21.15.114:6835", "nonce": 1325619883, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4256, "incarnation": 13, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:reconnect", "state_seq": 43 }, "gid_4297": { "addr": "172.21.15.114:6834/657027519", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6832", "nonce": 657027519, "type": "v2" }, { "addr": "172.21.15.114:6834", "nonce": 657027519, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4297, "incarnation": 9, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4300": { "addr": "172.21.15.114:6837/1448950017", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6836", "nonce": 1448950017, "type": "v2" }, { "addr": "172.21.15.114:6837", "nonce": 1448950017, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4300, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 26, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-05-27T09:29:55.401852+0000", "qdb_cluster": [ 4297, 4300 ], "qdb_leader": 4300, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4300, "mds_1": 4297, "mds_2": 4256 } } } ], "standbys": [ { "addr": "172.21.15.163:6835/3513522318", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 3513522318, "type": "v2" }, { "addr": "172.21.15.163:6835", "nonce": 3513522318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6838/1753596112", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6836", "nonce": 1753596112, "type": "v2" }, { "addr": "172.21.15.163:6838", "nonce": 1753596112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-05-27T09:30:01.953 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:30:02.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.239+0000 7fbacb200640 1 Processor -- start 2024-05-27T09:30:02.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.240+0000 7fbacb200640 1 -- start start 2024-05-27T09:30:02.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.242+0000 7fbacb200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbac40741f0 0x7fbac40c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:02.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.242+0000 7fbacb200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbac406fe50 con 0x7fbac41547d0 2024-05-27T09:30:02.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.242+0000 7fbacb200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbac40700f0 con 0x7fbac4074840 2024-05-27T09:30:02.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.242+0000 7fbacb200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fbac40703d0 con 0x7fbac40741f0 2024-05-27T09:30:02.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.242+0000 7fbac9400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fbac41547d0 0x7fbac406edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:44782/0 (socket says 172.21.15.114:44782) 2024-05-27T09:30:02.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.242+0000 7fbac9400640 1 -- 172.21.15.114:0/3745394781 learned_addr learned my addr 172.21.15.114:0/3745394781 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:30:02.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.242+0000 7fbac9e00640 1 --2- 172.21.15.114:0/3745394781 >> v2:172.21.15.163:3300/0 conn(0x7fbac40741f0 0x7fbac40c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:02.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.243+0000 7fbac9e00640 1 -- 172.21.15.114:0/3745394781 >> v1:172.21.15.163:6789/0 conn(0x7fbac4074840 legacy=0x7fbac4164b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:02.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.243+0000 7fbac9e00640 1 -- 172.21.15.114:0/3745394781 >> v1:172.21.15.114:6789/0 conn(0x7fbac41547d0 legacy=0x7fbac406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:02.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.243+0000 7fbac8a00640 1 -- 172.21.15.114:0/3745394781 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2421429597 0 0) 0x7fbac406fe50 con 0x7fbac41547d0 2024-05-27T09:30:02.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.243+0000 7fbac9e00640 1 -- 172.21.15.114:0/3745394781 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac40731d0 con 0x7fbac40741f0 2024-05-27T09:30:02.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.244+0000 7fbac9e00640 1 --2- 172.21.15.114:0/3745394781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbac40741f0 0x7fbac40c63a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbab80097a0 tx=0x7fbab802a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f5f504917a3960d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:02.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.244+0000 7fbac8a00640 1 -- 172.21.15.114:0/3745394781 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab8037070 con 0x7fbac40741f0 2024-05-27T09:30:02.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.244+0000 7fbac8a00640 1 -- 172.21.15.114:0/3745394781 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab8032640 con 0x7fbac40741f0 2024-05-27T09:30:02.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.244+0000 7fbac8a00640 1 -- 172.21.15.114:0/3745394781 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab8032a10 con 0x7fbac40741f0 2024-05-27T09:30:02.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.245+0000 7fbacb200640 1 -- 172.21.15.114:0/3745394781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbac40741f0 msgr2=0x7fbac40c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:02.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.245+0000 7fbacb200640 1 --2- 172.21.15.114:0/3745394781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbac40741f0 0x7fbac40c63a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbab80097a0 tx=0x7fbab802a2c0 comp rx=0 tx=0).stop 2024-05-27T09:30:02.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.245+0000 7fbacb200640 1 -- 172.21.15.114:0/3745394781 shutdown_connections 2024-05-27T09:30:02.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.245+0000 7fbacb200640 1 --2- 172.21.15.114:0/3745394781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbac40741f0 0x7fbac40c63a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:02.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.245+0000 7fbacb200640 1 -- 172.21.15.114:0/3745394781 >> 172.21.15.114:0/3745394781 conn(0x7fbac40ca320 msgr2=0x7fbac40ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:02.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.245+0000 7fbacb200640 1 -- 172.21.15.114:0/3745394781 shutdown_connections 2024-05-27T09:30:02.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.245+0000 7fbacb200640 1 -- 172.21.15.114:0/3745394781 wait complete. 2024-05-27T09:30:02.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.248+0000 7fbacb200640 1 Processor -- start 2024-05-27T09:30:02.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.248+0000 7fbacb200640 1 -- start start 2024-05-27T09:30:02.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.249+0000 7fbacb200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbac40741f0 0x7fbac4153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:02.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.249+0000 7fbacb200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbac4074840 0x7fbac41e58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:02.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.249+0000 7fbacb200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbac41547d0 0x7fbac40c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:02.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.249+0000 7fbacb200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbac406f500 con 0x7fbac4074840 2024-05-27T09:30:02.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.249+0000 7fbacb200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbac4073790 con 0x7fbac40741f0 2024-05-27T09:30:02.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.249+0000 7fbacb200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fbac406f680 con 0x7fbac41547d0 2024-05-27T09:30:02.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.250+0000 7fbac9400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbac4074840 0x7fbac41e58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:02.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.250+0000 7fbac9400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbac4074840 0x7fbac41e58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42832/0 (socket says 172.21.15.114:42832) 2024-05-27T09:30:02.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.250+0000 7fbac9400640 1 -- 172.21.15.114:0/4152035882 learned_addr learned my addr 172.21.15.114:0/4152035882 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:30:02.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.250+0000 7fbac9e00640 1 --2- 172.21.15.114:0/4152035882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbac40741f0 0x7fbac4153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:02.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.250+0000 7fbaca800640 1 --2- 172.21.15.114:0/4152035882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbac41547d0 0x7fbac40c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:02.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.250+0000 7fbac9e00640 1 -- 172.21.15.114:0/4152035882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbac41547d0 msgr2=0x7fbac40c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:02.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.250+0000 7fbac9e00640 1 --2- 172.21.15.114:0/4152035882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbac41547d0 0x7fbac40c9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:02.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.250+0000 7fbac9e00640 1 -- 172.21.15.114:0/4152035882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbac4074840 msgr2=0x7fbac41e58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:02.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.250+0000 7fbac9e00640 1 --2- 172.21.15.114:0/4152035882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbac4074840 0x7fbac41e58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:02.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.250+0000 7fbac9e00640 1 -- 172.21.15.114:0/4152035882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac40c4410 con 0x7fbac40741f0 2024-05-27T09:30:02.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.250+0000 7fbac9400640 1 --2- 172.21.15.114:0/4152035882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbac4074840 0x7fbac41e58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:30:02.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.251+0000 7fbac9e00640 1 --2- 172.21.15.114:0/4152035882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbac40741f0 0x7fbac4153160 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbab8031000 tx=0x7fbab80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:02.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.251+0000 7fbabe000640 1 -- 172.21.15.114:0/4152035882 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab8037040 con 0x7fbac40741f0 2024-05-27T09:30:02.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.251+0000 7fbabe000640 1 -- 172.21.15.114:0/4152035882 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab8030770 con 0x7fbac40741f0 2024-05-27T09:30:02.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.251+0000 7fbacb200640 1 -- 172.21.15.114:0/4152035882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac41536a0 con 0x7fbac40741f0 2024-05-27T09:30:02.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.251+0000 7fbabe000640 1 -- 172.21.15.114:0/4152035882 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab8030b60 con 0x7fbac40741f0 2024-05-27T09:30:02.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.251+0000 7fbacb200640 1 -- 172.21.15.114:0/4152035882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac41f25a0 con 0x7fbac40741f0 2024-05-27T09:30:02.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.253+0000 7fbacb200640 1 -- 172.21.15.114:0/4152035882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac41f3330 con 0x7fbac40741f0 2024-05-27T09:30:02.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.253+0000 7fbabe000640 1 -- 172.21.15.114:0/4152035882 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbab8044030 con 0x7fbac40741f0 2024-05-27T09:30:02.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.254+0000 7fbabe000640 1 --2- 172.21.15.114:0/4152035882 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbaa8076300 0x7fbaa8078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:02.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.255+0000 7fbabe000640 1 -- 172.21.15.114:0/4152035882 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5824+0+0 (secure 0 0 0) 0x7fbab80b7690 con 0x7fbac40741f0 2024-05-27T09:30:02.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.255+0000 7fbac9400640 1 --2- 172.21.15.114:0/4152035882 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbaa8076300 0x7fbaa8078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:02.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.255+0000 7fbac9400640 1 --2- 172.21.15.114:0/4152035882 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbaa8076300 0x7fbaa8078810 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbab4000f60 tx=0x7fbab4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:02.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.259+0000 7fbabe000640 1 -- 172.21.15.114:0/4152035882 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbab804b060 con 0x7fbac40741f0 2024-05-27T09:30:02.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.505+0000 7fbacb200640 1 -- 172.21.15.114:0/4152035882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbac4157cb0 con 0x7fbac40741f0 2024-05-27T09:30:02.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.507+0000 7fbabe000640 1 -- 172.21.15.114:0/4152035882 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6050 (secure 0 0 0) 0x7fbab80808b0 con 0x7fbac40741f0 2024-05-27T09:30:02.507 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:30:02.507 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:30:00.875897+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"a","rank":2,"incarnation":13,"state":"up:rejoin","state_seq":45,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4297]},"id":1}]} 2024-05-27T09:30:02.507 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 16 2024-05-27T09:30:02.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.509+0000 7fbacb200640 1 -- 172.21.15.114:0/4152035882 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbaa8076300 msgr2=0x7fbaa8078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:02.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.509+0000 7fbacb200640 1 --2- 172.21.15.114:0/4152035882 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbaa8076300 0x7fbaa8078810 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbab4000f60 tx=0x7fbab4009040 comp rx=0 tx=0).stop 2024-05-27T09:30:02.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.509+0000 7fbacb200640 1 -- 172.21.15.114:0/4152035882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbac40741f0 msgr2=0x7fbac4153160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:02.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.509+0000 7fbacb200640 1 --2- 172.21.15.114:0/4152035882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbac40741f0 0x7fbac4153160 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbab8031000 tx=0x7fbab80046c0 comp rx=0 tx=0).stop 2024-05-27T09:30:02.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.509+0000 7fbacb200640 1 -- 172.21.15.114:0/4152035882 shutdown_connections 2024-05-27T09:30:02.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.509+0000 7fbacb200640 1 --2- 172.21.15.114:0/4152035882 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbaa8076300 0x7fbaa8078810 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:02.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.509+0000 7fbacb200640 1 --2- 172.21.15.114:0/4152035882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbac41547d0 0x7fbac40c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:02.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.509+0000 7fbacb200640 1 --2- 172.21.15.114:0/4152035882 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbac4074840 0x7fbac41e58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:02.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.509+0000 7fbacb200640 1 --2- 172.21.15.114:0/4152035882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbac40741f0 0x7fbac4153160 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:02.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.509+0000 7fbacb200640 1 -- 172.21.15.114:0/4152035882 >> 172.21.15.114:0/4152035882 conn(0x7fbac40ca320 msgr2=0x7fbac40c7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:02.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.509+0000 7fbacb200640 1 -- 172.21.15.114:0/4152035882 shutdown_connections 2024-05-27T09:30:02.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:02.509+0000 7fbacb200640 1 -- 172.21.15.114:0/4152035882 wait complete. 2024-05-27T09:30:02.537 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:30:04.537 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:30:04.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.819+0000 7f1d47c00640 1 Processor -- start 2024-05-27T09:30:04.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.820+0000 7f1d47c00640 1 -- start start 2024-05-27T09:30:04.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.822+0000 7f1d47c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d40154820 0x7f1d400ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:04.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.822+0000 7f1d47c00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1d400ca810 0x7f1d4015dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:04.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.822+0000 7f1d47c00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d400cac20 con 0x7f1d40074a60 2024-05-27T09:30:04.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.822+0000 7f1d47c00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f1d40154c30 con 0x7f1d400ca810 2024-05-27T09:30:04.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.822+0000 7f1d47c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f1d4015db60 con 0x7f1d40154820 2024-05-27T09:30:04.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.822+0000 7f1d45c00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1d400ca810 0x7f1d4015dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:04.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.822+0000 7f1d45200640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1d40074a60 0x7f1d40073260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:56186/0 (socket says 172.21.15.114:56186) 2024-05-27T09:30:04.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.822+0000 7f1d45200640 1 -- 172.21.15.114:0/108661416 learned_addr learned my addr 172.21.15.114:0/108661416 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:30:04.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.822+0000 7f1d37e00640 1 --2- 172.21.15.114:0/108661416 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d40154820 0x7f1d400ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:04.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.823+0000 7f1d3fe00640 1 -- 172.21.15.114:0/108661416 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 215771173 0 0) 0x7f1d400cac20 con 0x7f1d40074a60 2024-05-27T09:30:04.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.823+0000 7f1d37e00640 1 -- 172.21.15.114:0/108661416 >> v2:172.21.15.163:3300/0 conn(0x7f1d400ca810 msgr2=0x7f1d4015dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:30:04.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.823+0000 7f1d37e00640 1 --2- 172.21.15.114:0/108661416 >> v2:172.21.15.163:3300/0 conn(0x7f1d400ca810 0x7f1d4015dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:04.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.823+0000 7f1d37e00640 1 -- 172.21.15.114:0/108661416 >> v1:172.21.15.163:6789/0 conn(0x7f1d40074a60 legacy=0x7f1d40073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:04.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.823+0000 7f1d37e00640 1 -- 172.21.15.114:0/108661416 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4015a810 con 0x7f1d40154820 2024-05-27T09:30:04.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.824+0000 7f1d37e00640 1 --2- 172.21.15.114:0/108661416 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d40154820 0x7f1d400ca2d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1d280097a0 tx=0x7f1d2802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=322edab77b77d651 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:04.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.824+0000 7f1d3fe00640 1 -- 172.21.15.114:0/108661416 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28037070 con 0x7f1d40154820 2024-05-27T09:30:04.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.824+0000 7f1d3fe00640 1 -- 172.21.15.114:0/108661416 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d28032440 con 0x7f1d40154820 2024-05-27T09:30:04.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.824+0000 7f1d3fe00640 1 -- 172.21.15.114:0/108661416 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28032810 con 0x7f1d40154820 2024-05-27T09:30:04.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.825+0000 7f1d47c00640 1 -- 172.21.15.114:0/108661416 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d40154820 msgr2=0x7f1d400ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:04.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.825+0000 7f1d47c00640 1 --2- 172.21.15.114:0/108661416 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d40154820 0x7f1d400ca2d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1d280097a0 tx=0x7f1d2802a210 comp rx=0 tx=0).stop 2024-05-27T09:30:04.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.825+0000 7f1d47c00640 1 -- 172.21.15.114:0/108661416 shutdown_connections 2024-05-27T09:30:04.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.825+0000 7f1d47c00640 1 --2- 172.21.15.114:0/108661416 >> v2:172.21.15.163:3300/0 conn(0x7f1d400ca810 0x7f1d4015dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:04.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.825+0000 7f1d47c00640 1 --2- 172.21.15.114:0/108661416 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d40154820 0x7f1d400ca2d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:04.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.825+0000 7f1d47c00640 1 -- 172.21.15.114:0/108661416 >> 172.21.15.114:0/108661416 conn(0x7f1d4006f6e0 msgr2=0x7f1d40070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:04.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.825+0000 7f1d47c00640 1 -- 172.21.15.114:0/108661416 shutdown_connections 2024-05-27T09:30:04.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.825+0000 7f1d47c00640 1 -- 172.21.15.114:0/108661416 wait complete. 2024-05-27T09:30:04.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.827+0000 7f1d47c00640 1 Processor -- start 2024-05-27T09:30:04.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.827+0000 7f1d47c00640 1 -- start start 2024-05-27T09:30:04.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.829+0000 7f1d47c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d40074a60 0x7f1d4015c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:04.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.829+0000 7f1d47c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d400ca810 0x7f1d401426f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:04.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.829+0000 7f1d47c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d40154820 0x7f1d4013f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:04.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.829+0000 7f1d47c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4015b0b0 con 0x7f1d40154820 2024-05-27T09:30:04.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.829+0000 7f1d47c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4015add0 con 0x7f1d40074a60 2024-05-27T09:30:04.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.829+0000 7f1d47c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f1d4015b390 con 0x7f1d400ca810 2024-05-27T09:30:04.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.829+0000 7f1d45c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d40154820 0x7f1d4013f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:04.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.829+0000 7f1d45c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d40154820 0x7f1d4013f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39736/0 (socket says 172.21.15.114:39736) 2024-05-27T09:30:04.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.829+0000 7f1d45c00640 1 -- 172.21.15.114:0/2336337890 learned_addr learned my addr 172.21.15.114:0/2336337890 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:30:04.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.829+0000 7f1d37e00640 1 --2- 172.21.15.114:0/2336337890 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d400ca810 0x7f1d401426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:04.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.829+0000 7f1d45200640 1 --2- 172.21.15.114:0/2336337890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d40074a60 0x7f1d4015c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:04.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.830+0000 7f1d37e00640 1 -- 172.21.15.114:0/2336337890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d40074a60 msgr2=0x7f1d4015c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:04.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.830+0000 7f1d37e00640 1 --2- 172.21.15.114:0/2336337890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d40074a60 0x7f1d4015c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:04.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.830+0000 7f1d37e00640 1 -- 172.21.15.114:0/2336337890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d40154820 msgr2=0x7f1d4013f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:04.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.830+0000 7f1d37e00640 1 --2- 172.21.15.114:0/2336337890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d40154820 0x7f1d4013f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:04.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.830+0000 7f1d37e00640 1 -- 172.21.15.114:0/2336337890 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d401f3eb0 con 0x7f1d400ca810 2024-05-27T09:30:04.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.830+0000 7f1d45c00640 1 --2- 172.21.15.114:0/2336337890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d40154820 0x7f1d4013f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:30:04.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.830+0000 7f1d37e00640 1 --2- 172.21.15.114:0/2336337890 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d400ca810 0x7f1d401426f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1d28031000 tx=0x7f1d280046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:04.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.830+0000 7f1d3d600640 1 -- 172.21.15.114:0/2336337890 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28041020 con 0x7f1d400ca810 2024-05-27T09:30:04.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.830+0000 7f1d3d600640 1 -- 172.21.15.114:0/2336337890 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d28030700 con 0x7f1d400ca810 2024-05-27T09:30:04.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.830+0000 7f1d47c00640 1 -- 172.21.15.114:0/2336337890 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d401f4190 con 0x7f1d400ca810 2024-05-27T09:30:04.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.830+0000 7f1d3d600640 1 -- 172.21.15.114:0/2336337890 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28037040 con 0x7f1d400ca810 2024-05-27T09:30:04.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.830+0000 7f1d47c00640 1 -- 172.21.15.114:0/2336337890 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d40141400 con 0x7f1d400ca810 2024-05-27T09:30:04.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.831+0000 7f1d47c00640 1 -- 172.21.15.114:0/2336337890 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d40148740 con 0x7f1d400ca810 2024-05-27T09:30:04.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.833+0000 7f1d3d600640 1 -- 172.21.15.114:0/2336337890 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1d28044020 con 0x7f1d400ca810 2024-05-27T09:30:04.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.834+0000 7f1d3d600640 1 --2- 172.21.15.114:0/2336337890 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1d1c0762d0 0x7f1d1c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:04.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.834+0000 7f1d3d600640 1 -- 172.21.15.114:0/2336337890 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 5824+0+0 (secure 0 0 0) 0x7f1d280b7cb0 con 0x7f1d400ca810 2024-05-27T09:30:04.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.834+0000 7f1d45200640 1 --2- 172.21.15.114:0/2336337890 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1d1c0762d0 0x7f1d1c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:04.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.835+0000 7f1d45200640 1 --2- 172.21.15.114:0/2336337890 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1d1c0762d0 0x7f1d1c0787e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1d30000f60 tx=0x7f1d30007e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:04.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:04.840+0000 7f1d3d600640 1 -- 172.21.15.114:0/2336337890 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1d2804b030 con 0x7f1d400ca810 2024-05-27T09:30:05.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:05.081+0000 7f1d47c00640 1 -- 172.21.15.114:0/2336337890 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d4006a380 con 0x7f1d400ca810 2024-05-27T09:30:05.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:05.082+0000 7f1d3d600640 1 -- 172.21.15.114:0/2336337890 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6050 (secure 0 0 0) 0x7f1d28080f60 con 0x7f1d400ca810 2024-05-27T09:30:05.083 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 16 2024-05-27T09:30:05.083 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:30:05.083 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:30:00.875897+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"a","rank":2,"incarnation":13,"state":"up:rejoin","state_seq":45,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4297]},"id":1}]} 2024-05-27T09:30:05.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:05.085+0000 7f1d47c00640 1 -- 172.21.15.114:0/2336337890 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1d1c0762d0 msgr2=0x7f1d1c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:05.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:05.085+0000 7f1d47c00640 1 --2- 172.21.15.114:0/2336337890 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1d1c0762d0 0x7f1d1c0787e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1d30000f60 tx=0x7f1d30007e40 comp rx=0 tx=0).stop 2024-05-27T09:30:05.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:05.085+0000 7f1d47c00640 1 -- 172.21.15.114:0/2336337890 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d400ca810 msgr2=0x7f1d401426f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:05.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:05.085+0000 7f1d47c00640 1 --2- 172.21.15.114:0/2336337890 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d400ca810 0x7f1d401426f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1d28031000 tx=0x7f1d280046c0 comp rx=0 tx=0).stop 2024-05-27T09:30:05.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:05.085+0000 7f1d47c00640 1 -- 172.21.15.114:0/2336337890 shutdown_connections 2024-05-27T09:30:05.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:05.085+0000 7f1d47c00640 1 --2- 172.21.15.114:0/2336337890 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1d1c0762d0 0x7f1d1c0787e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:05.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:05.085+0000 7f1d47c00640 1 --2- 172.21.15.114:0/2336337890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d40154820 0x7f1d4013f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:05.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:05.085+0000 7f1d47c00640 1 --2- 172.21.15.114:0/2336337890 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d400ca810 0x7f1d401426f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:05.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:05.085+0000 7f1d47c00640 1 --2- 172.21.15.114:0/2336337890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d40074a60 0x7f1d4015c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:05.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:05.085+0000 7f1d47c00640 1 -- 172.21.15.114:0/2336337890 >> 172.21.15.114:0/2336337890 conn(0x7f1d4006f6e0 msgr2=0x7f1d400ca0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:05.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:05.085+0000 7f1d47c00640 1 -- 172.21.15.114:0/2336337890 shutdown_connections 2024-05-27T09:30:05.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:05.085+0000 7f1d47c00640 1 -- 172.21.15.114:0/2336337890 wait complete. 2024-05-27T09:30:05.110 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:30:07.110 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:30:07.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.385+0000 7f2ff5e00640 1 Processor -- start 2024-05-27T09:30:07.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.385+0000 7f2ff5e00640 1 -- start start 2024-05-27T09:30:07.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.387+0000 7f2ff5e00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2ff0164db0 0x7f2ff00c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:07.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.387+0000 7f2ff5e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2ff0074860 0x7f2ff015aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:07.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.387+0000 7f2ff5e00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ff015b000 con 0x7f2ff0160b40 2024-05-27T09:30:07.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.387+0000 7f2ff5e00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2ff00cade0 con 0x7f2ff0074860 2024-05-27T09:30:07.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.387+0000 7f2ff5e00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f2ff00c3d60 con 0x7f2ff0164db0 2024-05-27T09:30:07.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.388+0000 7f2fe7e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2ff0074860 0x7f2ff015aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:07.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.388+0000 7f2fef400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f2ff0160b40 0x7f2ff0073310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54942/0 (socket says 172.21.15.114:54942) 2024-05-27T09:30:07.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.388+0000 7f2fef400640 1 -- 172.21.15.114:0/4033120732 learned_addr learned my addr 172.21.15.114:0/4033120732 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:30:07.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.388+0000 7f2fe7e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2ff0074860 0x7f2ff015aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39748/0 (socket says 172.21.15.114:39748) 2024-05-27T09:30:07.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.388+0000 7f2feea00640 1 --2- 172.21.15.114:0/4033120732 >> v2:172.21.15.163:3300/0 conn(0x7f2ff0164db0 0x7f2ff00c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:07.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.388+0000 7f2fe7e00640 1 -- 172.21.15.114:0/4033120732 >> v2:172.21.15.163:3300/0 conn(0x7f2ff0164db0 msgr2=0x7f2ff00c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:30:07.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.388+0000 7f2fe7e00640 1 --2- 172.21.15.114:0/4033120732 >> v2:172.21.15.163:3300/0 conn(0x7f2ff0164db0 0x7f2ff00c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:07.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.388+0000 7f2fe7e00640 1 -- 172.21.15.114:0/4033120732 >> v1:172.21.15.114:6789/0 conn(0x7f2ff0160b40 legacy=0x7f2ff0073310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:07.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.388+0000 7f2fe7e00640 1 -- 172.21.15.114:0/4033120732 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff01615d0 con 0x7f2ff0074860 2024-05-27T09:30:07.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.388+0000 7f2fee000640 1 -- 172.21.15.114:0/4033120732 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2069434943 0 0) 0x7f2ff015b000 con 0x7f2ff0160b40 2024-05-27T09:30:07.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.389+0000 7f2fe7e00640 1 --2- 172.21.15.114:0/4033120732 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ff0074860 0x7f2ff015aac0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2ff006d780 tx=0x7f2fd802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1cb7482e1722ef7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:07.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.390+0000 7f2fee000640 1 -- 172.21.15.114:0/4033120732 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8037070 con 0x7f2ff0074860 2024-05-27T09:30:07.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.390+0000 7f2fee000640 1 -- 172.21.15.114:0/4033120732 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd8032440 con 0x7f2ff0074860 2024-05-27T09:30:07.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.390+0000 7f2fee000640 1 -- 172.21.15.114:0/4033120732 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8032810 con 0x7f2ff0074860 2024-05-27T09:30:07.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.390+0000 7f2ff5e00640 1 -- 172.21.15.114:0/4033120732 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ff0074860 msgr2=0x7f2ff015aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:07.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.390+0000 7f2ff5e00640 1 --2- 172.21.15.114:0/4033120732 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ff0074860 0x7f2ff015aac0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2ff006d780 tx=0x7f2fd802a230 comp rx=0 tx=0).stop 2024-05-27T09:30:07.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.390+0000 7f2ff5e00640 1 -- 172.21.15.114:0/4033120732 shutdown_connections 2024-05-27T09:30:07.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.390+0000 7f2ff5e00640 1 --2- 172.21.15.114:0/4033120732 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ff0074860 0x7f2ff015aac0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:07.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.390+0000 7f2ff5e00640 1 --2- 172.21.15.114:0/4033120732 >> v2:172.21.15.163:3300/0 conn(0x7f2ff0164db0 0x7f2ff00c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:07.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.390+0000 7f2ff5e00640 1 -- 172.21.15.114:0/4033120732 >> 172.21.15.114:0/4033120732 conn(0x7f2ff006f6e0 msgr2=0x7f2ff0070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:07.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.391+0000 7f2ff5e00640 1 -- 172.21.15.114:0/4033120732 shutdown_connections 2024-05-27T09:30:07.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.391+0000 7f2ff5e00640 1 -- 172.21.15.114:0/4033120732 wait complete. 2024-05-27T09:30:07.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.393+0000 7f2ff5e00640 1 Processor -- start 2024-05-27T09:30:07.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.393+0000 7f2ff5e00640 1 -- start start 2024-05-27T09:30:07.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.395+0000 7f2ff5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ff0074860 0x7f2ff01ecc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:07.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.395+0000 7f2ff5e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ff0160b40 0x7f2ff0158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:07.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.395+0000 7f2ff5e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ff0164db0 0x7f2ff0158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:07.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.395+0000 7f2ff5e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff01645b0 con 0x7f2ff0164db0 2024-05-27T09:30:07.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.395+0000 7f2ff5e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff01642d0 con 0x7f2ff0074860 2024-05-27T09:30:07.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.395+0000 7f2ff5e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f2ff0164890 con 0x7f2ff0160b40 2024-05-27T09:30:07.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.395+0000 7f2fef400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ff0164db0 0x7f2ff0158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:07.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.395+0000 7f2fef400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ff0164db0 0x7f2ff0158d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39750/0 (socket says 172.21.15.114:39750) 2024-05-27T09:30:07.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.395+0000 7f2fef400640 1 -- 172.21.15.114:0/2334453011 learned_addr learned my addr 172.21.15.114:0/2334453011 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:30:07.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.395+0000 7f2fe7e00640 1 --2- 172.21.15.114:0/2334453011 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ff0160b40 0x7f2ff0158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:07.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.395+0000 7f2feea00640 1 --2- 172.21.15.114:0/2334453011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ff0074860 0x7f2ff01ecc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:07.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.395+0000 7f2fef400640 1 -- 172.21.15.114:0/2334453011 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ff0160b40 msgr2=0x7f2ff0158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:07.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.395+0000 7f2fef400640 1 --2- 172.21.15.114:0/2334453011 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ff0160b40 0x7f2ff0158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:07.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.395+0000 7f2fef400640 1 -- 172.21.15.114:0/2334453011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ff0074860 msgr2=0x7f2ff01ecc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:07.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.395+0000 7f2fef400640 1 --2- 172.21.15.114:0/2334453011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ff0074860 0x7f2ff01ecc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:07.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.395+0000 7f2fef400640 1 -- 172.21.15.114:0/2334453011 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff01f6b20 con 0x7f2ff0164db0 2024-05-27T09:30:07.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.396+0000 7f2fef400640 1 --2- 172.21.15.114:0/2334453011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ff0164db0 0x7f2ff0158d70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800cad0 tx=0x7f2fe800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:07.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.396+0000 7f2fe7400640 1 -- 172.21.15.114:0/2334453011 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe800abd0 con 0x7f2ff0164db0 2024-05-27T09:30:07.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.396+0000 7f2fe7400640 1 -- 172.21.15.114:0/2334453011 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe800ad70 con 0x7f2ff0164db0 2024-05-27T09:30:07.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.396+0000 7f2fe7400640 1 -- 172.21.15.114:0/2334453011 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8012640 con 0x7f2ff0164db0 2024-05-27T09:30:07.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.396+0000 7f2ff5e00640 1 -- 172.21.15.114:0/2334453011 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff00cd5d0 con 0x7f2ff0164db0 2024-05-27T09:30:07.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.396+0000 7f2ff5e00640 1 -- 172.21.15.114:0/2334453011 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff01f6de0 con 0x7f2ff0164db0 2024-05-27T09:30:07.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.397+0000 7f2ff5e00640 1 -- 172.21.15.114:0/2334453011 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff00cbb60 con 0x7f2ff0164db0 2024-05-27T09:30:07.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.399+0000 7f2fe7400640 1 -- 172.21.15.114:0/2334453011 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2fe80127e0 con 0x7f2ff0164db0 2024-05-27T09:30:07.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.399+0000 7f2fe7400640 1 --2- 172.21.15.114:0/2334453011 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2fcc076300 0x7f2fcc078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:07.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.400+0000 7f2fe7400640 1 -- 172.21.15.114:0/2334453011 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5824+0+0 (secure 0 0 0) 0x7f2fe8097a60 con 0x7f2ff0164db0 2024-05-27T09:30:07.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.400+0000 7f2feea00640 1 --2- 172.21.15.114:0/2334453011 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2fcc076300 0x7f2fcc078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:07.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.401+0000 7f2feea00640 1 --2- 172.21.15.114:0/2334453011 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2fcc076300 0x7f2fcc078810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009aa0 tx=0x7f2fe0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:07.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.405+0000 7f2fe7400640 1 -- 172.21.15.114:0/2334453011 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2fe8060d10 con 0x7f2ff0164db0 2024-05-27T09:30:07.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.655+0000 7f2ff5e00640 1 -- 172.21.15.114:0/2334453011 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ff015e790 con 0x7f2ff0164db0 2024-05-27T09:30:07.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.657+0000 7f2fe7400640 1 -- 172.21.15.114:0/2334453011 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6055 (secure 0 0 0) 0x7f2fe8065010 con 0x7f2ff0164db0 2024-05-27T09:30:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:30:07.657 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:30:05.919439+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"a","rank":2,"incarnation":13,"state":"up:active","state_seq":47,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4256,4297,4300]},"id":1}]} 2024-05-27T09:30:07.657 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 17 2024-05-27T09:30:07.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.659+0000 7f2ff5e00640 1 -- 172.21.15.114:0/2334453011 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2fcc076300 msgr2=0x7f2fcc078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:07.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.659+0000 7f2ff5e00640 1 --2- 172.21.15.114:0/2334453011 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2fcc076300 0x7f2fcc078810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009aa0 tx=0x7f2fe0009190 comp rx=0 tx=0).stop 2024-05-27T09:30:07.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.659+0000 7f2ff5e00640 1 -- 172.21.15.114:0/2334453011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ff0164db0 msgr2=0x7f2ff0158d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:07.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.659+0000 7f2ff5e00640 1 --2- 172.21.15.114:0/2334453011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ff0164db0 0x7f2ff0158d70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800cad0 tx=0x7f2fe800cf90 comp rx=0 tx=0).stop 2024-05-27T09:30:07.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.660+0000 7f2ff5e00640 1 -- 172.21.15.114:0/2334453011 shutdown_connections 2024-05-27T09:30:07.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.660+0000 7f2ff5e00640 1 --2- 172.21.15.114:0/2334453011 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2fcc076300 0x7f2fcc078810 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:07.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.660+0000 7f2ff5e00640 1 --2- 172.21.15.114:0/2334453011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ff0164db0 0x7f2ff0158d70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:07.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.660+0000 7f2ff5e00640 1 --2- 172.21.15.114:0/2334453011 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ff0160b40 0x7f2ff0158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:07.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.660+0000 7f2ff5e00640 1 --2- 172.21.15.114:0/2334453011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ff0074860 0x7f2ff01ecc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:07.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.660+0000 7f2ff5e00640 1 -- 172.21.15.114:0/2334453011 >> 172.21.15.114:0/2334453011 conn(0x7f2ff006f6e0 msgr2=0x7f2ff015aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:07.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.660+0000 7f2ff5e00640 1 -- 172.21.15.114:0/2334453011 shutdown_connections 2024-05-27T09:30:07.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:07.660+0000 7f2ff5e00640 1 -- 172.21.15.114:0/2334453011 wait complete. 2024-05-27T09:30:07.685 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=2, replacing gid=4279 2024-05-27T09:30:07.685 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 9 secs before reviving mds.f 2024-05-27T09:30:08.041 INFO:tasks.ceph.mds.e.smithi114.stderr:2024-05-27T09:30:08.041+0000 7f4ffb600640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-27T09:30:10.962 INFO:tasks.workunit.client.0.smithi114.stdout:Create dir 200 to 299 ... 2024-05-27T09:30:10.966 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 200 2024-05-27T09:30:10.990 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 201 2024-05-27T09:30:11.013 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 202 2024-05-27T09:30:11.035 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 203 2024-05-27T09:30:11.064 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 204 2024-05-27T09:30:11.096 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 205 2024-05-27T09:30:11.127 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 206 2024-05-27T09:30:11.160 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 207 2024-05-27T09:30:11.196 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 208 2024-05-27T09:30:11.231 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 209 2024-05-27T09:30:11.266 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 210 2024-05-27T09:30:11.301 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 211 2024-05-27T09:30:11.337 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 212 2024-05-27T09:30:11.375 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 213 2024-05-27T09:30:11.414 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 214 2024-05-27T09:30:11.458 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 215 2024-05-27T09:30:11.494 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 216 2024-05-27T09:30:11.533 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 217 2024-05-27T09:30:11.572 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 218 2024-05-27T09:30:11.613 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 219 2024-05-27T09:30:11.657 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 220 2024-05-27T09:30:11.694 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 221 2024-05-27T09:30:11.733 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 222 2024-05-27T09:30:11.772 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 223 2024-05-27T09:30:11.810 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 224 2024-05-27T09:30:11.848 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 225 2024-05-27T09:30:11.886 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 226 2024-05-27T09:30:11.923 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 227 2024-05-27T09:30:11.955 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 228 2024-05-27T09:30:11.994 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 229 2024-05-27T09:30:12.021 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 230 2024-05-27T09:30:12.057 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 231 2024-05-27T09:30:12.095 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 232 2024-05-27T09:30:12.132 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 233 2024-05-27T09:30:12.169 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 234 2024-05-27T09:30:12.207 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 235 2024-05-27T09:30:12.245 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 236 2024-05-27T09:30:12.282 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 237 2024-05-27T09:30:12.320 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 238 2024-05-27T09:30:12.357 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 239 2024-05-27T09:30:12.390 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 240 2024-05-27T09:30:12.425 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 241 2024-05-27T09:30:12.464 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 242 2024-05-27T09:30:12.498 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 243 2024-05-27T09:30:12.532 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 244 2024-05-27T09:30:12.567 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 245 2024-05-27T09:30:12.602 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 246 2024-05-27T09:30:12.637 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 247 2024-05-27T09:30:12.678 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 248 2024-05-27T09:30:12.711 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 249 2024-05-27T09:30:12.751 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 250 2024-05-27T09:30:12.805 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 251 2024-05-27T09:30:13.680 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 252 2024-05-27T09:30:13.714 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 253 2024-05-27T09:30:13.752 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 254 2024-05-27T09:30:13.789 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 255 2024-05-27T09:30:13.825 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 256 2024-05-27T09:30:13.858 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 257 2024-05-27T09:30:13.894 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 258 2024-05-27T09:30:13.931 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 259 2024-05-27T09:30:13.969 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 260 2024-05-27T09:30:14.008 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 261 2024-05-27T09:30:14.045 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 262 2024-05-27T09:30:14.081 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 263 2024-05-27T09:30:14.113 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 264 2024-05-27T09:30:14.156 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 265 2024-05-27T09:30:14.208 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 266 2024-05-27T09:30:14.260 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 267 2024-05-27T09:30:14.303 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 268 2024-05-27T09:30:14.345 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 269 2024-05-27T09:30:14.393 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 270 2024-05-27T09:30:14.429 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 271 2024-05-27T09:30:14.461 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 272 2024-05-27T09:30:14.494 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 273 2024-05-27T09:30:14.528 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 274 2024-05-27T09:30:14.564 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 275 2024-05-27T09:30:14.606 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 276 2024-05-27T09:30:14.643 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 277 2024-05-27T09:30:14.683 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 278 2024-05-27T09:30:14.712 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 279 2024-05-27T09:30:14.749 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 280 2024-05-27T09:30:14.787 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 281 2024-05-27T09:30:14.825 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 282 2024-05-27T09:30:14.875 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 283 2024-05-27T09:30:14.913 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 284 2024-05-27T09:30:14.952 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 285 2024-05-27T09:30:14.984 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 286 2024-05-27T09:30:15.021 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 287 2024-05-27T09:30:15.055 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 288 2024-05-27T09:30:15.093 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 289 2024-05-27T09:30:15.130 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 290 2024-05-27T09:30:15.169 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 291 2024-05-27T09:30:15.206 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 292 2024-05-27T09:30:15.245 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 293 2024-05-27T09:30:15.282 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 294 2024-05-27T09:30:15.319 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 295 2024-05-27T09:30:15.358 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 296 2024-05-27T09:30:15.396 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 297 2024-05-27T09:30:15.434 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 298 2024-05-27T09:30:15.471 INFO:tasks.workunit.client.0.smithi114.stdout: create dir 299 2024-05-27T09:30:15.508 INFO:tasks.workunit.client.0.smithi114.stdout:Create a snapshot in every first level dir ... 2024-05-27T09:30:15.731 INFO:tasks.workunit.client.0.smithi114.stdout: create 100/.snap/snap-subdir-test 2024-05-27T09:30:15.769 INFO:tasks.workunit.client.0.smithi114.stdout: create 100/30 file after the snapshot 2024-05-27T09:30:15.772 INFO:tasks.workunit.client.0.smithi114.stdout: create 100/31 file after the snapshot 2024-05-27T09:30:15.774 INFO:tasks.workunit.client.0.smithi114.stdout: create 100/32 file after the snapshot 2024-05-27T09:30:15.777 INFO:tasks.workunit.client.0.smithi114.stdout: create 100/33 file after the snapshot 2024-05-27T09:30:15.780 INFO:tasks.workunit.client.0.smithi114.stdout: create 100/34 file after the snapshot 2024-05-27T09:30:15.783 INFO:tasks.workunit.client.0.smithi114.stdout: create 100/35 file after the snapshot 2024-05-27T09:30:15.785 INFO:tasks.workunit.client.0.smithi114.stdout: create 100/36 file after the snapshot 2024-05-27T09:30:15.788 INFO:tasks.workunit.client.0.smithi114.stdout: create 100/37 file after the snapshot 2024-05-27T09:30:15.791 INFO:tasks.workunit.client.0.smithi114.stdout: create 100/38 file after the snapshot 2024-05-27T09:30:15.794 INFO:tasks.workunit.client.0.smithi114.stdout: create 100/39 file after the snapshot 2024-05-27T09:30:15.797 INFO:tasks.workunit.client.0.smithi114.stdout: create 101/.snap/snap-subdir-test 2024-05-27T09:30:15.840 INFO:tasks.workunit.client.0.smithi114.stdout: create 101/30 file after the snapshot 2024-05-27T09:30:15.842 INFO:tasks.workunit.client.0.smithi114.stdout: create 101/31 file after the snapshot 2024-05-27T09:30:15.845 INFO:tasks.workunit.client.0.smithi114.stdout: create 101/32 file after the snapshot 2024-05-27T09:30:15.847 INFO:tasks.workunit.client.0.smithi114.stdout: create 101/33 file after the snapshot 2024-05-27T09:30:15.850 INFO:tasks.workunit.client.0.smithi114.stdout: create 101/34 file after the snapshot 2024-05-27T09:30:15.853 INFO:tasks.workunit.client.0.smithi114.stdout: create 101/35 file after the snapshot 2024-05-27T09:30:15.855 INFO:tasks.workunit.client.0.smithi114.stdout: create 101/36 file after the snapshot 2024-05-27T09:30:15.858 INFO:tasks.workunit.client.0.smithi114.stdout: create 101/37 file after the snapshot 2024-05-27T09:30:15.861 INFO:tasks.workunit.client.0.smithi114.stdout: create 101/38 file after the snapshot 2024-05-27T09:30:15.864 INFO:tasks.workunit.client.0.smithi114.stdout: create 101/39 file after the snapshot 2024-05-27T09:30:15.866 INFO:tasks.workunit.client.0.smithi114.stdout: create 102/.snap/snap-subdir-test 2024-05-27T09:30:15.915 INFO:tasks.workunit.client.0.smithi114.stdout: create 102/30 file after the snapshot 2024-05-27T09:30:15.918 INFO:tasks.workunit.client.0.smithi114.stdout: create 102/31 file after the snapshot 2024-05-27T09:30:15.920 INFO:tasks.workunit.client.0.smithi114.stdout: create 102/32 file after the snapshot 2024-05-27T09:30:15.923 INFO:tasks.workunit.client.0.smithi114.stdout: create 102/33 file after the snapshot 2024-05-27T09:30:15.925 INFO:tasks.workunit.client.0.smithi114.stdout: create 102/34 file after the snapshot 2024-05-27T09:30:15.928 INFO:tasks.workunit.client.0.smithi114.stdout: create 102/35 file after the snapshot 2024-05-27T09:30:15.930 INFO:tasks.workunit.client.0.smithi114.stdout: create 102/36 file after the snapshot 2024-05-27T09:30:15.933 INFO:tasks.workunit.client.0.smithi114.stdout: create 102/37 file after the snapshot 2024-05-27T09:30:15.936 INFO:tasks.workunit.client.0.smithi114.stdout: create 102/38 file after the snapshot 2024-05-27T09:30:15.939 INFO:tasks.workunit.client.0.smithi114.stdout: create 102/39 file after the snapshot 2024-05-27T09:30:15.941 INFO:tasks.workunit.client.0.smithi114.stdout: create 103/.snap/snap-subdir-test 2024-05-27T09:30:15.988 INFO:tasks.workunit.client.0.smithi114.stdout: create 103/30 file after the snapshot 2024-05-27T09:30:15.991 INFO:tasks.workunit.client.0.smithi114.stdout: create 103/31 file after the snapshot 2024-05-27T09:30:15.994 INFO:tasks.workunit.client.0.smithi114.stdout: create 103/32 file after the snapshot 2024-05-27T09:30:15.997 INFO:tasks.workunit.client.0.smithi114.stdout: create 103/33 file after the snapshot 2024-05-27T09:30:16.000 INFO:tasks.workunit.client.0.smithi114.stdout: create 103/34 file after the snapshot 2024-05-27T09:30:16.002 INFO:tasks.workunit.client.0.smithi114.stdout: create 103/35 file after the snapshot 2024-05-27T09:30:16.006 INFO:tasks.workunit.client.0.smithi114.stdout: create 103/36 file after the snapshot 2024-05-27T09:30:16.008 INFO:tasks.workunit.client.0.smithi114.stdout: create 103/37 file after the snapshot 2024-05-27T09:30:16.011 INFO:tasks.workunit.client.0.smithi114.stdout: create 103/38 file after the snapshot 2024-05-27T09:30:16.014 INFO:tasks.workunit.client.0.smithi114.stdout: create 103/39 file after the snapshot 2024-05-27T09:30:16.017 INFO:tasks.workunit.client.0.smithi114.stdout: create 104/.snap/snap-subdir-test 2024-05-27T09:30:16.066 INFO:tasks.workunit.client.0.smithi114.stdout: create 104/30 file after the snapshot 2024-05-27T09:30:16.069 INFO:tasks.workunit.client.0.smithi114.stdout: create 104/31 file after the snapshot 2024-05-27T09:30:16.071 INFO:tasks.workunit.client.0.smithi114.stdout: create 104/32 file after the snapshot 2024-05-27T09:30:16.074 INFO:tasks.workunit.client.0.smithi114.stdout: create 104/33 file after the snapshot 2024-05-27T09:30:16.077 INFO:tasks.workunit.client.0.smithi114.stdout: create 104/34 file after the snapshot 2024-05-27T09:30:16.080 INFO:tasks.workunit.client.0.smithi114.stdout: create 104/35 file after the snapshot 2024-05-27T09:30:16.082 INFO:tasks.workunit.client.0.smithi114.stdout: create 104/36 file after the snapshot 2024-05-27T09:30:16.085 INFO:tasks.workunit.client.0.smithi114.stdout: create 104/37 file after the snapshot 2024-05-27T09:30:16.088 INFO:tasks.workunit.client.0.smithi114.stdout: create 104/38 file after the snapshot 2024-05-27T09:30:16.091 INFO:tasks.workunit.client.0.smithi114.stdout: create 104/39 file after the snapshot 2024-05-27T09:30:16.093 INFO:tasks.workunit.client.0.smithi114.stdout: create 105/.snap/snap-subdir-test 2024-05-27T09:30:16.142 INFO:tasks.workunit.client.0.smithi114.stdout: create 105/30 file after the snapshot 2024-05-27T09:30:16.145 INFO:tasks.workunit.client.0.smithi114.stdout: create 105/31 file after the snapshot 2024-05-27T09:30:16.147 INFO:tasks.workunit.client.0.smithi114.stdout: create 105/32 file after the snapshot 2024-05-27T09:30:16.150 INFO:tasks.workunit.client.0.smithi114.stdout: create 105/33 file after the snapshot 2024-05-27T09:30:16.153 INFO:tasks.workunit.client.0.smithi114.stdout: create 105/34 file after the snapshot 2024-05-27T09:30:16.156 INFO:tasks.workunit.client.0.smithi114.stdout: create 105/35 file after the snapshot 2024-05-27T09:30:16.159 INFO:tasks.workunit.client.0.smithi114.stdout: create 105/36 file after the snapshot 2024-05-27T09:30:16.161 INFO:tasks.workunit.client.0.smithi114.stdout: create 105/37 file after the snapshot 2024-05-27T09:30:16.164 INFO:tasks.workunit.client.0.smithi114.stdout: create 105/38 file after the snapshot 2024-05-27T09:30:16.167 INFO:tasks.workunit.client.0.smithi114.stdout: create 105/39 file after the snapshot 2024-05-27T09:30:16.170 INFO:tasks.workunit.client.0.smithi114.stdout: create 106/.snap/snap-subdir-test 2024-05-27T09:30:16.214 INFO:tasks.workunit.client.0.smithi114.stdout: create 106/30 file after the snapshot 2024-05-27T09:30:16.216 INFO:tasks.workunit.client.0.smithi114.stdout: create 106/31 file after the snapshot 2024-05-27T09:30:16.219 INFO:tasks.workunit.client.0.smithi114.stdout: create 106/32 file after the snapshot 2024-05-27T09:30:16.222 INFO:tasks.workunit.client.0.smithi114.stdout: create 106/33 file after the snapshot 2024-05-27T09:30:16.225 INFO:tasks.workunit.client.0.smithi114.stdout: create 106/34 file after the snapshot 2024-05-27T09:30:16.228 INFO:tasks.workunit.client.0.smithi114.stdout: create 106/35 file after the snapshot 2024-05-27T09:30:16.231 INFO:tasks.workunit.client.0.smithi114.stdout: create 106/36 file after the snapshot 2024-05-27T09:30:16.234 INFO:tasks.workunit.client.0.smithi114.stdout: create 106/37 file after the snapshot 2024-05-27T09:30:16.237 INFO:tasks.workunit.client.0.smithi114.stdout: create 106/38 file after the snapshot 2024-05-27T09:30:16.239 INFO:tasks.workunit.client.0.smithi114.stdout: create 106/39 file after the snapshot 2024-05-27T09:30:16.242 INFO:tasks.workunit.client.0.smithi114.stdout: create 107/.snap/snap-subdir-test 2024-05-27T09:30:16.297 INFO:tasks.workunit.client.0.smithi114.stdout: create 107/30 file after the snapshot 2024-05-27T09:30:16.299 INFO:tasks.workunit.client.0.smithi114.stdout: create 107/31 file after the snapshot 2024-05-27T09:30:16.300 INFO:tasks.workunit.client.0.smithi114.stdout: create 107/32 file after the snapshot 2024-05-27T09:30:16.302 INFO:tasks.workunit.client.0.smithi114.stdout: create 107/33 file after the snapshot 2024-05-27T09:30:16.304 INFO:tasks.workunit.client.0.smithi114.stdout: create 107/34 file after the snapshot 2024-05-27T09:30:16.306 INFO:tasks.workunit.client.0.smithi114.stdout: create 107/35 file after the snapshot 2024-05-27T09:30:16.308 INFO:tasks.workunit.client.0.smithi114.stdout: create 107/36 file after the snapshot 2024-05-27T09:30:16.310 INFO:tasks.workunit.client.0.smithi114.stdout: create 107/37 file after the snapshot 2024-05-27T09:30:16.311 INFO:tasks.workunit.client.0.smithi114.stdout: create 107/38 file after the snapshot 2024-05-27T09:30:16.313 INFO:tasks.workunit.client.0.smithi114.stdout: create 107/39 file after the snapshot 2024-05-27T09:30:16.315 INFO:tasks.workunit.client.0.smithi114.stdout: create 108/.snap/snap-subdir-test 2024-05-27T09:30:16.370 INFO:tasks.workunit.client.0.smithi114.stdout: create 108/30 file after the snapshot 2024-05-27T09:30:16.373 INFO:tasks.workunit.client.0.smithi114.stdout: create 108/31 file after the snapshot 2024-05-27T09:30:16.376 INFO:tasks.workunit.client.0.smithi114.stdout: create 108/32 file after the snapshot 2024-05-27T09:30:16.379 INFO:tasks.workunit.client.0.smithi114.stdout: create 108/33 file after the snapshot 2024-05-27T09:30:16.381 INFO:tasks.workunit.client.0.smithi114.stdout: create 108/34 file after the snapshot 2024-05-27T09:30:16.384 INFO:tasks.workunit.client.0.smithi114.stdout: create 108/35 file after the snapshot 2024-05-27T09:30:16.387 INFO:tasks.workunit.client.0.smithi114.stdout: create 108/36 file after the snapshot 2024-05-27T09:30:16.390 INFO:tasks.workunit.client.0.smithi114.stdout: create 108/37 file after the snapshot 2024-05-27T09:30:16.393 INFO:tasks.workunit.client.0.smithi114.stdout: create 108/38 file after the snapshot 2024-05-27T09:30:16.395 INFO:tasks.workunit.client.0.smithi114.stdout: create 108/39 file after the snapshot 2024-05-27T09:30:16.398 INFO:tasks.workunit.client.0.smithi114.stdout: create 109/.snap/snap-subdir-test 2024-05-27T09:30:16.446 INFO:tasks.workunit.client.0.smithi114.stdout: create 109/30 file after the snapshot 2024-05-27T09:30:16.449 INFO:tasks.workunit.client.0.smithi114.stdout: create 109/31 file after the snapshot 2024-05-27T09:30:16.452 INFO:tasks.workunit.client.0.smithi114.stdout: create 109/32 file after the snapshot 2024-05-27T09:30:16.454 INFO:tasks.workunit.client.0.smithi114.stdout: create 109/33 file after the snapshot 2024-05-27T09:30:16.457 INFO:tasks.workunit.client.0.smithi114.stdout: create 109/34 file after the snapshot 2024-05-27T09:30:16.460 INFO:tasks.workunit.client.0.smithi114.stdout: create 109/35 file after the snapshot 2024-05-27T09:30:16.463 INFO:tasks.workunit.client.0.smithi114.stdout: create 109/36 file after the snapshot 2024-05-27T09:30:16.465 INFO:tasks.workunit.client.0.smithi114.stdout: create 109/37 file after the snapshot 2024-05-27T09:30:16.468 INFO:tasks.workunit.client.0.smithi114.stdout: create 109/38 file after the snapshot 2024-05-27T09:30:16.471 INFO:tasks.workunit.client.0.smithi114.stdout: create 109/39 file after the snapshot 2024-05-27T09:30:16.474 INFO:tasks.workunit.client.0.smithi114.stdout: create 110/.snap/snap-subdir-test 2024-05-27T09:30:16.523 INFO:tasks.workunit.client.0.smithi114.stdout: create 110/30 file after the snapshot 2024-05-27T09:30:16.527 INFO:tasks.workunit.client.0.smithi114.stdout: create 110/31 file after the snapshot 2024-05-27T09:30:16.532 INFO:tasks.workunit.client.0.smithi114.stdout: create 110/32 file after the snapshot 2024-05-27T09:30:16.536 INFO:tasks.workunit.client.0.smithi114.stdout: create 110/33 file after the snapshot 2024-05-27T09:30:16.540 INFO:tasks.workunit.client.0.smithi114.stdout: create 110/34 file after the snapshot 2024-05-27T09:30:16.544 INFO:tasks.workunit.client.0.smithi114.stdout: create 110/35 file after the snapshot 2024-05-27T09:30:16.549 INFO:tasks.workunit.client.0.smithi114.stdout: create 110/36 file after the snapshot 2024-05-27T09:30:16.553 INFO:tasks.workunit.client.0.smithi114.stdout: create 110/37 file after the snapshot 2024-05-27T09:30:16.557 INFO:tasks.workunit.client.0.smithi114.stdout: create 110/38 file after the snapshot 2024-05-27T09:30:16.562 INFO:tasks.workunit.client.0.smithi114.stdout: create 110/39 file after the snapshot 2024-05-27T09:30:16.566 INFO:tasks.workunit.client.0.smithi114.stdout: create 111/.snap/snap-subdir-test 2024-05-27T09:30:16.617 INFO:tasks.workunit.client.0.smithi114.stdout: create 111/30 file after the snapshot 2024-05-27T09:30:16.622 INFO:tasks.workunit.client.0.smithi114.stdout: create 111/31 file after the snapshot 2024-05-27T09:30:16.626 INFO:tasks.workunit.client.0.smithi114.stdout: create 111/32 file after the snapshot 2024-05-27T09:30:16.630 INFO:tasks.workunit.client.0.smithi114.stdout: create 111/33 file after the snapshot 2024-05-27T09:30:16.634 INFO:tasks.workunit.client.0.smithi114.stdout: create 111/34 file after the snapshot 2024-05-27T09:30:16.639 INFO:tasks.workunit.client.0.smithi114.stdout: create 111/35 file after the snapshot 2024-05-27T09:30:16.643 INFO:tasks.workunit.client.0.smithi114.stdout: create 111/36 file after the snapshot 2024-05-27T09:30:16.647 INFO:tasks.workunit.client.0.smithi114.stdout: create 111/37 file after the snapshot 2024-05-27T09:30:16.652 INFO:tasks.workunit.client.0.smithi114.stdout: create 111/38 file after the snapshot 2024-05-27T09:30:16.656 INFO:tasks.workunit.client.0.smithi114.stdout: create 111/39 file after the snapshot 2024-05-27T09:30:16.661 INFO:tasks.workunit.client.0.smithi114.stdout: create 112/.snap/snap-subdir-test 2024-05-27T09:30:16.686 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2024-05-27T09:30:16.686 INFO:tasks.ceph.mds.f:Restarting daemon 2024-05-27T09:30:16.687 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-05-27T09:30:16.690 INFO:tasks.ceph.mds.f:Started 2024-05-27T09:30:16.690 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:30:16.710 INFO:tasks.workunit.client.0.smithi114.stdout: create 112/30 file after the snapshot 2024-05-27T09:30:16.713 INFO:tasks.workunit.client.0.smithi114.stdout: create 112/31 file after the snapshot 2024-05-27T09:30:16.716 INFO:tasks.workunit.client.0.smithi114.stdout: create 112/32 file after the snapshot 2024-05-27T09:30:16.718 INFO:tasks.workunit.client.0.smithi114.stdout: create 112/33 file after the snapshot 2024-05-27T09:30:16.721 INFO:tasks.workunit.client.0.smithi114.stdout: create 112/34 file after the snapshot 2024-05-27T09:30:16.724 INFO:tasks.workunit.client.0.smithi114.stdout: create 112/35 file after the snapshot 2024-05-27T09:30:16.726 INFO:tasks.workunit.client.0.smithi114.stdout: create 112/36 file after the snapshot 2024-05-27T09:30:16.729 INFO:tasks.workunit.client.0.smithi114.stdout: create 112/37 file after the snapshot 2024-05-27T09:30:16.732 INFO:tasks.workunit.client.0.smithi114.stdout: create 112/38 file after the snapshot 2024-05-27T09:30:16.735 INFO:tasks.workunit.client.0.smithi114.stdout: create 112/39 file after the snapshot 2024-05-27T09:30:16.737 INFO:tasks.workunit.client.0.smithi114.stdout: create 113/.snap/snap-subdir-test 2024-05-27T09:30:16.765 INFO:tasks.workunit.client.0.smithi114.stdout: create 113/30 file after the snapshot 2024-05-27T09:30:16.766 INFO:tasks.workunit.client.0.smithi114.stdout: create 113/31 file after the snapshot 2024-05-27T09:30:16.768 INFO:tasks.workunit.client.0.smithi114.stdout: create 113/32 file after the snapshot 2024-05-27T09:30:16.769 INFO:tasks.workunit.client.0.smithi114.stdout: create 113/33 file after the snapshot 2024-05-27T09:30:16.770 INFO:tasks.workunit.client.0.smithi114.stdout: create 113/34 file after the snapshot 2024-05-27T09:30:16.772 INFO:tasks.workunit.client.0.smithi114.stdout: create 113/35 file after the snapshot 2024-05-27T09:30:16.773 INFO:tasks.workunit.client.0.smithi114.stdout: create 113/36 file after the snapshot 2024-05-27T09:30:16.774 INFO:tasks.workunit.client.0.smithi114.stdout: create 113/37 file after the snapshot 2024-05-27T09:30:16.775 INFO:tasks.workunit.client.0.smithi114.stdout: create 113/38 file after the snapshot 2024-05-27T09:30:16.776 INFO:tasks.workunit.client.0.smithi114.stdout: create 113/39 file after the snapshot 2024-05-27T09:30:16.777 INFO:tasks.workunit.client.0.smithi114.stdout: create 114/.snap/snap-subdir-test 2024-05-27T09:30:16.783 INFO:tasks.ceph.mds.f.smithi163.stdout:starting mds.f at 2024-05-27T09:30:16.806 INFO:tasks.workunit.client.0.smithi114.stdout: create 114/30 file after the snapshot 2024-05-27T09:30:16.808 INFO:tasks.workunit.client.0.smithi114.stdout: create 114/31 file after the snapshot 2024-05-27T09:30:16.809 INFO:tasks.workunit.client.0.smithi114.stdout: create 114/32 file after the snapshot 2024-05-27T09:30:16.814 INFO:tasks.workunit.client.0.smithi114.stdout: create 114/33 file after the snapshot 2024-05-27T09:30:16.816 INFO:tasks.workunit.client.0.smithi114.stdout: create 114/34 file after the snapshot 2024-05-27T09:30:16.819 INFO:tasks.workunit.client.0.smithi114.stdout: create 114/35 file after the snapshot 2024-05-27T09:30:16.821 INFO:tasks.workunit.client.0.smithi114.stdout: create 114/36 file after the snapshot 2024-05-27T09:30:16.824 INFO:tasks.workunit.client.0.smithi114.stdout: create 114/37 file after the snapshot 2024-05-27T09:30:16.826 INFO:tasks.workunit.client.0.smithi114.stdout: create 114/38 file after the snapshot 2024-05-27T09:30:16.828 INFO:tasks.workunit.client.0.smithi114.stdout: create 114/39 file after the snapshot 2024-05-27T09:30:16.831 INFO:tasks.workunit.client.0.smithi114.stdout: create 115/.snap/snap-subdir-test 2024-05-27T09:30:16.868 INFO:tasks.workunit.client.0.smithi114.stdout: create 115/30 file after the snapshot 2024-05-27T09:30:16.872 INFO:tasks.workunit.client.0.smithi114.stdout: create 115/31 file after the snapshot 2024-05-27T09:30:16.876 INFO:tasks.workunit.client.0.smithi114.stdout: create 115/32 file after the snapshot 2024-05-27T09:30:16.879 INFO:tasks.workunit.client.0.smithi114.stdout: create 115/33 file after the snapshot 2024-05-27T09:30:16.883 INFO:tasks.workunit.client.0.smithi114.stdout: create 115/34 file after the snapshot 2024-05-27T09:30:16.886 INFO:tasks.workunit.client.0.smithi114.stdout: create 115/35 file after the snapshot 2024-05-27T09:30:16.890 INFO:tasks.workunit.client.0.smithi114.stdout: create 115/36 file after the snapshot 2024-05-27T09:30:16.893 INFO:tasks.workunit.client.0.smithi114.stdout: create 115/37 file after the snapshot 2024-05-27T09:30:16.897 INFO:tasks.workunit.client.0.smithi114.stdout: create 115/38 file after the snapshot 2024-05-27T09:30:16.900 INFO:tasks.workunit.client.0.smithi114.stdout: create 115/39 file after the snapshot 2024-05-27T09:30:16.904 INFO:tasks.workunit.client.0.smithi114.stdout: create 116/.snap/snap-subdir-test 2024-05-27T09:30:16.944 INFO:tasks.workunit.client.0.smithi114.stdout: create 116/30 file after the snapshot 2024-05-27T09:30:16.946 INFO:tasks.workunit.client.0.smithi114.stdout: create 116/31 file after the snapshot 2024-05-27T09:30:16.949 INFO:tasks.workunit.client.0.smithi114.stdout: create 116/32 file after the snapshot 2024-05-27T09:30:16.952 INFO:tasks.workunit.client.0.smithi114.stdout: create 116/33 file after the snapshot 2024-05-27T09:30:16.954 INFO:tasks.workunit.client.0.smithi114.stdout: create 116/34 file after the snapshot 2024-05-27T09:30:16.956 INFO:tasks.workunit.client.0.smithi114.stdout: create 116/35 file after the snapshot 2024-05-27T09:30:16.959 INFO:tasks.workunit.client.0.smithi114.stdout: create 116/36 file after the snapshot 2024-05-27T09:30:16.961 INFO:tasks.workunit.client.0.smithi114.stdout: create 116/37 file after the snapshot 2024-05-27T09:30:16.962 INFO:tasks.workunit.client.0.smithi114.stdout: create 116/38 file after the snapshot 2024-05-27T09:30:16.964 INFO:tasks.workunit.client.0.smithi114.stdout: create 116/39 file after the snapshot 2024-05-27T09:30:16.965 INFO:tasks.workunit.client.0.smithi114.stdout: create 117/.snap/snap-subdir-test 2024-05-27T09:30:16.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.969+0000 7f881b000640 1 Processor -- start 2024-05-27T09:30:16.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.969+0000 7f881b000640 1 -- start start 2024-05-27T09:30:16.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.972+0000 7f881b000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88140c8780 0x7f881415fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:16.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.972+0000 7f881b000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f88140b75b0 0x7f88140d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:16.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.972+0000 7f881b000640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88140cd1e0 con 0x7f88140b46e0 2024-05-27T09:30:16.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.972+0000 7f881b000640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f88140cd4c0 con 0x7f88140b75b0 2024-05-27T09:30:16.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.972+0000 7f881b000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f88140c7f80 con 0x7f88140c8780 2024-05-27T09:30:16.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.972+0000 7f881a600640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f88140b46e0 0x7f88140ce310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:53620/0 (socket says 172.21.15.114:53620) 2024-05-27T09:30:16.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.972+0000 7f8819200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f88140b75b0 0x7f88140d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:16.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.972+0000 7f881a600640 1 -- 172.21.15.114:0/380115253 learned_addr learned my addr 172.21.15.114:0/380115253 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:30:16.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.972+0000 7f8819c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88140c8780 0x7f881415fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:16.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.973+0000 7f8813e00640 1 -- 172.21.15.114:0/380115253 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4179361885 0 0) 0x7f88140cd1e0 con 0x7f88140b46e0 2024-05-27T09:30:16.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.973+0000 7f8813e00640 1 -- 172.21.15.114:0/380115253 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87fc003520 con 0x7f88140b46e0 2024-05-27T09:30:16.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.973+0000 7f8819200640 1 -- 172.21.15.114:0/380115253 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88140c8780 msgr2=0x7f881415fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:30:16.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.973+0000 7f8819200640 1 --2- 172.21.15.114:0/380115253 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88140c8780 0x7f881415fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:16.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.973+0000 7f8819200640 1 -- 172.21.15.114:0/380115253 >> v1:172.21.15.163:6789/0 conn(0x7f88140b46e0 legacy=0x7f88140ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:16.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.973+0000 7f8819200640 1 -- 172.21.15.114:0/380115253 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8814160160 con 0x7f88140b75b0 2024-05-27T09:30:16.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.973+0000 7f8819c00640 1 --2- 172.21.15.114:0/380115253 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88140c8780 0x7f881415fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:30:16.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.974+0000 7f8819200640 1 --2- 172.21.15.114:0/380115253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88140b75b0 0x7f88140d1e50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f88000097a0 tx=0x7f880002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eedf76836645f16 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:16.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.974+0000 7f8813e00640 1 -- 172.21.15.114:0/380115253 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800037070 con 0x7f88140b75b0 2024-05-27T09:30:16.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.974+0000 7f8813e00640 1 -- 172.21.15.114:0/380115253 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8800032440 con 0x7f88140b75b0 2024-05-27T09:30:16.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.974+0000 7f8813e00640 1 -- 172.21.15.114:0/380115253 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800032810 con 0x7f88140b75b0 2024-05-27T09:30:16.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.975+0000 7f881b000640 1 -- 172.21.15.114:0/380115253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88140b75b0 msgr2=0x7f88140d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:16.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.975+0000 7f881b000640 1 --2- 172.21.15.114:0/380115253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88140b75b0 0x7f88140d1e50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f88000097a0 tx=0x7f880002a210 comp rx=0 tx=0).stop 2024-05-27T09:30:16.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.975+0000 7f881b000640 1 -- 172.21.15.114:0/380115253 shutdown_connections 2024-05-27T09:30:16.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.975+0000 7f881b000640 1 --2- 172.21.15.114:0/380115253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88140b75b0 0x7f88140d1e50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:16.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.975+0000 7f881b000640 1 --2- 172.21.15.114:0/380115253 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88140c8780 0x7f881415fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:16.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.975+0000 7f881b000640 1 -- 172.21.15.114:0/380115253 >> 172.21.15.114:0/380115253 conn(0x7f881406e040 msgr2=0x7f881406e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:16.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.975+0000 7f881b000640 1 -- 172.21.15.114:0/380115253 shutdown_connections 2024-05-27T09:30:16.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.975+0000 7f881b000640 1 -- 172.21.15.114:0/380115253 wait complete. 2024-05-27T09:30:16.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.978+0000 7f881b000640 1 Processor -- start 2024-05-27T09:30:16.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.978+0000 7f881b000640 1 -- start start 2024-05-27T09:30:16.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.979+0000 7f881b000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88140b46e0 0x7f8814153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:16.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.979+0000 7f881b000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88140b75b0 0x7f88141f29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:16.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.979+0000 7f881b000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88140c8780 0x7f88140b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:16.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.979+0000 7f881b000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f88140c6800 con 0x7f88140c8780 2024-05-27T09:30:16.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.979+0000 7f881b000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f88140cdf30 con 0x7f88140b75b0 2024-05-27T09:30:16.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.979+0000 7f881b000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f8814152b20 con 0x7f88140b46e0 2024-05-27T09:30:16.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.980+0000 7f8819200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88140b75b0 0x7f88141f29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:16.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.980+0000 7f8819200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88140b75b0 0x7f88141f29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:47792/0 (socket says 172.21.15.114:47792) 2024-05-27T09:30:16.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.980+0000 7f8819200640 1 -- 172.21.15.114:0/3197577497 learned_addr learned my addr 172.21.15.114:0/3197577497 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:30:16.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.980+0000 7f881a600640 1 --2- 172.21.15.114:0/3197577497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88140c8780 0x7f88140b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:16.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.980+0000 7f8819200640 1 -- 172.21.15.114:0/3197577497 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88140b46e0 msgr2=0x7f8814153b60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T09:30:16.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.980+0000 7f8819200640 1 --2- 172.21.15.114:0/3197577497 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88140b46e0 0x7f8814153b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:16.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.980+0000 7f8819200640 1 -- 172.21.15.114:0/3197577497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88140c8780 msgr2=0x7f88140b69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:16.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.980+0000 7f8819200640 1 --2- 172.21.15.114:0/3197577497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88140c8780 0x7f88140b69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:16.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.980+0000 7f8819200640 1 -- 172.21.15.114:0/3197577497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88141f2ee0 con 0x7f88140b75b0 2024-05-27T09:30:16.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.981+0000 7f8819200640 1 --2- 172.21.15.114:0/3197577497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88140b75b0 0x7f88141f29a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8800000c00 tx=0x7f88000046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:16.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.981+0000 7f8811600640 1 -- 172.21.15.114:0/3197577497 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800041020 con 0x7f88140b75b0 2024-05-27T09:30:16.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.981+0000 7f8811600640 1 -- 172.21.15.114:0/3197577497 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f880003d6e0 con 0x7f88140b75b0 2024-05-27T09:30:16.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.981+0000 7f8811600640 1 -- 172.21.15.114:0/3197577497 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800037040 con 0x7f88140b75b0 2024-05-27T09:30:16.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.981+0000 7f881b000640 1 -- 172.21.15.114:0/3197577497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88141f21c0 con 0x7f88140b75b0 2024-05-27T09:30:16.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.981+0000 7f881b000640 1 -- 172.21.15.114:0/3197577497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8814159b80 con 0x7f88140b75b0 2024-05-27T09:30:16.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.982+0000 7f881b000640 1 -- 172.21.15.114:0/3197577497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8814050860 con 0x7f88140b75b0 2024-05-27T09:30:16.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.984+0000 7f8811600640 1 -- 172.21.15.114:0/3197577497 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8800044020 con 0x7f88140b75b0 2024-05-27T09:30:16.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.985+0000 7f8811600640 1 --2- 172.21.15.114:0/3197577497 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f87f4076300 0x7f87f4078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:16.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.985+0000 7f8811600640 1 -- 172.21.15.114:0/3197577497 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5824+0+0 (secure 0 0 0) 0x7f88000b7c90 con 0x7f88140b75b0 2024-05-27T09:30:16.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.987+0000 7f8819c00640 1 --2- 172.21.15.114:0/3197577497 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f87f4076300 0x7f87f4078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:16.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.988+0000 7f8819c00640 1 --2- 172.21.15.114:0/3197577497 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f87f4076300 0x7f87f4078810 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f881408f8b0 tx=0x7f8808009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:16.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:16.989+0000 7f8811600640 1 -- 172.21.15.114:0/3197577497 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f880004b030 con 0x7f88140b75b0 2024-05-27T09:30:16.991 INFO:tasks.workunit.client.0.smithi114.stdout: create 117/30 file after the snapshot 2024-05-27T09:30:16.992 INFO:tasks.workunit.client.0.smithi114.stdout: create 117/31 file after the snapshot 2024-05-27T09:30:16.993 INFO:tasks.workunit.client.0.smithi114.stdout: create 117/32 file after the snapshot 2024-05-27T09:30:16.994 INFO:tasks.workunit.client.0.smithi114.stdout: create 117/33 file after the snapshot 2024-05-27T09:30:16.995 INFO:tasks.workunit.client.0.smithi114.stdout: create 117/34 file after the snapshot 2024-05-27T09:30:16.996 INFO:tasks.workunit.client.0.smithi114.stdout: create 117/35 file after the snapshot 2024-05-27T09:30:16.997 INFO:tasks.workunit.client.0.smithi114.stdout: create 117/36 file after the snapshot 2024-05-27T09:30:16.998 INFO:tasks.workunit.client.0.smithi114.stdout: create 117/37 file after the snapshot 2024-05-27T09:30:16.999 INFO:tasks.workunit.client.0.smithi114.stdout: create 117/38 file after the snapshot 2024-05-27T09:30:17.000 INFO:tasks.workunit.client.0.smithi114.stdout: create 117/39 file after the snapshot 2024-05-27T09:30:17.001 INFO:tasks.workunit.client.0.smithi114.stdout: create 118/.snap/snap-subdir-test 2024-05-27T09:30:17.029 INFO:tasks.workunit.client.0.smithi114.stdout: create 118/30 file after the snapshot 2024-05-27T09:30:17.031 INFO:tasks.workunit.client.0.smithi114.stdout: create 118/31 file after the snapshot 2024-05-27T09:30:17.032 INFO:tasks.workunit.client.0.smithi114.stdout: create 118/32 file after the snapshot 2024-05-27T09:30:17.033 INFO:tasks.workunit.client.0.smithi114.stdout: create 118/33 file after the snapshot 2024-05-27T09:30:17.034 INFO:tasks.workunit.client.0.smithi114.stdout: create 118/34 file after the snapshot 2024-05-27T09:30:17.035 INFO:tasks.workunit.client.0.smithi114.stdout: create 118/35 file after the snapshot 2024-05-27T09:30:17.037 INFO:tasks.workunit.client.0.smithi114.stdout: create 118/36 file after the snapshot 2024-05-27T09:30:17.038 INFO:tasks.workunit.client.0.smithi114.stdout: create 118/37 file after the snapshot 2024-05-27T09:30:17.039 INFO:tasks.workunit.client.0.smithi114.stdout: create 118/38 file after the snapshot 2024-05-27T09:30:17.040 INFO:tasks.workunit.client.0.smithi114.stdout: create 118/39 file after the snapshot 2024-05-27T09:30:17.041 INFO:tasks.workunit.client.0.smithi114.stdout: create 119/.snap/snap-subdir-test 2024-05-27T09:30:17.068 INFO:tasks.workunit.client.0.smithi114.stdout: create 119/30 file after the snapshot 2024-05-27T09:30:17.071 INFO:tasks.workunit.client.0.smithi114.stdout: create 119/31 file after the snapshot 2024-05-27T09:30:17.073 INFO:tasks.workunit.client.0.smithi114.stdout: create 119/32 file after the snapshot 2024-05-27T09:30:17.075 INFO:tasks.workunit.client.0.smithi114.stdout: create 119/33 file after the snapshot 2024-05-27T09:30:17.077 INFO:tasks.workunit.client.0.smithi114.stdout: create 119/34 file after the snapshot 2024-05-27T09:30:17.078 INFO:tasks.workunit.client.0.smithi114.stdout: create 119/35 file after the snapshot 2024-05-27T09:30:17.080 INFO:tasks.workunit.client.0.smithi114.stdout: create 119/36 file after the snapshot 2024-05-27T09:30:17.083 INFO:tasks.workunit.client.0.smithi114.stdout: create 119/37 file after the snapshot 2024-05-27T09:30:17.085 INFO:tasks.workunit.client.0.smithi114.stdout: create 119/38 file after the snapshot 2024-05-27T09:30:17.087 INFO:tasks.workunit.client.0.smithi114.stdout: create 119/39 file after the snapshot 2024-05-27T09:30:17.089 INFO:tasks.workunit.client.0.smithi114.stdout: create 120/.snap/snap-subdir-test 2024-05-27T09:30:17.125 INFO:tasks.workunit.client.0.smithi114.stdout: create 120/30 file after the snapshot 2024-05-27T09:30:17.128 INFO:tasks.workunit.client.0.smithi114.stdout: create 120/31 file after the snapshot 2024-05-27T09:30:17.130 INFO:tasks.workunit.client.0.smithi114.stdout: create 120/32 file after the snapshot 2024-05-27T09:30:17.134 INFO:tasks.workunit.client.0.smithi114.stdout: create 120/33 file after the snapshot 2024-05-27T09:30:17.136 INFO:tasks.workunit.client.0.smithi114.stdout: create 120/34 file after the snapshot 2024-05-27T09:30:17.138 INFO:tasks.workunit.client.0.smithi114.stdout: create 120/35 file after the snapshot 2024-05-27T09:30:17.140 INFO:tasks.workunit.client.0.smithi114.stdout: create 120/36 file after the snapshot 2024-05-27T09:30:17.142 INFO:tasks.workunit.client.0.smithi114.stdout: create 120/37 file after the snapshot 2024-05-27T09:30:17.144 INFO:tasks.workunit.client.0.smithi114.stdout: create 120/38 file after the snapshot 2024-05-27T09:30:17.146 INFO:tasks.workunit.client.0.smithi114.stdout: create 120/39 file after the snapshot 2024-05-27T09:30:17.148 INFO:tasks.workunit.client.0.smithi114.stdout: create 121/.snap/snap-subdir-test 2024-05-27T09:30:17.183 INFO:tasks.workunit.client.0.smithi114.stdout: create 121/30 file after the snapshot 2024-05-27T09:30:17.186 INFO:tasks.workunit.client.0.smithi114.stdout: create 121/31 file after the snapshot 2024-05-27T09:30:17.189 INFO:tasks.workunit.client.0.smithi114.stdout: create 121/32 file after the snapshot 2024-05-27T09:30:17.191 INFO:tasks.workunit.client.0.smithi114.stdout: create 121/33 file after the snapshot 2024-05-27T09:30:17.194 INFO:tasks.workunit.client.0.smithi114.stdout: create 121/34 file after the snapshot 2024-05-27T09:30:17.197 INFO:tasks.workunit.client.0.smithi114.stdout: create 121/35 file after the snapshot 2024-05-27T09:30:17.200 INFO:tasks.workunit.client.0.smithi114.stdout: create 121/36 file after the snapshot 2024-05-27T09:30:17.203 INFO:tasks.workunit.client.0.smithi114.stdout: create 121/37 file after the snapshot 2024-05-27T09:30:17.206 INFO:tasks.workunit.client.0.smithi114.stdout: create 121/38 file after the snapshot 2024-05-27T09:30:17.209 INFO:tasks.workunit.client.0.smithi114.stdout: create 121/39 file after the snapshot 2024-05-27T09:30:17.212 INFO:tasks.workunit.client.0.smithi114.stdout: create 122/.snap/snap-subdir-test 2024-05-27T09:30:17.246 INFO:tasks.workunit.client.0.smithi114.stdout: create 122/30 file after the snapshot 2024-05-27T09:30:17.249 INFO:tasks.workunit.client.0.smithi114.stdout: create 122/31 file after the snapshot 2024-05-27T09:30:17.250 INFO:tasks.workunit.client.0.smithi114.stdout: create 122/32 file after the snapshot 2024-05-27T09:30:17.252 INFO:tasks.workunit.client.0.smithi114.stdout: create 122/33 file after the snapshot 2024-05-27T09:30:17.255 INFO:tasks.workunit.client.0.smithi114.stdout: create 122/34 file after the snapshot 2024-05-27T09:30:17.257 INFO:tasks.workunit.client.0.smithi114.stdout: create 122/35 file after the snapshot 2024-05-27T09:30:17.259 INFO:tasks.workunit.client.0.smithi114.stdout: create 122/36 file after the snapshot 2024-05-27T09:30:17.262 INFO:tasks.workunit.client.0.smithi114.stdout: create 122/37 file after the snapshot 2024-05-27T09:30:17.265 INFO:tasks.workunit.client.0.smithi114.stdout: create 122/38 file after the snapshot 2024-05-27T09:30:17.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:17.266+0000 7f881b000640 1 -- 172.21.15.114:0/3197577497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88141598d0 con 0x7f88140b75b0 2024-05-27T09:30:17.268 INFO:tasks.workunit.client.0.smithi114.stdout: create 122/39 file after the snapshot 2024-05-27T09:30:17.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:17.267+0000 7f8811600640 1 -- 172.21.15.114:0/3197577497 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6883 (secure 0 0 0) 0x7f8800081510 con 0x7f88140b75b0 2024-05-27T09:30:17.268 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:30:17.269 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4828,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:30:05.919439+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"a","rank":2,"incarnation":13,"state":"up:active","state_seq":47,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4297,4256]},"id":1}]} 2024-05-27T09:30:17.269 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 18 2024-05-27T09:30:17.270 INFO:tasks.workunit.client.0.smithi114.stdout: create 123/.snap/snap-subdir-test 2024-05-27T09:30:17.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:17.270+0000 7f881b000640 1 -- 172.21.15.114:0/3197577497 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f87f4076300 msgr2=0x7f87f4078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:17.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:17.270+0000 7f881b000640 1 --2- 172.21.15.114:0/3197577497 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f87f4076300 0x7f87f4078810 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f881408f8b0 tx=0x7f8808009190 comp rx=0 tx=0).stop 2024-05-27T09:30:17.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:17.270+0000 7f881b000640 1 -- 172.21.15.114:0/3197577497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88140b75b0 msgr2=0x7f88141f29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:17.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:17.270+0000 7f881b000640 1 --2- 172.21.15.114:0/3197577497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88140b75b0 0x7f88141f29a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8800000c00 tx=0x7f88000046c0 comp rx=0 tx=0).stop 2024-05-27T09:30:17.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:17.270+0000 7f881b000640 1 -- 172.21.15.114:0/3197577497 shutdown_connections 2024-05-27T09:30:17.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:17.270+0000 7f881b000640 1 --2- 172.21.15.114:0/3197577497 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f87f4076300 0x7f87f4078810 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:17.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:17.270+0000 7f881b000640 1 --2- 172.21.15.114:0/3197577497 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f88140c8780 0x7f88140b69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:17.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:17.270+0000 7f881b000640 1 --2- 172.21.15.114:0/3197577497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88140b75b0 0x7f88141f29a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:17.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:17.270+0000 7f881b000640 1 --2- 172.21.15.114:0/3197577497 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88140b46e0 0x7f8814153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:17.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:17.270+0000 7f881b000640 1 -- 172.21.15.114:0/3197577497 >> 172.21.15.114:0/3197577497 conn(0x7f881406e040 msgr2=0x7f88140cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:17.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:17.270+0000 7f881b000640 1 -- 172.21.15.114:0/3197577497 shutdown_connections 2024-05-27T09:30:17.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:17.270+0000 7f881b000640 1 -- 172.21.15.114:0/3197577497 wait complete. 2024-05-27T09:30:17.301 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2024-05-27T09:30:17.301 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 26 secs before thrashing 2024-05-27T09:30:17.301 INFO:tasks.workunit.client.0.smithi114.stdout: create 123/30 file after the snapshot 2024-05-27T09:30:17.303 INFO:tasks.workunit.client.0.smithi114.stdout: create 123/31 file after the snapshot 2024-05-27T09:30:17.305 INFO:tasks.workunit.client.0.smithi114.stdout: create 123/32 file after the snapshot 2024-05-27T09:30:17.308 INFO:tasks.workunit.client.0.smithi114.stdout: create 123/33 file after the snapshot 2024-05-27T09:30:17.311 INFO:tasks.workunit.client.0.smithi114.stdout: create 123/34 file after the snapshot 2024-05-27T09:30:17.312 INFO:tasks.workunit.client.0.smithi114.stdout: create 123/35 file after the snapshot 2024-05-27T09:30:17.314 INFO:tasks.workunit.client.0.smithi114.stdout: create 123/36 file after the snapshot 2024-05-27T09:30:17.316 INFO:tasks.workunit.client.0.smithi114.stdout: create 123/37 file after the snapshot 2024-05-27T09:30:17.317 INFO:tasks.workunit.client.0.smithi114.stdout: create 123/38 file after the snapshot 2024-05-27T09:30:17.319 INFO:tasks.workunit.client.0.smithi114.stdout: create 123/39 file after the snapshot 2024-05-27T09:30:17.321 INFO:tasks.workunit.client.0.smithi114.stdout: create 124/.snap/snap-subdir-test 2024-05-27T09:30:17.365 INFO:tasks.workunit.client.0.smithi114.stdout: create 124/30 file after the snapshot 2024-05-27T09:30:17.366 INFO:tasks.workunit.client.0.smithi114.stdout: create 124/31 file after the snapshot 2024-05-27T09:30:17.368 INFO:tasks.workunit.client.0.smithi114.stdout: create 124/32 file after the snapshot 2024-05-27T09:30:17.371 INFO:tasks.workunit.client.0.smithi114.stdout: create 124/33 file after the snapshot 2024-05-27T09:30:17.374 INFO:tasks.workunit.client.0.smithi114.stdout: create 124/34 file after the snapshot 2024-05-27T09:30:17.376 INFO:tasks.workunit.client.0.smithi114.stdout: create 124/35 file after the snapshot 2024-05-27T09:30:17.379 INFO:tasks.workunit.client.0.smithi114.stdout: create 124/36 file after the snapshot 2024-05-27T09:30:17.382 INFO:tasks.workunit.client.0.smithi114.stdout: create 124/37 file after the snapshot 2024-05-27T09:30:17.384 INFO:tasks.workunit.client.0.smithi114.stdout: create 124/38 file after the snapshot 2024-05-27T09:30:17.387 INFO:tasks.workunit.client.0.smithi114.stdout: create 124/39 file after the snapshot 2024-05-27T09:30:17.390 INFO:tasks.workunit.client.0.smithi114.stdout: create 125/.snap/snap-subdir-test 2024-05-27T09:30:17.436 INFO:tasks.workunit.client.0.smithi114.stdout: create 125/30 file after the snapshot 2024-05-27T09:30:17.440 INFO:tasks.workunit.client.0.smithi114.stdout: create 125/31 file after the snapshot 2024-05-27T09:30:17.445 INFO:tasks.workunit.client.0.smithi114.stdout: create 125/32 file after the snapshot 2024-05-27T09:30:17.449 INFO:tasks.workunit.client.0.smithi114.stdout: create 125/33 file after the snapshot 2024-05-27T09:30:17.454 INFO:tasks.workunit.client.0.smithi114.stdout: create 125/34 file after the snapshot 2024-05-27T09:30:17.458 INFO:tasks.workunit.client.0.smithi114.stdout: create 125/35 file after the snapshot 2024-05-27T09:30:17.462 INFO:tasks.workunit.client.0.smithi114.stdout: create 125/36 file after the snapshot 2024-05-27T09:30:17.467 INFO:tasks.workunit.client.0.smithi114.stdout: create 125/37 file after the snapshot 2024-05-27T09:30:17.471 INFO:tasks.workunit.client.0.smithi114.stdout: create 125/38 file after the snapshot 2024-05-27T09:30:17.475 INFO:tasks.workunit.client.0.smithi114.stdout: create 125/39 file after the snapshot 2024-05-27T09:30:17.480 INFO:tasks.workunit.client.0.smithi114.stdout: create 126/.snap/snap-subdir-test 2024-05-27T09:30:17.524 INFO:tasks.workunit.client.0.smithi114.stdout: create 126/30 file after the snapshot 2024-05-27T09:30:17.526 INFO:tasks.workunit.client.0.smithi114.stdout: create 126/31 file after the snapshot 2024-05-27T09:30:17.528 INFO:tasks.workunit.client.0.smithi114.stdout: create 126/32 file after the snapshot 2024-05-27T09:30:17.530 INFO:tasks.workunit.client.0.smithi114.stdout: create 126/33 file after the snapshot 2024-05-27T09:30:17.532 INFO:tasks.workunit.client.0.smithi114.stdout: create 126/34 file after the snapshot 2024-05-27T09:30:17.535 INFO:tasks.workunit.client.0.smithi114.stdout: create 126/35 file after the snapshot 2024-05-27T09:30:17.538 INFO:tasks.workunit.client.0.smithi114.stdout: create 126/36 file after the snapshot 2024-05-27T09:30:17.540 INFO:tasks.workunit.client.0.smithi114.stdout: create 126/37 file after the snapshot 2024-05-27T09:30:17.543 INFO:tasks.workunit.client.0.smithi114.stdout: create 126/38 file after the snapshot 2024-05-27T09:30:17.546 INFO:tasks.workunit.client.0.smithi114.stdout: create 126/39 file after the snapshot 2024-05-27T09:30:17.549 INFO:tasks.workunit.client.0.smithi114.stdout: create 127/.snap/snap-subdir-test 2024-05-27T09:30:17.601 INFO:tasks.workunit.client.0.smithi114.stdout: create 127/30 file after the snapshot 2024-05-27T09:30:17.603 INFO:tasks.workunit.client.0.smithi114.stdout: create 127/31 file after the snapshot 2024-05-27T09:30:17.606 INFO:tasks.workunit.client.0.smithi114.stdout: create 127/32 file after the snapshot 2024-05-27T09:30:17.608 INFO:tasks.workunit.client.0.smithi114.stdout: create 127/33 file after the snapshot 2024-05-27T09:30:17.611 INFO:tasks.workunit.client.0.smithi114.stdout: create 127/34 file after the snapshot 2024-05-27T09:30:17.614 INFO:tasks.workunit.client.0.smithi114.stdout: create 127/35 file after the snapshot 2024-05-27T09:30:17.617 INFO:tasks.workunit.client.0.smithi114.stdout: create 127/36 file after the snapshot 2024-05-27T09:30:17.619 INFO:tasks.workunit.client.0.smithi114.stdout: create 127/37 file after the snapshot 2024-05-27T09:30:17.622 INFO:tasks.workunit.client.0.smithi114.stdout: create 127/38 file after the snapshot 2024-05-27T09:30:17.625 INFO:tasks.workunit.client.0.smithi114.stdout: create 127/39 file after the snapshot 2024-05-27T09:30:17.628 INFO:tasks.workunit.client.0.smithi114.stdout: create 128/.snap/snap-subdir-test 2024-05-27T09:30:17.666 INFO:tasks.workunit.client.0.smithi114.stdout: create 128/30 file after the snapshot 2024-05-27T09:30:17.669 INFO:tasks.workunit.client.0.smithi114.stdout: create 128/31 file after the snapshot 2024-05-27T09:30:17.672 INFO:tasks.workunit.client.0.smithi114.stdout: create 128/32 file after the snapshot 2024-05-27T09:30:17.675 INFO:tasks.workunit.client.0.smithi114.stdout: create 128/33 file after the snapshot 2024-05-27T09:30:17.677 INFO:tasks.workunit.client.0.smithi114.stdout: create 128/34 file after the snapshot 2024-05-27T09:30:17.680 INFO:tasks.workunit.client.0.smithi114.stdout: create 128/35 file after the snapshot 2024-05-27T09:30:17.683 INFO:tasks.workunit.client.0.smithi114.stdout: create 128/36 file after the snapshot 2024-05-27T09:30:17.686 INFO:tasks.workunit.client.0.smithi114.stdout: create 128/37 file after the snapshot 2024-05-27T09:30:17.689 INFO:tasks.workunit.client.0.smithi114.stdout: create 128/38 file after the snapshot 2024-05-27T09:30:17.691 INFO:tasks.workunit.client.0.smithi114.stdout: create 128/39 file after the snapshot 2024-05-27T09:30:17.695 INFO:tasks.workunit.client.0.smithi114.stdout: create 129/.snap/snap-subdir-test 2024-05-27T09:30:17.731 INFO:tasks.workunit.client.0.smithi114.stdout: create 129/30 file after the snapshot 2024-05-27T09:30:17.733 INFO:tasks.workunit.client.0.smithi114.stdout: create 129/31 file after the snapshot 2024-05-27T09:30:17.735 INFO:tasks.workunit.client.0.smithi114.stdout: create 129/32 file after the snapshot 2024-05-27T09:30:17.737 INFO:tasks.workunit.client.0.smithi114.stdout: create 129/33 file after the snapshot 2024-05-27T09:30:17.739 INFO:tasks.workunit.client.0.smithi114.stdout: create 129/34 file after the snapshot 2024-05-27T09:30:17.741 INFO:tasks.workunit.client.0.smithi114.stdout: create 129/35 file after the snapshot 2024-05-27T09:30:17.743 INFO:tasks.workunit.client.0.smithi114.stdout: create 129/36 file after the snapshot 2024-05-27T09:30:17.745 INFO:tasks.workunit.client.0.smithi114.stdout: create 129/37 file after the snapshot 2024-05-27T09:30:17.747 INFO:tasks.workunit.client.0.smithi114.stdout: create 129/38 file after the snapshot 2024-05-27T09:30:17.750 INFO:tasks.workunit.client.0.smithi114.stdout: create 129/39 file after the snapshot 2024-05-27T09:30:17.753 INFO:tasks.workunit.client.0.smithi114.stdout: create 130/.snap/snap-subdir-test 2024-05-27T09:30:17.794 INFO:tasks.workunit.client.0.smithi114.stdout: create 130/30 file after the snapshot 2024-05-27T09:30:17.796 INFO:tasks.workunit.client.0.smithi114.stdout: create 130/31 file after the snapshot 2024-05-27T09:30:17.799 INFO:tasks.workunit.client.0.smithi114.stdout: create 130/32 file after the snapshot 2024-05-27T09:30:17.802 INFO:tasks.workunit.client.0.smithi114.stdout: create 130/33 file after the snapshot 2024-05-27T09:30:17.805 INFO:tasks.workunit.client.0.smithi114.stdout: create 130/34 file after the snapshot 2024-05-27T09:30:17.808 INFO:tasks.workunit.client.0.smithi114.stdout: create 130/35 file after the snapshot 2024-05-27T09:30:17.811 INFO:tasks.workunit.client.0.smithi114.stdout: create 130/36 file after the snapshot 2024-05-27T09:30:17.814 INFO:tasks.workunit.client.0.smithi114.stdout: create 130/37 file after the snapshot 2024-05-27T09:30:17.817 INFO:tasks.workunit.client.0.smithi114.stdout: create 130/38 file after the snapshot 2024-05-27T09:30:17.820 INFO:tasks.workunit.client.0.smithi114.stdout: create 130/39 file after the snapshot 2024-05-27T09:30:17.823 INFO:tasks.workunit.client.0.smithi114.stdout: create 131/.snap/snap-subdir-test 2024-05-27T09:30:17.867 INFO:tasks.workunit.client.0.smithi114.stdout: create 131/30 file after the snapshot 2024-05-27T09:30:17.870 INFO:tasks.workunit.client.0.smithi114.stdout: create 131/31 file after the snapshot 2024-05-27T09:30:17.873 INFO:tasks.workunit.client.0.smithi114.stdout: create 131/32 file after the snapshot 2024-05-27T09:30:17.875 INFO:tasks.workunit.client.0.smithi114.stdout: create 131/33 file after the snapshot 2024-05-27T09:30:17.878 INFO:tasks.workunit.client.0.smithi114.stdout: create 131/34 file after the snapshot 2024-05-27T09:30:17.882 INFO:tasks.workunit.client.0.smithi114.stdout: create 131/35 file after the snapshot 2024-05-27T09:30:17.885 INFO:tasks.workunit.client.0.smithi114.stdout: create 131/36 file after the snapshot 2024-05-27T09:30:17.888 INFO:tasks.workunit.client.0.smithi114.stdout: create 131/37 file after the snapshot 2024-05-27T09:30:17.891 INFO:tasks.workunit.client.0.smithi114.stdout: create 131/38 file after the snapshot 2024-05-27T09:30:17.894 INFO:tasks.workunit.client.0.smithi114.stdout: create 131/39 file after the snapshot 2024-05-27T09:30:17.897 INFO:tasks.workunit.client.0.smithi114.stdout: create 132/.snap/snap-subdir-test 2024-05-27T09:30:17.936 INFO:tasks.workunit.client.0.smithi114.stdout: create 132/30 file after the snapshot 2024-05-27T09:30:17.938 INFO:tasks.workunit.client.0.smithi114.stdout: create 132/31 file after the snapshot 2024-05-27T09:30:17.941 INFO:tasks.workunit.client.0.smithi114.stdout: create 132/32 file after the snapshot 2024-05-27T09:30:17.944 INFO:tasks.workunit.client.0.smithi114.stdout: create 132/33 file after the snapshot 2024-05-27T09:30:17.946 INFO:tasks.workunit.client.0.smithi114.stdout: create 132/34 file after the snapshot 2024-05-27T09:30:17.949 INFO:tasks.workunit.client.0.smithi114.stdout: create 132/35 file after the snapshot 2024-05-27T09:30:17.952 INFO:tasks.workunit.client.0.smithi114.stdout: create 132/36 file after the snapshot 2024-05-27T09:30:17.954 INFO:tasks.workunit.client.0.smithi114.stdout: create 132/37 file after the snapshot 2024-05-27T09:30:17.957 INFO:tasks.workunit.client.0.smithi114.stdout: create 132/38 file after the snapshot 2024-05-27T09:30:17.960 INFO:tasks.workunit.client.0.smithi114.stdout: create 132/39 file after the snapshot 2024-05-27T09:30:17.962 INFO:tasks.workunit.client.0.smithi114.stdout: create 133/.snap/snap-subdir-test 2024-05-27T09:30:18.005 INFO:tasks.workunit.client.0.smithi114.stdout: create 133/30 file after the snapshot 2024-05-27T09:30:18.009 INFO:tasks.workunit.client.0.smithi114.stdout: create 133/31 file after the snapshot 2024-05-27T09:30:18.010 INFO:tasks.workunit.client.0.smithi114.stdout: create 133/32 file after the snapshot 2024-05-27T09:30:18.013 INFO:tasks.workunit.client.0.smithi114.stdout: create 133/33 file after the snapshot 2024-05-27T09:30:18.015 INFO:tasks.workunit.client.0.smithi114.stdout: create 133/34 file after the snapshot 2024-05-27T09:30:18.018 INFO:tasks.workunit.client.0.smithi114.stdout: create 133/35 file after the snapshot 2024-05-27T09:30:18.021 INFO:tasks.workunit.client.0.smithi114.stdout: create 133/36 file after the snapshot 2024-05-27T09:30:18.024 INFO:tasks.workunit.client.0.smithi114.stdout: create 133/37 file after the snapshot 2024-05-27T09:30:18.026 INFO:tasks.workunit.client.0.smithi114.stdout: create 133/38 file after the snapshot 2024-05-27T09:30:18.029 INFO:tasks.workunit.client.0.smithi114.stdout: create 133/39 file after the snapshot 2024-05-27T09:30:18.032 INFO:tasks.workunit.client.0.smithi114.stdout: create 134/.snap/snap-subdir-test 2024-05-27T09:30:18.082 INFO:tasks.workunit.client.0.smithi114.stdout: create 134/30 file after the snapshot 2024-05-27T09:30:18.084 INFO:tasks.workunit.client.0.smithi114.stdout: create 134/31 file after the snapshot 2024-05-27T09:30:18.087 INFO:tasks.workunit.client.0.smithi114.stdout: create 134/32 file after the snapshot 2024-05-27T09:30:18.090 INFO:tasks.workunit.client.0.smithi114.stdout: create 134/33 file after the snapshot 2024-05-27T09:30:18.093 INFO:tasks.workunit.client.0.smithi114.stdout: create 134/34 file after the snapshot 2024-05-27T09:30:18.095 INFO:tasks.workunit.client.0.smithi114.stdout: create 134/35 file after the snapshot 2024-05-27T09:30:18.098 INFO:tasks.workunit.client.0.smithi114.stdout: create 134/36 file after the snapshot 2024-05-27T09:30:18.101 INFO:tasks.workunit.client.0.smithi114.stdout: create 134/37 file after the snapshot 2024-05-27T09:30:18.104 INFO:tasks.workunit.client.0.smithi114.stdout: create 134/38 file after the snapshot 2024-05-27T09:30:18.106 INFO:tasks.workunit.client.0.smithi114.stdout: create 134/39 file after the snapshot 2024-05-27T09:30:18.109 INFO:tasks.workunit.client.0.smithi114.stdout: create 135/.snap/snap-subdir-test 2024-05-27T09:30:18.159 INFO:tasks.workunit.client.0.smithi114.stdout: create 135/30 file after the snapshot 2024-05-27T09:30:18.163 INFO:tasks.workunit.client.0.smithi114.stdout: create 135/31 file after the snapshot 2024-05-27T09:30:18.167 INFO:tasks.workunit.client.0.smithi114.stdout: create 135/32 file after the snapshot 2024-05-27T09:30:18.171 INFO:tasks.workunit.client.0.smithi114.stdout: create 135/33 file after the snapshot 2024-05-27T09:30:18.176 INFO:tasks.workunit.client.0.smithi114.stdout: create 135/34 file after the snapshot 2024-05-27T09:30:18.180 INFO:tasks.workunit.client.0.smithi114.stdout: create 135/35 file after the snapshot 2024-05-27T09:30:18.184 INFO:tasks.workunit.client.0.smithi114.stdout: create 135/36 file after the snapshot 2024-05-27T09:30:18.187 INFO:tasks.workunit.client.0.smithi114.stdout: create 135/37 file after the snapshot 2024-05-27T09:30:18.189 INFO:tasks.workunit.client.0.smithi114.stdout: create 135/38 file after the snapshot 2024-05-27T09:30:18.192 INFO:tasks.workunit.client.0.smithi114.stdout: create 135/39 file after the snapshot 2024-05-27T09:30:18.195 INFO:tasks.workunit.client.0.smithi114.stdout: create 136/.snap/snap-subdir-test 2024-05-27T09:30:18.239 INFO:tasks.workunit.client.0.smithi114.stdout: create 136/30 file after the snapshot 2024-05-27T09:30:18.240 INFO:tasks.workunit.client.0.smithi114.stdout: create 136/31 file after the snapshot 2024-05-27T09:30:18.242 INFO:tasks.workunit.client.0.smithi114.stdout: create 136/32 file after the snapshot 2024-05-27T09:30:18.244 INFO:tasks.workunit.client.0.smithi114.stdout: create 136/33 file after the snapshot 2024-05-27T09:30:18.246 INFO:tasks.workunit.client.0.smithi114.stdout: create 136/34 file after the snapshot 2024-05-27T09:30:18.249 INFO:tasks.workunit.client.0.smithi114.stdout: create 136/35 file after the snapshot 2024-05-27T09:30:18.251 INFO:tasks.workunit.client.0.smithi114.stdout: create 136/36 file after the snapshot 2024-05-27T09:30:18.254 INFO:tasks.workunit.client.0.smithi114.stdout: create 136/37 file after the snapshot 2024-05-27T09:30:18.257 INFO:tasks.workunit.client.0.smithi114.stdout: create 136/38 file after the snapshot 2024-05-27T09:30:18.260 INFO:tasks.workunit.client.0.smithi114.stdout: create 136/39 file after the snapshot 2024-05-27T09:30:18.262 INFO:tasks.workunit.client.0.smithi114.stdout: create 137/.snap/snap-subdir-test 2024-05-27T09:30:18.301 INFO:tasks.workunit.client.0.smithi114.stdout: create 137/30 file after the snapshot 2024-05-27T09:30:18.306 INFO:tasks.workunit.client.0.smithi114.stdout: create 137/31 file after the snapshot 2024-05-27T09:30:18.310 INFO:tasks.workunit.client.0.smithi114.stdout: create 137/32 file after the snapshot 2024-05-27T09:30:18.314 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:30:18.315 INFO:tasks.workunit.client.0.smithi114.stdout: create 137/33 file after the snapshot 2024-05-27T09:30:18.321 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:30:18.322 INFO:tasks.workunit.client.0.smithi114.stdout: create 137/34 file after the snapshot 2024-05-27T09:30:18.323 INFO:tasks.workunit.client.0.smithi114.stdout: create 137/35 file after the snapshot 2024-05-27T09:30:18.327 INFO:tasks.workunit.client.0.smithi114.stdout: create 137/36 file after the snapshot 2024-05-27T09:30:18.331 INFO:tasks.workunit.client.0.smithi114.stdout: create 137/37 file after the snapshot 2024-05-27T09:30:18.334 INFO:tasks.workunit.client.0.smithi114.stdout: create 137/38 file after the snapshot 2024-05-27T09:30:18.338 INFO:tasks.workunit.client.0.smithi114.stdout: create 137/39 file after the snapshot 2024-05-27T09:30:18.342 INFO:tasks.workunit.client.0.smithi114.stdout: create 138/.snap/snap-subdir-test 2024-05-27T09:30:18.372 INFO:tasks.workunit.client.0.smithi114.stdout: create 138/30 file after the snapshot 2024-05-27T09:30:18.373 INFO:tasks.workunit.client.0.smithi114.stdout: create 138/31 file after the snapshot 2024-05-27T09:30:18.374 INFO:tasks.workunit.client.0.smithi114.stdout: create 138/32 file after the snapshot 2024-05-27T09:30:18.375 INFO:tasks.workunit.client.0.smithi114.stdout: create 138/33 file after the snapshot 2024-05-27T09:30:18.376 INFO:tasks.workunit.client.0.smithi114.stdout: create 138/34 file after the snapshot 2024-05-27T09:30:18.377 INFO:tasks.workunit.client.0.smithi114.stdout: create 138/35 file after the snapshot 2024-05-27T09:30:18.379 INFO:tasks.workunit.client.0.smithi114.stdout: create 138/36 file after the snapshot 2024-05-27T09:30:18.380 INFO:tasks.workunit.client.0.smithi114.stdout: create 138/37 file after the snapshot 2024-05-27T09:30:18.381 INFO:tasks.workunit.client.0.smithi114.stdout: create 138/38 file after the snapshot 2024-05-27T09:30:18.382 INFO:tasks.workunit.client.0.smithi114.stdout: create 138/39 file after the snapshot 2024-05-27T09:30:18.383 INFO:tasks.workunit.client.0.smithi114.stdout: create 139/.snap/snap-subdir-test 2024-05-27T09:30:18.412 INFO:tasks.workunit.client.0.smithi114.stdout: create 139/30 file after the snapshot 2024-05-27T09:30:18.413 INFO:tasks.workunit.client.0.smithi114.stdout: create 139/31 file after the snapshot 2024-05-27T09:30:18.414 INFO:tasks.workunit.client.0.smithi114.stdout: create 139/32 file after the snapshot 2024-05-27T09:30:18.415 INFO:tasks.workunit.client.0.smithi114.stdout: create 139/33 file after the snapshot 2024-05-27T09:30:18.417 INFO:tasks.workunit.client.0.smithi114.stdout: create 139/34 file after the snapshot 2024-05-27T09:30:18.418 INFO:tasks.workunit.client.0.smithi114.stdout: create 139/35 file after the snapshot 2024-05-27T09:30:18.419 INFO:tasks.workunit.client.0.smithi114.stdout: create 139/36 file after the snapshot 2024-05-27T09:30:18.420 INFO:tasks.workunit.client.0.smithi114.stdout: create 139/37 file after the snapshot 2024-05-27T09:30:18.422 INFO:tasks.workunit.client.0.smithi114.stdout: create 139/38 file after the snapshot 2024-05-27T09:30:18.423 INFO:tasks.workunit.client.0.smithi114.stdout: create 139/39 file after the snapshot 2024-05-27T09:30:18.424 INFO:tasks.workunit.client.0.smithi114.stdout: create 140/.snap/snap-subdir-test 2024-05-27T09:30:18.456 INFO:tasks.workunit.client.0.smithi114.stdout: create 140/30 file after the snapshot 2024-05-27T09:30:18.458 INFO:tasks.workunit.client.0.smithi114.stdout: create 140/31 file after the snapshot 2024-05-27T09:30:18.460 INFO:tasks.workunit.client.0.smithi114.stdout: create 140/32 file after the snapshot 2024-05-27T09:30:18.463 INFO:tasks.workunit.client.0.smithi114.stdout: create 140/33 file after the snapshot 2024-05-27T09:30:18.466 INFO:tasks.workunit.client.0.smithi114.stdout: create 140/34 file after the snapshot 2024-05-27T09:30:18.469 INFO:tasks.workunit.client.0.smithi114.stdout: create 140/35 file after the snapshot 2024-05-27T09:30:18.471 INFO:tasks.workunit.client.0.smithi114.stdout: create 140/36 file after the snapshot 2024-05-27T09:30:18.475 INFO:tasks.workunit.client.0.smithi114.stdout: create 140/37 file after the snapshot 2024-05-27T09:30:18.479 INFO:tasks.workunit.client.0.smithi114.stdout: create 140/38 file after the snapshot 2024-05-27T09:30:18.483 INFO:tasks.workunit.client.0.smithi114.stdout: create 140/39 file after the snapshot 2024-05-27T09:30:18.487 INFO:tasks.workunit.client.0.smithi114.stdout: create 141/.snap/snap-subdir-test 2024-05-27T09:30:18.537 INFO:tasks.workunit.client.0.smithi114.stdout: create 141/30 file after the snapshot 2024-05-27T09:30:18.539 INFO:tasks.workunit.client.0.smithi114.stdout: create 141/31 file after the snapshot 2024-05-27T09:30:18.541 INFO:tasks.workunit.client.0.smithi114.stdout: create 141/32 file after the snapshot 2024-05-27T09:30:18.543 INFO:tasks.workunit.client.0.smithi114.stdout: create 141/33 file after the snapshot 2024-05-27T09:30:18.545 INFO:tasks.workunit.client.0.smithi114.stdout: create 141/34 file after the snapshot 2024-05-27T09:30:18.547 INFO:tasks.workunit.client.0.smithi114.stdout: create 141/35 file after the snapshot 2024-05-27T09:30:18.549 INFO:tasks.workunit.client.0.smithi114.stdout: create 141/36 file after the snapshot 2024-05-27T09:30:18.551 INFO:tasks.workunit.client.0.smithi114.stdout: create 141/37 file after the snapshot 2024-05-27T09:30:18.553 INFO:tasks.workunit.client.0.smithi114.stdout: create 141/38 file after the snapshot 2024-05-27T09:30:18.555 INFO:tasks.workunit.client.0.smithi114.stdout: create 141/39 file after the snapshot 2024-05-27T09:30:18.557 INFO:tasks.workunit.client.0.smithi114.stdout: create 142/.snap/snap-subdir-test 2024-05-27T09:30:18.607 INFO:tasks.workunit.client.0.smithi114.stdout: create 142/30 file after the snapshot 2024-05-27T09:30:18.619 INFO:tasks.workunit.client.0.smithi114.stdout: create 142/31 file after the snapshot 2024-05-27T09:30:18.621 INFO:tasks.workunit.client.0.smithi114.stdout: create 142/32 file after the snapshot 2024-05-27T09:30:18.622 INFO:tasks.workunit.client.0.smithi114.stdout: create 142/33 file after the snapshot 2024-05-27T09:30:18.625 INFO:tasks.workunit.client.0.smithi114.stdout: create 142/34 file after the snapshot 2024-05-27T09:30:18.627 INFO:tasks.workunit.client.0.smithi114.stdout: create 142/35 file after the snapshot 2024-05-27T09:30:18.630 INFO:tasks.workunit.client.0.smithi114.stdout: create 142/36 file after the snapshot 2024-05-27T09:30:18.632 INFO:tasks.workunit.client.0.smithi114.stdout: create 142/37 file after the snapshot 2024-05-27T09:30:18.635 INFO:tasks.workunit.client.0.smithi114.stdout: create 142/38 file after the snapshot 2024-05-27T09:30:18.638 INFO:tasks.workunit.client.0.smithi114.stdout: create 142/39 file after the snapshot 2024-05-27T09:30:18.641 INFO:tasks.workunit.client.0.smithi114.stdout: create 143/.snap/snap-subdir-test 2024-05-27T09:30:18.686 INFO:tasks.workunit.client.0.smithi114.stdout: create 143/30 file after the snapshot 2024-05-27T09:30:18.689 INFO:tasks.workunit.client.0.smithi114.stdout: create 143/31 file after the snapshot 2024-05-27T09:30:18.692 INFO:tasks.workunit.client.0.smithi114.stdout: create 143/32 file after the snapshot 2024-05-27T09:30:18.695 INFO:tasks.workunit.client.0.smithi114.stdout: create 143/33 file after the snapshot 2024-05-27T09:30:18.698 INFO:tasks.workunit.client.0.smithi114.stdout: create 143/34 file after the snapshot 2024-05-27T09:30:18.700 INFO:tasks.workunit.client.0.smithi114.stdout: create 143/35 file after the snapshot 2024-05-27T09:30:18.703 INFO:tasks.workunit.client.0.smithi114.stdout: create 143/36 file after the snapshot 2024-05-27T09:30:18.706 INFO:tasks.workunit.client.0.smithi114.stdout: create 143/37 file after the snapshot 2024-05-27T09:30:18.709 INFO:tasks.workunit.client.0.smithi114.stdout: create 143/38 file after the snapshot 2024-05-27T09:30:18.712 INFO:tasks.workunit.client.0.smithi114.stdout: create 143/39 file after the snapshot 2024-05-27T09:30:18.715 INFO:tasks.workunit.client.0.smithi114.stdout: create 144/.snap/snap-subdir-test 2024-05-27T09:30:18.761 INFO:tasks.workunit.client.0.smithi114.stdout: create 144/30 file after the snapshot 2024-05-27T09:30:18.764 INFO:tasks.workunit.client.0.smithi114.stdout: create 144/31 file after the snapshot 2024-05-27T09:30:18.767 INFO:tasks.workunit.client.0.smithi114.stdout: create 144/32 file after the snapshot 2024-05-27T09:30:18.770 INFO:tasks.workunit.client.0.smithi114.stdout: create 144/33 file after the snapshot 2024-05-27T09:30:18.773 INFO:tasks.workunit.client.0.smithi114.stdout: create 144/34 file after the snapshot 2024-05-27T09:30:18.776 INFO:tasks.workunit.client.0.smithi114.stdout: create 144/35 file after the snapshot 2024-05-27T09:30:18.779 INFO:tasks.workunit.client.0.smithi114.stdout: create 144/36 file after the snapshot 2024-05-27T09:30:18.782 INFO:tasks.workunit.client.0.smithi114.stdout: create 144/37 file after the snapshot 2024-05-27T09:30:18.785 INFO:tasks.workunit.client.0.smithi114.stdout: create 144/38 file after the snapshot 2024-05-27T09:30:18.788 INFO:tasks.workunit.client.0.smithi114.stdout: create 144/39 file after the snapshot 2024-05-27T09:30:18.791 INFO:tasks.workunit.client.0.smithi114.stdout: create 145/.snap/snap-subdir-test 2024-05-27T09:30:18.840 INFO:tasks.workunit.client.0.smithi114.stdout: create 145/30 file after the snapshot 2024-05-27T09:30:18.843 INFO:tasks.workunit.client.0.smithi114.stdout: create 145/31 file after the snapshot 2024-05-27T09:30:18.846 INFO:tasks.workunit.client.0.smithi114.stdout: create 145/32 file after the snapshot 2024-05-27T09:30:18.849 INFO:tasks.workunit.client.0.smithi114.stdout: create 145/33 file after the snapshot 2024-05-27T09:30:18.852 INFO:tasks.workunit.client.0.smithi114.stdout: create 145/34 file after the snapshot 2024-05-27T09:30:18.855 INFO:tasks.workunit.client.0.smithi114.stdout: create 145/35 file after the snapshot 2024-05-27T09:30:18.858 INFO:tasks.workunit.client.0.smithi114.stdout: create 145/36 file after the snapshot 2024-05-27T09:30:18.861 INFO:tasks.workunit.client.0.smithi114.stdout: create 145/37 file after the snapshot 2024-05-27T09:30:18.864 INFO:tasks.workunit.client.0.smithi114.stdout: create 145/38 file after the snapshot 2024-05-27T09:30:18.866 INFO:tasks.workunit.client.0.smithi114.stdout: create 145/39 file after the snapshot 2024-05-27T09:30:18.869 INFO:tasks.workunit.client.0.smithi114.stdout: create 146/.snap/snap-subdir-test 2024-05-27T09:30:18.915 INFO:tasks.workunit.client.0.smithi114.stdout: create 146/30 file after the snapshot 2024-05-27T09:30:18.917 INFO:tasks.workunit.client.0.smithi114.stdout: create 146/31 file after the snapshot 2024-05-27T09:30:18.920 INFO:tasks.workunit.client.0.smithi114.stdout: create 146/32 file after the snapshot 2024-05-27T09:30:18.922 INFO:tasks.workunit.client.0.smithi114.stdout: create 146/33 file after the snapshot 2024-05-27T09:30:18.925 INFO:tasks.workunit.client.0.smithi114.stdout: create 146/34 file after the snapshot 2024-05-27T09:30:18.928 INFO:tasks.workunit.client.0.smithi114.stdout: create 146/35 file after the snapshot 2024-05-27T09:30:18.931 INFO:tasks.workunit.client.0.smithi114.stdout: create 146/36 file after the snapshot 2024-05-27T09:30:18.934 INFO:tasks.workunit.client.0.smithi114.stdout: create 146/37 file after the snapshot 2024-05-27T09:30:18.937 INFO:tasks.workunit.client.0.smithi114.stdout: create 146/38 file after the snapshot 2024-05-27T09:30:18.940 INFO:tasks.workunit.client.0.smithi114.stdout: create 146/39 file after the snapshot 2024-05-27T09:30:18.942 INFO:tasks.workunit.client.0.smithi114.stdout: create 147/.snap/snap-subdir-test 2024-05-27T09:30:18.979 INFO:tasks.workunit.client.0.smithi114.stdout: create 147/30 file after the snapshot 2024-05-27T09:30:18.984 INFO:tasks.workunit.client.0.smithi114.stdout: create 147/31 file after the snapshot 2024-05-27T09:30:18.988 INFO:tasks.workunit.client.0.smithi114.stdout: create 147/32 file after the snapshot 2024-05-27T09:30:18.993 INFO:tasks.workunit.client.0.smithi114.stdout: create 147/33 file after the snapshot 2024-05-27T09:30:18.997 INFO:tasks.workunit.client.0.smithi114.stdout: create 147/34 file after the snapshot 2024-05-27T09:30:19.002 INFO:tasks.workunit.client.0.smithi114.stdout: create 147/35 file after the snapshot 2024-05-27T09:30:19.006 INFO:tasks.workunit.client.0.smithi114.stdout: create 147/36 file after the snapshot 2024-05-27T09:30:19.009 INFO:tasks.workunit.client.0.smithi114.stdout: create 147/37 file after the snapshot 2024-05-27T09:30:19.012 INFO:tasks.workunit.client.0.smithi114.stdout: create 147/38 file after the snapshot 2024-05-27T09:30:19.014 INFO:tasks.workunit.client.0.smithi114.stdout: create 147/39 file after the snapshot 2024-05-27T09:30:19.017 INFO:tasks.workunit.client.0.smithi114.stdout: create 148/.snap/snap-subdir-test 2024-05-27T09:30:19.057 INFO:tasks.workunit.client.0.smithi114.stdout: create 148/30 file after the snapshot 2024-05-27T09:30:19.060 INFO:tasks.workunit.client.0.smithi114.stdout: create 148/31 file after the snapshot 2024-05-27T09:30:19.062 INFO:tasks.workunit.client.0.smithi114.stdout: create 148/32 file after the snapshot 2024-05-27T09:30:19.065 INFO:tasks.workunit.client.0.smithi114.stdout: create 148/33 file after the snapshot 2024-05-27T09:30:19.067 INFO:tasks.workunit.client.0.smithi114.stdout: create 148/34 file after the snapshot 2024-05-27T09:30:19.070 INFO:tasks.workunit.client.0.smithi114.stdout: create 148/35 file after the snapshot 2024-05-27T09:30:19.073 INFO:tasks.workunit.client.0.smithi114.stdout: create 148/36 file after the snapshot 2024-05-27T09:30:19.075 INFO:tasks.workunit.client.0.smithi114.stdout: create 148/37 file after the snapshot 2024-05-27T09:30:19.078 INFO:tasks.workunit.client.0.smithi114.stdout: create 148/38 file after the snapshot 2024-05-27T09:30:19.081 INFO:tasks.workunit.client.0.smithi114.stdout: create 148/39 file after the snapshot 2024-05-27T09:30:19.083 INFO:tasks.workunit.client.0.smithi114.stdout: create 149/.snap/snap-subdir-test 2024-05-27T09:30:19.122 INFO:tasks.workunit.client.0.smithi114.stdout: create 149/30 file after the snapshot 2024-05-27T09:30:19.125 INFO:tasks.workunit.client.0.smithi114.stdout: create 149/31 file after the snapshot 2024-05-27T09:30:19.127 INFO:tasks.workunit.client.0.smithi114.stdout: create 149/32 file after the snapshot 2024-05-27T09:30:19.130 INFO:tasks.workunit.client.0.smithi114.stdout: create 149/33 file after the snapshot 2024-05-27T09:30:19.133 INFO:tasks.workunit.client.0.smithi114.stdout: create 149/34 file after the snapshot 2024-05-27T09:30:19.136 INFO:tasks.workunit.client.0.smithi114.stdout: create 149/35 file after the snapshot 2024-05-27T09:30:19.139 INFO:tasks.workunit.client.0.smithi114.stdout: create 149/36 file after the snapshot 2024-05-27T09:30:19.142 INFO:tasks.workunit.client.0.smithi114.stdout: create 149/37 file after the snapshot 2024-05-27T09:30:19.144 INFO:tasks.workunit.client.0.smithi114.stdout: create 149/38 file after the snapshot 2024-05-27T09:30:19.147 INFO:tasks.workunit.client.0.smithi114.stdout: create 149/39 file after the snapshot 2024-05-27T09:30:19.150 INFO:tasks.workunit.client.0.smithi114.stdout: create 150/.snap/snap-subdir-test 2024-05-27T09:30:19.200 INFO:tasks.workunit.client.0.smithi114.stdout: create 150/30 file after the snapshot 2024-05-27T09:30:19.203 INFO:tasks.workunit.client.0.smithi114.stdout: create 150/31 file after the snapshot 2024-05-27T09:30:19.206 INFO:tasks.workunit.client.0.smithi114.stdout: create 150/32 file after the snapshot 2024-05-27T09:30:19.209 INFO:tasks.workunit.client.0.smithi114.stdout: create 150/33 file after the snapshot 2024-05-27T09:30:19.212 INFO:tasks.workunit.client.0.smithi114.stdout: create 150/34 file after the snapshot 2024-05-27T09:30:19.215 INFO:tasks.workunit.client.0.smithi114.stdout: create 150/35 file after the snapshot 2024-05-27T09:30:19.218 INFO:tasks.workunit.client.0.smithi114.stdout: create 150/36 file after the snapshot 2024-05-27T09:30:19.221 INFO:tasks.workunit.client.0.smithi114.stdout: create 150/37 file after the snapshot 2024-05-27T09:30:19.224 INFO:tasks.workunit.client.0.smithi114.stdout: create 150/38 file after the snapshot 2024-05-27T09:30:19.227 INFO:tasks.workunit.client.0.smithi114.stdout: create 150/39 file after the snapshot 2024-05-27T09:30:19.229 INFO:tasks.workunit.client.0.smithi114.stdout: create 151/.snap/snap-subdir-test 2024-05-27T09:30:19.278 INFO:tasks.workunit.client.0.smithi114.stdout: create 151/30 file after the snapshot 2024-05-27T09:30:19.281 INFO:tasks.workunit.client.0.smithi114.stdout: create 151/31 file after the snapshot 2024-05-27T09:30:19.284 INFO:tasks.workunit.client.0.smithi114.stdout: create 151/32 file after the snapshot 2024-05-27T09:30:19.287 INFO:tasks.workunit.client.0.smithi114.stdout: create 151/33 file after the snapshot 2024-05-27T09:30:19.289 INFO:tasks.workunit.client.0.smithi114.stdout: create 151/34 file after the snapshot 2024-05-27T09:30:19.292 INFO:tasks.workunit.client.0.smithi114.stdout: create 151/35 file after the snapshot 2024-05-27T09:30:19.295 INFO:tasks.workunit.client.0.smithi114.stdout: create 151/36 file after the snapshot 2024-05-27T09:30:19.298 INFO:tasks.workunit.client.0.smithi114.stdout: create 151/37 file after the snapshot 2024-05-27T09:30:19.300 INFO:tasks.workunit.client.0.smithi114.stdout: create 151/38 file after the snapshot 2024-05-27T09:30:19.303 INFO:tasks.workunit.client.0.smithi114.stdout: create 151/39 file after the snapshot 2024-05-27T09:30:19.306 INFO:tasks.workunit.client.0.smithi114.stdout: create 152/.snap/snap-subdir-test 2024-05-27T09:30:19.353 INFO:tasks.workunit.client.0.smithi114.stdout: create 152/30 file after the snapshot 2024-05-27T09:30:19.356 INFO:tasks.workunit.client.0.smithi114.stdout: create 152/31 file after the snapshot 2024-05-27T09:30:19.358 INFO:tasks.workunit.client.0.smithi114.stdout: create 152/32 file after the snapshot 2024-05-27T09:30:19.360 INFO:tasks.workunit.client.0.smithi114.stdout: create 152/33 file after the snapshot 2024-05-27T09:30:19.362 INFO:tasks.workunit.client.0.smithi114.stdout: create 152/34 file after the snapshot 2024-05-27T09:30:19.364 INFO:tasks.workunit.client.0.smithi114.stdout: create 152/35 file after the snapshot 2024-05-27T09:30:19.367 INFO:tasks.workunit.client.0.smithi114.stdout: create 152/36 file after the snapshot 2024-05-27T09:30:19.370 INFO:tasks.workunit.client.0.smithi114.stdout: create 152/37 file after the snapshot 2024-05-27T09:30:19.372 INFO:tasks.workunit.client.0.smithi114.stdout: create 152/38 file after the snapshot 2024-05-27T09:30:19.375 INFO:tasks.workunit.client.0.smithi114.stdout: create 152/39 file after the snapshot 2024-05-27T09:30:19.378 INFO:tasks.workunit.client.0.smithi114.stdout: create 153/.snap/snap-subdir-test 2024-05-27T09:30:19.424 INFO:tasks.workunit.client.0.smithi114.stdout: create 153/30 file after the snapshot 2024-05-27T09:30:19.427 INFO:tasks.workunit.client.0.smithi114.stdout: create 153/31 file after the snapshot 2024-05-27T09:30:19.430 INFO:tasks.workunit.client.0.smithi114.stdout: create 153/32 file after the snapshot 2024-05-27T09:30:19.433 INFO:tasks.workunit.client.0.smithi114.stdout: create 153/33 file after the snapshot 2024-05-27T09:30:19.437 INFO:tasks.workunit.client.0.smithi114.stdout: create 153/34 file after the snapshot 2024-05-27T09:30:19.439 INFO:tasks.workunit.client.0.smithi114.stdout: create 153/35 file after the snapshot 2024-05-27T09:30:19.442 INFO:tasks.workunit.client.0.smithi114.stdout: create 153/36 file after the snapshot 2024-05-27T09:30:19.444 INFO:tasks.workunit.client.0.smithi114.stdout: create 153/37 file after the snapshot 2024-05-27T09:30:19.447 INFO:tasks.workunit.client.0.smithi114.stdout: create 153/38 file after the snapshot 2024-05-27T09:30:19.450 INFO:tasks.workunit.client.0.smithi114.stdout: create 153/39 file after the snapshot 2024-05-27T09:30:19.453 INFO:tasks.workunit.client.0.smithi114.stdout: create 154/.snap/snap-subdir-test 2024-05-27T09:30:19.503 INFO:tasks.workunit.client.0.smithi114.stdout: create 154/30 file after the snapshot 2024-05-27T09:30:19.507 INFO:tasks.workunit.client.0.smithi114.stdout: create 154/31 file after the snapshot 2024-05-27T09:30:19.512 INFO:tasks.workunit.client.0.smithi114.stdout: create 154/32 file after the snapshot 2024-05-27T09:30:19.516 INFO:tasks.workunit.client.0.smithi114.stdout: create 154/33 file after the snapshot 2024-05-27T09:30:19.521 INFO:tasks.workunit.client.0.smithi114.stdout: create 154/34 file after the snapshot 2024-05-27T09:30:19.525 INFO:tasks.workunit.client.0.smithi114.stdout: create 154/35 file after the snapshot 2024-05-27T09:30:19.529 INFO:tasks.workunit.client.0.smithi114.stdout: create 154/36 file after the snapshot 2024-05-27T09:30:19.534 INFO:tasks.workunit.client.0.smithi114.stdout: create 154/37 file after the snapshot 2024-05-27T09:30:19.538 INFO:tasks.workunit.client.0.smithi114.stdout: create 154/38 file after the snapshot 2024-05-27T09:30:19.542 INFO:tasks.workunit.client.0.smithi114.stdout: create 154/39 file after the snapshot 2024-05-27T09:30:19.546 INFO:tasks.workunit.client.0.smithi114.stdout: create 155/.snap/snap-subdir-test 2024-05-27T09:30:19.587 INFO:tasks.workunit.client.0.smithi114.stdout: create 155/30 file after the snapshot 2024-05-27T09:30:19.589 INFO:tasks.workunit.client.0.smithi114.stdout: create 155/31 file after the snapshot 2024-05-27T09:30:19.592 INFO:tasks.workunit.client.0.smithi114.stdout: create 155/32 file after the snapshot 2024-05-27T09:30:19.594 INFO:tasks.workunit.client.0.smithi114.stdout: create 155/33 file after the snapshot 2024-05-27T09:30:19.596 INFO:tasks.workunit.client.0.smithi114.stdout: create 155/34 file after the snapshot 2024-05-27T09:30:19.598 INFO:tasks.workunit.client.0.smithi114.stdout: create 155/35 file after the snapshot 2024-05-27T09:30:19.601 INFO:tasks.workunit.client.0.smithi114.stdout: create 155/36 file after the snapshot 2024-05-27T09:30:19.603 INFO:tasks.workunit.client.0.smithi114.stdout: create 155/37 file after the snapshot 2024-05-27T09:30:19.606 INFO:tasks.workunit.client.0.smithi114.stdout: create 155/38 file after the snapshot 2024-05-27T09:30:19.609 INFO:tasks.workunit.client.0.smithi114.stdout: create 155/39 file after the snapshot 2024-05-27T09:30:19.611 INFO:tasks.workunit.client.0.smithi114.stdout: create 156/.snap/snap-subdir-test 2024-05-27T09:30:19.660 INFO:tasks.workunit.client.0.smithi114.stdout: create 156/30 file after the snapshot 2024-05-27T09:30:19.663 INFO:tasks.workunit.client.0.smithi114.stdout: create 156/31 file after the snapshot 2024-05-27T09:30:19.666 INFO:tasks.workunit.client.0.smithi114.stdout: create 156/32 file after the snapshot 2024-05-27T09:30:19.669 INFO:tasks.workunit.client.0.smithi114.stdout: create 156/33 file after the snapshot 2024-05-27T09:30:19.672 INFO:tasks.workunit.client.0.smithi114.stdout: create 156/34 file after the snapshot 2024-05-27T09:30:19.675 INFO:tasks.workunit.client.0.smithi114.stdout: create 156/35 file after the snapshot 2024-05-27T09:30:19.677 INFO:tasks.workunit.client.0.smithi114.stdout: create 156/36 file after the snapshot 2024-05-27T09:30:19.680 INFO:tasks.workunit.client.0.smithi114.stdout: create 156/37 file after the snapshot 2024-05-27T09:30:19.683 INFO:tasks.workunit.client.0.smithi114.stdout: create 156/38 file after the snapshot 2024-05-27T09:30:19.686 INFO:tasks.workunit.client.0.smithi114.stdout: create 156/39 file after the snapshot 2024-05-27T09:30:19.689 INFO:tasks.workunit.client.0.smithi114.stdout: create 157/.snap/snap-subdir-test 2024-05-27T09:30:19.730 INFO:tasks.workunit.client.0.smithi114.stdout: create 157/30 file after the snapshot 2024-05-27T09:30:19.732 INFO:tasks.workunit.client.0.smithi114.stdout: create 157/31 file after the snapshot 2024-05-27T09:30:19.735 INFO:tasks.workunit.client.0.smithi114.stdout: create 157/32 file after the snapshot 2024-05-27T09:30:19.738 INFO:tasks.workunit.client.0.smithi114.stdout: create 157/33 file after the snapshot 2024-05-27T09:30:19.740 INFO:tasks.workunit.client.0.smithi114.stdout: create 157/34 file after the snapshot 2024-05-27T09:30:19.743 INFO:tasks.workunit.client.0.smithi114.stdout: create 157/35 file after the snapshot 2024-05-27T09:30:19.746 INFO:tasks.workunit.client.0.smithi114.stdout: create 157/36 file after the snapshot 2024-05-27T09:30:19.749 INFO:tasks.workunit.client.0.smithi114.stdout: create 157/37 file after the snapshot 2024-05-27T09:30:19.751 INFO:tasks.workunit.client.0.smithi114.stdout: create 157/38 file after the snapshot 2024-05-27T09:30:19.754 INFO:tasks.workunit.client.0.smithi114.stdout: create 157/39 file after the snapshot 2024-05-27T09:30:19.757 INFO:tasks.workunit.client.0.smithi114.stdout: create 158/.snap/snap-subdir-test 2024-05-27T09:30:19.810 INFO:tasks.workunit.client.0.smithi114.stdout: create 158/30 file after the snapshot 2024-05-27T09:30:19.813 INFO:tasks.workunit.client.0.smithi114.stdout: create 158/31 file after the snapshot 2024-05-27T09:30:19.815 INFO:tasks.workunit.client.0.smithi114.stdout: create 158/32 file after the snapshot 2024-05-27T09:30:19.817 INFO:tasks.workunit.client.0.smithi114.stdout: create 158/33 file after the snapshot 2024-05-27T09:30:19.819 INFO:tasks.workunit.client.0.smithi114.stdout: create 158/34 file after the snapshot 2024-05-27T09:30:19.822 INFO:tasks.workunit.client.0.smithi114.stdout: create 158/35 file after the snapshot 2024-05-27T09:30:19.825 INFO:tasks.workunit.client.0.smithi114.stdout: create 158/36 file after the snapshot 2024-05-27T09:30:19.827 INFO:tasks.workunit.client.0.smithi114.stdout: create 158/37 file after the snapshot 2024-05-27T09:30:19.830 INFO:tasks.workunit.client.0.smithi114.stdout: create 158/38 file after the snapshot 2024-05-27T09:30:19.833 INFO:tasks.workunit.client.0.smithi114.stdout: create 158/39 file after the snapshot 2024-05-27T09:30:19.836 INFO:tasks.workunit.client.0.smithi114.stdout: create 159/.snap/snap-subdir-test 2024-05-27T09:30:19.889 INFO:tasks.workunit.client.0.smithi114.stdout: create 159/30 file after the snapshot 2024-05-27T09:30:19.891 INFO:tasks.workunit.client.0.smithi114.stdout: create 159/31 file after the snapshot 2024-05-27T09:30:19.893 INFO:tasks.workunit.client.0.smithi114.stdout: create 159/32 file after the snapshot 2024-05-27T09:30:19.894 INFO:tasks.workunit.client.0.smithi114.stdout: create 159/33 file after the snapshot 2024-05-27T09:30:19.897 INFO:tasks.workunit.client.0.smithi114.stdout: create 159/34 file after the snapshot 2024-05-27T09:30:19.900 INFO:tasks.workunit.client.0.smithi114.stdout: create 159/35 file after the snapshot 2024-05-27T09:30:19.902 INFO:tasks.workunit.client.0.smithi114.stdout: create 159/36 file after the snapshot 2024-05-27T09:30:19.905 INFO:tasks.workunit.client.0.smithi114.stdout: create 159/37 file after the snapshot 2024-05-27T09:30:19.908 INFO:tasks.workunit.client.0.smithi114.stdout: create 159/38 file after the snapshot 2024-05-27T09:30:19.911 INFO:tasks.workunit.client.0.smithi114.stdout: create 159/39 file after the snapshot 2024-05-27T09:30:19.914 INFO:tasks.workunit.client.0.smithi114.stdout: create 160/.snap/snap-subdir-test 2024-05-27T09:30:19.961 INFO:tasks.workunit.client.0.smithi114.stdout: create 160/30 file after the snapshot 2024-05-27T09:30:19.963 INFO:tasks.workunit.client.0.smithi114.stdout: create 160/31 file after the snapshot 2024-05-27T09:30:19.965 INFO:tasks.workunit.client.0.smithi114.stdout: create 160/32 file after the snapshot 2024-05-27T09:30:19.968 INFO:tasks.workunit.client.0.smithi114.stdout: create 160/33 file after the snapshot 2024-05-27T09:30:19.970 INFO:tasks.workunit.client.0.smithi114.stdout: create 160/34 file after the snapshot 2024-05-27T09:30:19.973 INFO:tasks.workunit.client.0.smithi114.stdout: create 160/35 file after the snapshot 2024-05-27T09:30:19.976 INFO:tasks.workunit.client.0.smithi114.stdout: create 160/36 file after the snapshot 2024-05-27T09:30:19.979 INFO:tasks.workunit.client.0.smithi114.stdout: create 160/37 file after the snapshot 2024-05-27T09:30:19.981 INFO:tasks.workunit.client.0.smithi114.stdout: create 160/38 file after the snapshot 2024-05-27T09:30:19.984 INFO:tasks.workunit.client.0.smithi114.stdout: create 160/39 file after the snapshot 2024-05-27T09:30:19.987 INFO:tasks.workunit.client.0.smithi114.stdout: create 161/.snap/snap-subdir-test 2024-05-27T09:30:20.030 INFO:tasks.workunit.client.0.smithi114.stdout: create 161/30 file after the snapshot 2024-05-27T09:30:20.032 INFO:tasks.workunit.client.0.smithi114.stdout: create 161/31 file after the snapshot 2024-05-27T09:30:20.035 INFO:tasks.workunit.client.0.smithi114.stdout: create 161/32 file after the snapshot 2024-05-27T09:30:20.037 INFO:tasks.workunit.client.0.smithi114.stdout: create 161/33 file after the snapshot 2024-05-27T09:30:20.040 INFO:tasks.workunit.client.0.smithi114.stdout: create 161/34 file after the snapshot 2024-05-27T09:30:20.043 INFO:tasks.workunit.client.0.smithi114.stdout: create 161/35 file after the snapshot 2024-05-27T09:30:20.046 INFO:tasks.workunit.client.0.smithi114.stdout: create 161/36 file after the snapshot 2024-05-27T09:30:20.049 INFO:tasks.workunit.client.0.smithi114.stdout: create 161/37 file after the snapshot 2024-05-27T09:30:20.052 INFO:tasks.workunit.client.0.smithi114.stdout: create 161/38 file after the snapshot 2024-05-27T09:30:20.054 INFO:tasks.workunit.client.0.smithi114.stdout: create 161/39 file after the snapshot 2024-05-27T09:30:20.057 INFO:tasks.workunit.client.0.smithi114.stdout: create 162/.snap/snap-subdir-test 2024-05-27T09:30:20.095 INFO:tasks.workunit.client.0.smithi114.stdout: create 162/30 file after the snapshot 2024-05-27T09:30:20.097 INFO:tasks.workunit.client.0.smithi114.stdout: create 162/31 file after the snapshot 2024-05-27T09:30:20.100 INFO:tasks.workunit.client.0.smithi114.stdout: create 162/32 file after the snapshot 2024-05-27T09:30:20.103 INFO:tasks.workunit.client.0.smithi114.stdout: create 162/33 file after the snapshot 2024-05-27T09:30:20.106 INFO:tasks.workunit.client.0.smithi114.stdout: create 162/34 file after the snapshot 2024-05-27T09:30:20.109 INFO:tasks.workunit.client.0.smithi114.stdout: create 162/35 file after the snapshot 2024-05-27T09:30:20.111 INFO:tasks.workunit.client.0.smithi114.stdout: create 162/36 file after the snapshot 2024-05-27T09:30:20.114 INFO:tasks.workunit.client.0.smithi114.stdout: create 162/37 file after the snapshot 2024-05-27T09:30:20.117 INFO:tasks.workunit.client.0.smithi114.stdout: create 162/38 file after the snapshot 2024-05-27T09:30:20.120 INFO:tasks.workunit.client.0.smithi114.stdout: create 162/39 file after the snapshot 2024-05-27T09:30:20.123 INFO:tasks.workunit.client.0.smithi114.stdout: create 163/.snap/snap-subdir-test 2024-05-27T09:30:20.171 INFO:tasks.workunit.client.0.smithi114.stdout: create 163/30 file after the snapshot 2024-05-27T09:30:20.174 INFO:tasks.workunit.client.0.smithi114.stdout: create 163/31 file after the snapshot 2024-05-27T09:30:20.177 INFO:tasks.workunit.client.0.smithi114.stdout: create 163/32 file after the snapshot 2024-05-27T09:30:20.180 INFO:tasks.workunit.client.0.smithi114.stdout: create 163/33 file after the snapshot 2024-05-27T09:30:20.183 INFO:tasks.workunit.client.0.smithi114.stdout: create 163/34 file after the snapshot 2024-05-27T09:30:20.185 INFO:tasks.workunit.client.0.smithi114.stdout: create 163/35 file after the snapshot 2024-05-27T09:30:20.188 INFO:tasks.workunit.client.0.smithi114.stdout: create 163/36 file after the snapshot 2024-05-27T09:30:20.191 INFO:tasks.workunit.client.0.smithi114.stdout: create 163/37 file after the snapshot 2024-05-27T09:30:20.194 INFO:tasks.workunit.client.0.smithi114.stdout: create 163/38 file after the snapshot 2024-05-27T09:30:20.197 INFO:tasks.workunit.client.0.smithi114.stdout: create 163/39 file after the snapshot 2024-05-27T09:30:20.200 INFO:tasks.workunit.client.0.smithi114.stdout: create 164/.snap/snap-subdir-test 2024-05-27T09:30:20.243 INFO:tasks.workunit.client.0.smithi114.stdout: create 164/30 file after the snapshot 2024-05-27T09:30:20.247 INFO:tasks.workunit.client.0.smithi114.stdout: create 164/31 file after the snapshot 2024-05-27T09:30:20.253 INFO:tasks.workunit.client.0.smithi114.stdout: create 164/32 file after the snapshot 2024-05-27T09:30:20.255 INFO:tasks.workunit.client.0.smithi114.stdout: create 164/33 file after the snapshot 2024-05-27T09:30:20.259 INFO:tasks.workunit.client.0.smithi114.stdout: create 164/34 file after the snapshot 2024-05-27T09:30:20.263 INFO:tasks.workunit.client.0.smithi114.stdout: create 164/35 file after the snapshot 2024-05-27T09:30:20.267 INFO:tasks.workunit.client.0.smithi114.stdout: create 164/36 file after the snapshot 2024-05-27T09:30:20.271 INFO:tasks.workunit.client.0.smithi114.stdout: create 164/37 file after the snapshot 2024-05-27T09:30:20.276 INFO:tasks.workunit.client.0.smithi114.stdout: create 164/38 file after the snapshot 2024-05-27T09:30:20.280 INFO:tasks.workunit.client.0.smithi114.stdout: create 164/39 file after the snapshot 2024-05-27T09:30:20.284 INFO:tasks.workunit.client.0.smithi114.stdout: create 165/.snap/snap-subdir-test 2024-05-27T09:30:20.334 INFO:tasks.workunit.client.0.smithi114.stdout: create 165/30 file after the snapshot 2024-05-27T09:30:20.339 INFO:tasks.workunit.client.0.smithi114.stdout: create 165/31 file after the snapshot 2024-05-27T09:30:20.343 INFO:tasks.workunit.client.0.smithi114.stdout: create 165/32 file after the snapshot 2024-05-27T09:30:20.348 INFO:tasks.workunit.client.0.smithi114.stdout: create 165/33 file after the snapshot 2024-05-27T09:30:20.352 INFO:tasks.workunit.client.0.smithi114.stdout: create 165/34 file after the snapshot 2024-05-27T09:30:20.356 INFO:tasks.workunit.client.0.smithi114.stdout: create 165/35 file after the snapshot 2024-05-27T09:30:20.360 INFO:tasks.workunit.client.0.smithi114.stdout: create 165/36 file after the snapshot 2024-05-27T09:30:20.365 INFO:tasks.workunit.client.0.smithi114.stdout: create 165/37 file after the snapshot 2024-05-27T09:30:20.369 INFO:tasks.workunit.client.0.smithi114.stdout: create 165/38 file after the snapshot 2024-05-27T09:30:20.373 INFO:tasks.workunit.client.0.smithi114.stdout: create 165/39 file after the snapshot 2024-05-27T09:30:20.377 INFO:tasks.workunit.client.0.smithi114.stdout: create 166/.snap/snap-subdir-test 2024-05-27T09:30:20.425 INFO:tasks.workunit.client.0.smithi114.stdout: create 166/30 file after the snapshot 2024-05-27T09:30:20.428 INFO:tasks.workunit.client.0.smithi114.stdout: create 166/31 file after the snapshot 2024-05-27T09:30:20.430 INFO:tasks.workunit.client.0.smithi114.stdout: create 166/32 file after the snapshot 2024-05-27T09:30:20.433 INFO:tasks.workunit.client.0.smithi114.stdout: create 166/33 file after the snapshot 2024-05-27T09:30:20.436 INFO:tasks.workunit.client.0.smithi114.stdout: create 166/34 file after the snapshot 2024-05-27T09:30:20.439 INFO:tasks.workunit.client.0.smithi114.stdout: create 166/35 file after the snapshot 2024-05-27T09:30:20.442 INFO:tasks.workunit.client.0.smithi114.stdout: create 166/36 file after the snapshot 2024-05-27T09:30:20.444 INFO:tasks.workunit.client.0.smithi114.stdout: create 166/37 file after the snapshot 2024-05-27T09:30:20.447 INFO:tasks.workunit.client.0.smithi114.stdout: create 166/38 file after the snapshot 2024-05-27T09:30:20.450 INFO:tasks.workunit.client.0.smithi114.stdout: create 166/39 file after the snapshot 2024-05-27T09:30:20.453 INFO:tasks.workunit.client.0.smithi114.stdout: create 167/.snap/snap-subdir-test 2024-05-27T09:30:20.495 INFO:tasks.workunit.client.0.smithi114.stdout: create 167/30 file after the snapshot 2024-05-27T09:30:20.497 INFO:tasks.workunit.client.0.smithi114.stdout: create 167/31 file after the snapshot 2024-05-27T09:30:20.499 INFO:tasks.workunit.client.0.smithi114.stdout: create 167/32 file after the snapshot 2024-05-27T09:30:20.502 INFO:tasks.workunit.client.0.smithi114.stdout: create 167/33 file after the snapshot 2024-05-27T09:30:20.505 INFO:tasks.workunit.client.0.smithi114.stdout: create 167/34 file after the snapshot 2024-05-27T09:30:20.508 INFO:tasks.workunit.client.0.smithi114.stdout: create 167/35 file after the snapshot 2024-05-27T09:30:20.510 INFO:tasks.workunit.client.0.smithi114.stdout: create 167/36 file after the snapshot 2024-05-27T09:30:20.513 INFO:tasks.workunit.client.0.smithi114.stdout: create 167/37 file after the snapshot 2024-05-27T09:30:20.516 INFO:tasks.workunit.client.0.smithi114.stdout: create 167/38 file after the snapshot 2024-05-27T09:30:20.519 INFO:tasks.workunit.client.0.smithi114.stdout: create 167/39 file after the snapshot 2024-05-27T09:30:20.522 INFO:tasks.workunit.client.0.smithi114.stdout: create 168/.snap/snap-subdir-test 2024-05-27T09:30:20.567 INFO:tasks.workunit.client.0.smithi114.stdout: create 168/30 file after the snapshot 2024-05-27T09:30:20.569 INFO:tasks.workunit.client.0.smithi114.stdout: create 168/31 file after the snapshot 2024-05-27T09:30:20.572 INFO:tasks.workunit.client.0.smithi114.stdout: create 168/32 file after the snapshot 2024-05-27T09:30:20.575 INFO:tasks.workunit.client.0.smithi114.stdout: create 168/33 file after the snapshot 2024-05-27T09:30:20.577 INFO:tasks.workunit.client.0.smithi114.stdout: create 168/34 file after the snapshot 2024-05-27T09:30:20.580 INFO:tasks.workunit.client.0.smithi114.stdout: create 168/35 file after the snapshot 2024-05-27T09:30:20.582 INFO:tasks.workunit.client.0.smithi114.stdout: create 168/36 file after the snapshot 2024-05-27T09:30:20.585 INFO:tasks.workunit.client.0.smithi114.stdout: create 168/37 file after the snapshot 2024-05-27T09:30:20.588 INFO:tasks.workunit.client.0.smithi114.stdout: create 168/38 file after the snapshot 2024-05-27T09:30:20.591 INFO:tasks.workunit.client.0.smithi114.stdout: create 168/39 file after the snapshot 2024-05-27T09:30:20.594 INFO:tasks.workunit.client.0.smithi114.stdout: create 169/.snap/snap-subdir-test 2024-05-27T09:30:20.640 INFO:tasks.workunit.client.0.smithi114.stdout: create 169/30 file after the snapshot 2024-05-27T09:30:20.642 INFO:tasks.workunit.client.0.smithi114.stdout: create 169/31 file after the snapshot 2024-05-27T09:30:20.645 INFO:tasks.workunit.client.0.smithi114.stdout: create 169/32 file after the snapshot 2024-05-27T09:30:20.648 INFO:tasks.workunit.client.0.smithi114.stdout: create 169/33 file after the snapshot 2024-05-27T09:30:20.650 INFO:tasks.workunit.client.0.smithi114.stdout: create 169/34 file after the snapshot 2024-05-27T09:30:20.653 INFO:tasks.workunit.client.0.smithi114.stdout: create 169/35 file after the snapshot 2024-05-27T09:30:20.656 INFO:tasks.workunit.client.0.smithi114.stdout: create 169/36 file after the snapshot 2024-05-27T09:30:20.659 INFO:tasks.workunit.client.0.smithi114.stdout: create 169/37 file after the snapshot 2024-05-27T09:30:20.662 INFO:tasks.workunit.client.0.smithi114.stdout: create 169/38 file after the snapshot 2024-05-27T09:30:20.670 INFO:tasks.workunit.client.0.smithi114.stdout: create 169/39 file after the snapshot 2024-05-27T09:30:20.670 INFO:tasks.workunit.client.0.smithi114.stdout: create 170/.snap/snap-subdir-test 2024-05-27T09:30:20.717 INFO:tasks.workunit.client.0.smithi114.stdout: create 170/30 file after the snapshot 2024-05-27T09:30:20.720 INFO:tasks.workunit.client.0.smithi114.stdout: create 170/31 file after the snapshot 2024-05-27T09:30:20.722 INFO:tasks.workunit.client.0.smithi114.stdout: create 170/32 file after the snapshot 2024-05-27T09:30:20.724 INFO:tasks.workunit.client.0.smithi114.stdout: create 170/33 file after the snapshot 2024-05-27T09:30:20.726 INFO:tasks.workunit.client.0.smithi114.stdout: create 170/34 file after the snapshot 2024-05-27T09:30:20.728 INFO:tasks.workunit.client.0.smithi114.stdout: create 170/35 file after the snapshot 2024-05-27T09:30:20.730 INFO:tasks.workunit.client.0.smithi114.stdout: create 170/36 file after the snapshot 2024-05-27T09:30:20.733 INFO:tasks.workunit.client.0.smithi114.stdout: create 170/37 file after the snapshot 2024-05-27T09:30:20.736 INFO:tasks.workunit.client.0.smithi114.stdout: create 170/38 file after the snapshot 2024-05-27T09:30:20.739 INFO:tasks.workunit.client.0.smithi114.stdout: create 170/39 file after the snapshot 2024-05-27T09:30:20.742 INFO:tasks.workunit.client.0.smithi114.stdout: create 171/.snap/snap-subdir-test 2024-05-27T09:30:20.788 INFO:tasks.workunit.client.0.smithi114.stdout: create 171/30 file after the snapshot 2024-05-27T09:30:20.790 INFO:tasks.workunit.client.0.smithi114.stdout: create 171/31 file after the snapshot 2024-05-27T09:30:20.792 INFO:tasks.workunit.client.0.smithi114.stdout: create 171/32 file after the snapshot 2024-05-27T09:30:20.794 INFO:tasks.workunit.client.0.smithi114.stdout: create 171/33 file after the snapshot 2024-05-27T09:30:20.796 INFO:tasks.workunit.client.0.smithi114.stdout: create 171/34 file after the snapshot 2024-05-27T09:30:20.798 INFO:tasks.workunit.client.0.smithi114.stdout: create 171/35 file after the snapshot 2024-05-27T09:30:20.800 INFO:tasks.workunit.client.0.smithi114.stdout: create 171/36 file after the snapshot 2024-05-27T09:30:20.802 INFO:tasks.workunit.client.0.smithi114.stdout: create 171/37 file after the snapshot 2024-05-27T09:30:20.804 INFO:tasks.workunit.client.0.smithi114.stdout: create 171/38 file after the snapshot 2024-05-27T09:30:20.806 INFO:tasks.workunit.client.0.smithi114.stdout: create 171/39 file after the snapshot 2024-05-27T09:30:20.808 INFO:tasks.workunit.client.0.smithi114.stdout: create 172/.snap/snap-subdir-test 2024-05-27T09:30:20.857 INFO:tasks.workunit.client.0.smithi114.stdout: create 172/30 file after the snapshot 2024-05-27T09:30:20.858 INFO:tasks.workunit.client.0.smithi114.stdout: create 172/31 file after the snapshot 2024-05-27T09:30:20.860 INFO:tasks.workunit.client.0.smithi114.stdout: create 172/32 file after the snapshot 2024-05-27T09:30:20.861 INFO:tasks.workunit.client.0.smithi114.stdout: create 172/33 file after the snapshot 2024-05-27T09:30:20.863 INFO:tasks.workunit.client.0.smithi114.stdout: create 172/34 file after the snapshot 2024-05-27T09:30:20.866 INFO:tasks.workunit.client.0.smithi114.stdout: create 172/35 file after the snapshot 2024-05-27T09:30:20.868 INFO:tasks.workunit.client.0.smithi114.stdout: create 172/36 file after the snapshot 2024-05-27T09:30:20.871 INFO:tasks.workunit.client.0.smithi114.stdout: create 172/37 file after the snapshot 2024-05-27T09:30:20.873 INFO:tasks.workunit.client.0.smithi114.stdout: create 172/38 file after the snapshot 2024-05-27T09:30:20.876 INFO:tasks.workunit.client.0.smithi114.stdout: create 172/39 file after the snapshot 2024-05-27T09:30:20.879 INFO:tasks.workunit.client.0.smithi114.stdout: create 173/.snap/snap-subdir-test 2024-05-27T09:30:20.931 INFO:tasks.workunit.client.0.smithi114.stdout: create 173/30 file after the snapshot 2024-05-27T09:30:20.934 INFO:tasks.workunit.client.0.smithi114.stdout: create 173/31 file after the snapshot 2024-05-27T09:30:20.937 INFO:tasks.workunit.client.0.smithi114.stdout: create 173/32 file after the snapshot 2024-05-27T09:30:20.939 INFO:tasks.workunit.client.0.smithi114.stdout: create 173/33 file after the snapshot 2024-05-27T09:30:20.942 INFO:tasks.workunit.client.0.smithi114.stdout: create 173/34 file after the snapshot 2024-05-27T09:30:20.945 INFO:tasks.workunit.client.0.smithi114.stdout: create 173/35 file after the snapshot 2024-05-27T09:30:20.949 INFO:tasks.workunit.client.0.smithi114.stdout: create 173/36 file after the snapshot 2024-05-27T09:30:20.951 INFO:tasks.workunit.client.0.smithi114.stdout: create 173/37 file after the snapshot 2024-05-27T09:30:20.954 INFO:tasks.workunit.client.0.smithi114.stdout: create 173/38 file after the snapshot 2024-05-27T09:30:20.957 INFO:tasks.workunit.client.0.smithi114.stdout: create 173/39 file after the snapshot 2024-05-27T09:30:20.960 INFO:tasks.workunit.client.0.smithi114.stdout: create 174/.snap/snap-subdir-test 2024-05-27T09:30:21.024 INFO:tasks.workunit.client.0.smithi114.stdout: create 174/30 file after the snapshot 2024-05-27T09:30:21.025 INFO:tasks.workunit.client.0.smithi114.stdout: create 174/31 file after the snapshot 2024-05-27T09:30:21.027 INFO:tasks.workunit.client.0.smithi114.stdout: create 174/32 file after the snapshot 2024-05-27T09:30:21.029 INFO:tasks.workunit.client.0.smithi114.stdout: create 174/33 file after the snapshot 2024-05-27T09:30:21.031 INFO:tasks.workunit.client.0.smithi114.stdout: create 174/34 file after the snapshot 2024-05-27T09:30:21.032 INFO:tasks.workunit.client.0.smithi114.stdout: create 174/35 file after the snapshot 2024-05-27T09:30:21.034 INFO:tasks.workunit.client.0.smithi114.stdout: create 174/36 file after the snapshot 2024-05-27T09:30:21.036 INFO:tasks.workunit.client.0.smithi114.stdout: create 174/37 file after the snapshot 2024-05-27T09:30:21.038 INFO:tasks.workunit.client.0.smithi114.stdout: create 174/38 file after the snapshot 2024-05-27T09:30:21.041 INFO:tasks.workunit.client.0.smithi114.stdout: create 174/39 file after the snapshot 2024-05-27T09:30:21.044 INFO:tasks.workunit.client.0.smithi114.stdout: create 175/.snap/snap-subdir-test 2024-05-27T09:30:21.104 INFO:tasks.workunit.client.0.smithi114.stdout: create 175/30 file after the snapshot 2024-05-27T09:30:21.106 INFO:tasks.workunit.client.0.smithi114.stdout: create 175/31 file after the snapshot 2024-05-27T09:30:21.107 INFO:tasks.workunit.client.0.smithi114.stdout: create 175/32 file after the snapshot 2024-05-27T09:30:21.109 INFO:tasks.workunit.client.0.smithi114.stdout: create 175/33 file after the snapshot 2024-05-27T09:30:21.110 INFO:tasks.workunit.client.0.smithi114.stdout: create 175/34 file after the snapshot 2024-05-27T09:30:21.112 INFO:tasks.workunit.client.0.smithi114.stdout: create 175/35 file after the snapshot 2024-05-27T09:30:21.114 INFO:tasks.workunit.client.0.smithi114.stdout: create 175/36 file after the snapshot 2024-05-27T09:30:21.115 INFO:tasks.workunit.client.0.smithi114.stdout: create 175/37 file after the snapshot 2024-05-27T09:30:21.117 INFO:tasks.workunit.client.0.smithi114.stdout: create 175/38 file after the snapshot 2024-05-27T09:30:21.119 INFO:tasks.workunit.client.0.smithi114.stdout: create 175/39 file after the snapshot 2024-05-27T09:30:21.120 INFO:tasks.workunit.client.0.smithi114.stdout: create 176/.snap/snap-subdir-test 2024-05-27T09:30:21.178 INFO:tasks.workunit.client.0.smithi114.stdout: create 176/30 file after the snapshot 2024-05-27T09:30:21.181 INFO:tasks.workunit.client.0.smithi114.stdout: create 176/31 file after the snapshot 2024-05-27T09:30:21.184 INFO:tasks.workunit.client.0.smithi114.stdout: create 176/32 file after the snapshot 2024-05-27T09:30:21.189 INFO:tasks.workunit.client.0.smithi114.stdout: create 176/33 file after the snapshot 2024-05-27T09:30:21.193 INFO:tasks.workunit.client.0.smithi114.stdout: create 176/34 file after the snapshot 2024-05-27T09:30:21.198 INFO:tasks.workunit.client.0.smithi114.stdout: create 176/35 file after the snapshot 2024-05-27T09:30:21.202 INFO:tasks.workunit.client.0.smithi114.stdout: create 176/36 file after the snapshot 2024-05-27T09:30:21.206 INFO:tasks.workunit.client.0.smithi114.stdout: create 176/37 file after the snapshot 2024-05-27T09:30:21.211 INFO:tasks.workunit.client.0.smithi114.stdout: create 176/38 file after the snapshot 2024-05-27T09:30:21.215 INFO:tasks.workunit.client.0.smithi114.stdout: create 176/39 file after the snapshot 2024-05-27T09:30:21.220 INFO:tasks.workunit.client.0.smithi114.stdout: create 177/.snap/snap-subdir-test 2024-05-27T09:30:21.277 INFO:tasks.workunit.client.0.smithi114.stdout: create 177/30 file after the snapshot 2024-05-27T09:30:21.280 INFO:tasks.workunit.client.0.smithi114.stdout: create 177/31 file after the snapshot 2024-05-27T09:30:21.283 INFO:tasks.workunit.client.0.smithi114.stdout: create 177/32 file after the snapshot 2024-05-27T09:30:21.286 INFO:tasks.workunit.client.0.smithi114.stdout: create 177/33 file after the snapshot 2024-05-27T09:30:21.289 INFO:tasks.workunit.client.0.smithi114.stdout: create 177/34 file after the snapshot 2024-05-27T09:30:21.292 INFO:tasks.workunit.client.0.smithi114.stdout: create 177/35 file after the snapshot 2024-05-27T09:30:21.295 INFO:tasks.workunit.client.0.smithi114.stdout: create 177/36 file after the snapshot 2024-05-27T09:30:21.297 INFO:tasks.workunit.client.0.smithi114.stdout: create 177/37 file after the snapshot 2024-05-27T09:30:21.300 INFO:tasks.workunit.client.0.smithi114.stdout: create 177/38 file after the snapshot 2024-05-27T09:30:21.303 INFO:tasks.workunit.client.0.smithi114.stdout: create 177/39 file after the snapshot 2024-05-27T09:30:21.306 INFO:tasks.workunit.client.0.smithi114.stdout: create 178/.snap/snap-subdir-test 2024-05-27T09:30:21.358 INFO:tasks.workunit.client.0.smithi114.stdout: create 178/30 file after the snapshot 2024-05-27T09:30:21.360 INFO:tasks.workunit.client.0.smithi114.stdout: create 178/31 file after the snapshot 2024-05-27T09:30:21.362 INFO:tasks.workunit.client.0.smithi114.stdout: create 178/32 file after the snapshot 2024-05-27T09:30:21.364 INFO:tasks.workunit.client.0.smithi114.stdout: create 178/33 file after the snapshot 2024-05-27T09:30:21.366 INFO:tasks.workunit.client.0.smithi114.stdout: create 178/34 file after the snapshot 2024-05-27T09:30:21.368 INFO:tasks.workunit.client.0.smithi114.stdout: create 178/35 file after the snapshot 2024-05-27T09:30:21.370 INFO:tasks.workunit.client.0.smithi114.stdout: create 178/36 file after the snapshot 2024-05-27T09:30:21.372 INFO:tasks.workunit.client.0.smithi114.stdout: create 178/37 file after the snapshot 2024-05-27T09:30:21.374 INFO:tasks.workunit.client.0.smithi114.stdout: create 178/38 file after the snapshot 2024-05-27T09:30:21.376 INFO:tasks.workunit.client.0.smithi114.stdout: create 178/39 file after the snapshot 2024-05-27T09:30:21.379 INFO:tasks.workunit.client.0.smithi114.stdout: create 179/.snap/snap-subdir-test 2024-05-27T09:30:21.431 INFO:tasks.workunit.client.0.smithi114.stdout: create 179/30 file after the snapshot 2024-05-27T09:30:21.434 INFO:tasks.workunit.client.0.smithi114.stdout: create 179/31 file after the snapshot 2024-05-27T09:30:21.437 INFO:tasks.workunit.client.0.smithi114.stdout: create 179/32 file after the snapshot 2024-05-27T09:30:21.440 INFO:tasks.workunit.client.0.smithi114.stdout: create 179/33 file after the snapshot 2024-05-27T09:30:21.443 INFO:tasks.workunit.client.0.smithi114.stdout: create 179/34 file after the snapshot 2024-05-27T09:30:21.446 INFO:tasks.workunit.client.0.smithi114.stdout: create 179/35 file after the snapshot 2024-05-27T09:30:21.449 INFO:tasks.workunit.client.0.smithi114.stdout: create 179/36 file after the snapshot 2024-05-27T09:30:21.452 INFO:tasks.workunit.client.0.smithi114.stdout: create 179/37 file after the snapshot 2024-05-27T09:30:21.455 INFO:tasks.workunit.client.0.smithi114.stdout: create 179/38 file after the snapshot 2024-05-27T09:30:21.457 INFO:tasks.workunit.client.0.smithi114.stdout: create 179/39 file after the snapshot 2024-05-27T09:30:21.460 INFO:tasks.workunit.client.0.smithi114.stdout: create 180/.snap/snap-subdir-test 2024-05-27T09:30:21.514 INFO:tasks.workunit.client.0.smithi114.stdout: create 180/30 file after the snapshot 2024-05-27T09:30:21.518 INFO:tasks.workunit.client.0.smithi114.stdout: create 180/31 file after the snapshot 2024-05-27T09:30:21.523 INFO:tasks.workunit.client.0.smithi114.stdout: create 180/32 file after the snapshot 2024-05-27T09:30:21.527 INFO:tasks.workunit.client.0.smithi114.stdout: create 180/33 file after the snapshot 2024-05-27T09:30:21.531 INFO:tasks.workunit.client.0.smithi114.stdout: create 180/34 file after the snapshot 2024-05-27T09:30:21.535 INFO:tasks.workunit.client.0.smithi114.stdout: create 180/35 file after the snapshot 2024-05-27T09:30:21.540 INFO:tasks.workunit.client.0.smithi114.stdout: create 180/36 file after the snapshot 2024-05-27T09:30:21.544 INFO:tasks.workunit.client.0.smithi114.stdout: create 180/37 file after the snapshot 2024-05-27T09:30:21.549 INFO:tasks.workunit.client.0.smithi114.stdout: create 180/38 file after the snapshot 2024-05-27T09:30:21.553 INFO:tasks.workunit.client.0.smithi114.stdout: create 180/39 file after the snapshot 2024-05-27T09:30:21.557 INFO:tasks.workunit.client.0.smithi114.stdout: create 181/.snap/snap-subdir-test 2024-05-27T09:30:21.621 INFO:tasks.workunit.client.0.smithi114.stdout: create 181/30 file after the snapshot 2024-05-27T09:30:21.626 INFO:tasks.workunit.client.0.smithi114.stdout: create 181/31 file after the snapshot 2024-05-27T09:30:21.633 INFO:tasks.workunit.client.0.smithi114.stdout: create 181/32 file after the snapshot 2024-05-27T09:30:21.636 INFO:tasks.workunit.client.0.smithi114.stdout: create 181/33 file after the snapshot 2024-05-27T09:30:21.638 INFO:tasks.workunit.client.0.smithi114.stdout: create 181/34 file after the snapshot 2024-05-27T09:30:21.640 INFO:tasks.workunit.client.0.smithi114.stdout: create 181/35 file after the snapshot 2024-05-27T09:30:21.642 INFO:tasks.workunit.client.0.smithi114.stdout: create 181/36 file after the snapshot 2024-05-27T09:30:21.710 INFO:tasks.workunit.client.0.smithi114.stdout: create 181/37 file after the snapshot 2024-05-27T09:30:21.840 INFO:tasks.workunit.client.0.smithi114.stdout: create 181/38 file after the snapshot 2024-05-27T09:30:22.279 INFO:tasks.workunit.client.0.smithi114.stdout: create 181/39 file after the snapshot 2024-05-27T09:30:22.635 INFO:tasks.workunit.client.0.smithi114.stdout: create 182/.snap/snap-subdir-test 2024-05-27T09:30:23.015 INFO:tasks.workunit.client.0.smithi114.stdout: create 182/30 file after the snapshot 2024-05-27T09:30:23.017 INFO:tasks.workunit.client.0.smithi114.stdout: create 182/31 file after the snapshot 2024-05-27T09:30:23.019 INFO:tasks.workunit.client.0.smithi114.stdout: create 182/32 file after the snapshot 2024-05-27T09:30:23.021 INFO:tasks.workunit.client.0.smithi114.stdout: create 182/33 file after the snapshot 2024-05-27T09:30:23.024 INFO:tasks.workunit.client.0.smithi114.stdout: create 182/34 file after the snapshot 2024-05-27T09:30:23.026 INFO:tasks.workunit.client.0.smithi114.stdout: create 182/35 file after the snapshot 2024-05-27T09:30:23.029 INFO:tasks.workunit.client.0.smithi114.stdout: create 182/36 file after the snapshot 2024-05-27T09:30:23.031 INFO:tasks.workunit.client.0.smithi114.stdout: create 182/37 file after the snapshot 2024-05-27T09:30:23.034 INFO:tasks.workunit.client.0.smithi114.stdout: create 182/38 file after the snapshot 2024-05-27T09:30:23.036 INFO:tasks.workunit.client.0.smithi114.stdout: create 182/39 file after the snapshot 2024-05-27T09:30:23.039 INFO:tasks.workunit.client.0.smithi114.stdout: create 183/.snap/snap-subdir-test 2024-05-27T09:30:23.084 INFO:tasks.workunit.client.0.smithi114.stdout: create 183/30 file after the snapshot 2024-05-27T09:30:23.086 INFO:tasks.workunit.client.0.smithi114.stdout: create 183/31 file after the snapshot 2024-05-27T09:30:23.087 INFO:tasks.workunit.client.0.smithi114.stdout: create 183/32 file after the snapshot 2024-05-27T09:30:23.088 INFO:tasks.workunit.client.0.smithi114.stdout: create 183/33 file after the snapshot 2024-05-27T09:30:23.089 INFO:tasks.workunit.client.0.smithi114.stdout: create 183/34 file after the snapshot 2024-05-27T09:30:23.090 INFO:tasks.workunit.client.0.smithi114.stdout: create 183/35 file after the snapshot 2024-05-27T09:30:23.091 INFO:tasks.workunit.client.0.smithi114.stdout: create 183/36 file after the snapshot 2024-05-27T09:30:23.092 INFO:tasks.workunit.client.0.smithi114.stdout: create 183/37 file after the snapshot 2024-05-27T09:30:23.093 INFO:tasks.workunit.client.0.smithi114.stdout: create 183/38 file after the snapshot 2024-05-27T09:30:23.094 INFO:tasks.workunit.client.0.smithi114.stdout: create 183/39 file after the snapshot 2024-05-27T09:30:23.095 INFO:tasks.workunit.client.0.smithi114.stdout: create 184/.snap/snap-subdir-test 2024-05-27T09:30:23.134 INFO:tasks.workunit.client.0.smithi114.stdout: create 184/30 file after the snapshot 2024-05-27T09:30:23.137 INFO:tasks.workunit.client.0.smithi114.stdout: create 184/31 file after the snapshot 2024-05-27T09:30:23.141 INFO:tasks.workunit.client.0.smithi114.stdout: create 184/32 file after the snapshot 2024-05-27T09:30:23.143 INFO:tasks.workunit.client.0.smithi114.stdout: create 184/33 file after the snapshot 2024-05-27T09:30:23.144 INFO:tasks.workunit.client.0.smithi114.stdout: create 184/34 file after the snapshot 2024-05-27T09:30:23.146 INFO:tasks.workunit.client.0.smithi114.stdout: create 184/35 file after the snapshot 2024-05-27T09:30:23.148 INFO:tasks.workunit.client.0.smithi114.stdout: create 184/36 file after the snapshot 2024-05-27T09:30:23.150 INFO:tasks.workunit.client.0.smithi114.stdout: create 184/37 file after the snapshot 2024-05-27T09:30:23.153 INFO:tasks.workunit.client.0.smithi114.stdout: create 184/38 file after the snapshot 2024-05-27T09:30:23.155 INFO:tasks.workunit.client.0.smithi114.stdout: create 184/39 file after the snapshot 2024-05-27T09:30:23.158 INFO:tasks.workunit.client.0.smithi114.stdout: create 185/.snap/snap-subdir-test 2024-05-27T09:30:23.199 INFO:tasks.workunit.client.0.smithi114.stdout: create 185/30 file after the snapshot 2024-05-27T09:30:23.202 INFO:tasks.workunit.client.0.smithi114.stdout: create 185/31 file after the snapshot 2024-05-27T09:30:23.205 INFO:tasks.workunit.client.0.smithi114.stdout: create 185/32 file after the snapshot 2024-05-27T09:30:23.207 INFO:tasks.workunit.client.0.smithi114.stdout: create 185/33 file after the snapshot 2024-05-27T09:30:23.210 INFO:tasks.workunit.client.0.smithi114.stdout: create 185/34 file after the snapshot 2024-05-27T09:30:23.213 INFO:tasks.workunit.client.0.smithi114.stdout: create 185/35 file after the snapshot 2024-05-27T09:30:23.216 INFO:tasks.workunit.client.0.smithi114.stdout: create 185/36 file after the snapshot 2024-05-27T09:30:23.219 INFO:tasks.workunit.client.0.smithi114.stdout: create 185/37 file after the snapshot 2024-05-27T09:30:23.222 INFO:tasks.workunit.client.0.smithi114.stdout: create 185/38 file after the snapshot 2024-05-27T09:30:23.225 INFO:tasks.workunit.client.0.smithi114.stdout: create 185/39 file after the snapshot 2024-05-27T09:30:23.228 INFO:tasks.workunit.client.0.smithi114.stdout: create 186/.snap/snap-subdir-test 2024-05-27T09:30:23.274 INFO:tasks.workunit.client.0.smithi114.stdout: create 186/30 file after the snapshot 2024-05-27T09:30:23.278 INFO:tasks.workunit.client.0.smithi114.stdout: create 186/31 file after the snapshot 2024-05-27T09:30:23.281 INFO:tasks.workunit.client.0.smithi114.stdout: create 186/32 file after the snapshot 2024-05-27T09:30:23.284 INFO:tasks.workunit.client.0.smithi114.stdout: create 186/33 file after the snapshot 2024-05-27T09:30:23.287 INFO:tasks.workunit.client.0.smithi114.stdout: create 186/34 file after the snapshot 2024-05-27T09:30:23.290 INFO:tasks.workunit.client.0.smithi114.stdout: create 186/35 file after the snapshot 2024-05-27T09:30:23.293 INFO:tasks.workunit.client.0.smithi114.stdout: create 186/36 file after the snapshot 2024-05-27T09:30:23.295 INFO:tasks.workunit.client.0.smithi114.stdout: create 186/37 file after the snapshot 2024-05-27T09:30:23.298 INFO:tasks.workunit.client.0.smithi114.stdout: create 186/38 file after the snapshot 2024-05-27T09:30:23.301 INFO:tasks.workunit.client.0.smithi114.stdout: create 186/39 file after the snapshot 2024-05-27T09:30:23.304 INFO:tasks.workunit.client.0.smithi114.stdout: create 187/.snap/snap-subdir-test 2024-05-27T09:30:23.352 INFO:tasks.workunit.client.0.smithi114.stdout: create 187/30 file after the snapshot 2024-05-27T09:30:23.356 INFO:tasks.workunit.client.0.smithi114.stdout: create 187/31 file after the snapshot 2024-05-27T09:30:23.358 INFO:tasks.workunit.client.0.smithi114.stdout: create 187/32 file after the snapshot 2024-05-27T09:30:23.361 INFO:tasks.workunit.client.0.smithi114.stdout: create 187/33 file after the snapshot 2024-05-27T09:30:23.364 INFO:tasks.workunit.client.0.smithi114.stdout: create 187/34 file after the snapshot 2024-05-27T09:30:23.367 INFO:tasks.workunit.client.0.smithi114.stdout: create 187/35 file after the snapshot 2024-05-27T09:30:23.369 INFO:tasks.workunit.client.0.smithi114.stdout: create 187/36 file after the snapshot 2024-05-27T09:30:23.372 INFO:tasks.workunit.client.0.smithi114.stdout: create 187/37 file after the snapshot 2024-05-27T09:30:23.375 INFO:tasks.workunit.client.0.smithi114.stdout: create 187/38 file after the snapshot 2024-05-27T09:30:23.377 INFO:tasks.workunit.client.0.smithi114.stdout: create 187/39 file after the snapshot 2024-05-27T09:30:23.380 INFO:tasks.workunit.client.0.smithi114.stdout: create 188/.snap/snap-subdir-test 2024-05-27T09:30:23.428 INFO:tasks.workunit.client.0.smithi114.stdout: create 188/30 file after the snapshot 2024-05-27T09:30:23.431 INFO:tasks.workunit.client.0.smithi114.stdout: create 188/31 file after the snapshot 2024-05-27T09:30:23.434 INFO:tasks.workunit.client.0.smithi114.stdout: create 188/32 file after the snapshot 2024-05-27T09:30:23.436 INFO:tasks.workunit.client.0.smithi114.stdout: create 188/33 file after the snapshot 2024-05-27T09:30:23.439 INFO:tasks.workunit.client.0.smithi114.stdout: create 188/34 file after the snapshot 2024-05-27T09:30:23.442 INFO:tasks.workunit.client.0.smithi114.stdout: create 188/35 file after the snapshot 2024-05-27T09:30:23.444 INFO:tasks.workunit.client.0.smithi114.stdout: create 188/36 file after the snapshot 2024-05-27T09:30:23.447 INFO:tasks.workunit.client.0.smithi114.stdout: create 188/37 file after the snapshot 2024-05-27T09:30:23.450 INFO:tasks.workunit.client.0.smithi114.stdout: create 188/38 file after the snapshot 2024-05-27T09:30:23.453 INFO:tasks.workunit.client.0.smithi114.stdout: create 188/39 file after the snapshot 2024-05-27T09:30:23.455 INFO:tasks.workunit.client.0.smithi114.stdout: create 189/.snap/snap-subdir-test 2024-05-27T09:30:23.493 INFO:tasks.workunit.client.0.smithi114.stdout: create 189/30 file after the snapshot 2024-05-27T09:30:23.495 INFO:tasks.workunit.client.0.smithi114.stdout: create 189/31 file after the snapshot 2024-05-27T09:30:23.497 INFO:tasks.workunit.client.0.smithi114.stdout: create 189/32 file after the snapshot 2024-05-27T09:30:23.499 INFO:tasks.workunit.client.0.smithi114.stdout: create 189/33 file after the snapshot 2024-05-27T09:30:23.502 INFO:tasks.workunit.client.0.smithi114.stdout: create 189/34 file after the snapshot 2024-05-27T09:30:23.505 INFO:tasks.workunit.client.0.smithi114.stdout: create 189/35 file after the snapshot 2024-05-27T09:30:23.508 INFO:tasks.workunit.client.0.smithi114.stdout: create 189/36 file after the snapshot 2024-05-27T09:30:23.511 INFO:tasks.workunit.client.0.smithi114.stdout: create 189/37 file after the snapshot 2024-05-27T09:30:23.514 INFO:tasks.workunit.client.0.smithi114.stdout: create 189/38 file after the snapshot 2024-05-27T09:30:23.517 INFO:tasks.workunit.client.0.smithi114.stdout: create 189/39 file after the snapshot 2024-05-27T09:30:23.520 INFO:tasks.workunit.client.0.smithi114.stdout: create 190/.snap/snap-subdir-test 2024-05-27T09:30:23.566 INFO:tasks.workunit.client.0.smithi114.stdout: create 190/30 file after the snapshot 2024-05-27T09:30:23.569 INFO:tasks.workunit.client.0.smithi114.stdout: create 190/31 file after the snapshot 2024-05-27T09:30:23.572 INFO:tasks.workunit.client.0.smithi114.stdout: create 190/32 file after the snapshot 2024-05-27T09:30:23.575 INFO:tasks.workunit.client.0.smithi114.stdout: create 190/33 file after the snapshot 2024-05-27T09:30:23.578 INFO:tasks.workunit.client.0.smithi114.stdout: create 190/34 file after the snapshot 2024-05-27T09:30:23.580 INFO:tasks.workunit.client.0.smithi114.stdout: create 190/35 file after the snapshot 2024-05-27T09:30:23.583 INFO:tasks.workunit.client.0.smithi114.stdout: create 190/36 file after the snapshot 2024-05-27T09:30:23.586 INFO:tasks.workunit.client.0.smithi114.stdout: create 190/37 file after the snapshot 2024-05-27T09:30:23.589 INFO:tasks.workunit.client.0.smithi114.stdout: create 190/38 file after the snapshot 2024-05-27T09:30:23.591 INFO:tasks.workunit.client.0.smithi114.stdout: create 190/39 file after the snapshot 2024-05-27T09:30:23.594 INFO:tasks.workunit.client.0.smithi114.stdout: create 191/.snap/snap-subdir-test 2024-05-27T09:30:23.647 INFO:tasks.workunit.client.0.smithi114.stdout: create 191/30 file after the snapshot 2024-05-27T09:30:23.649 INFO:tasks.workunit.client.0.smithi114.stdout: create 191/31 file after the snapshot 2024-05-27T09:30:23.651 INFO:tasks.workunit.client.0.smithi114.stdout: create 191/32 file after the snapshot 2024-05-27T09:30:23.652 INFO:tasks.workunit.client.0.smithi114.stdout: create 191/33 file after the snapshot 2024-05-27T09:30:23.654 INFO:tasks.workunit.client.0.smithi114.stdout: create 191/34 file after the snapshot 2024-05-27T09:30:23.657 INFO:tasks.workunit.client.0.smithi114.stdout: create 191/35 file after the snapshot 2024-05-27T09:30:23.660 INFO:tasks.workunit.client.0.smithi114.stdout: create 191/36 file after the snapshot 2024-05-27T09:30:23.663 INFO:tasks.workunit.client.0.smithi114.stdout: create 191/37 file after the snapshot 2024-05-27T09:30:23.666 INFO:tasks.workunit.client.0.smithi114.stdout: create 191/38 file after the snapshot 2024-05-27T09:30:23.669 INFO:tasks.workunit.client.0.smithi114.stdout: create 191/39 file after the snapshot 2024-05-27T09:30:23.671 INFO:tasks.workunit.client.0.smithi114.stdout: create 192/.snap/snap-subdir-test 2024-05-27T09:30:23.719 INFO:tasks.workunit.client.0.smithi114.stdout: create 192/30 file after the snapshot 2024-05-27T09:30:23.721 INFO:tasks.workunit.client.0.smithi114.stdout: create 192/31 file after the snapshot 2024-05-27T09:30:23.722 INFO:tasks.workunit.client.0.smithi114.stdout: create 192/32 file after the snapshot 2024-05-27T09:30:23.724 INFO:tasks.workunit.client.0.smithi114.stdout: create 192/33 file after the snapshot 2024-05-27T09:30:23.725 INFO:tasks.workunit.client.0.smithi114.stdout: create 192/34 file after the snapshot 2024-05-27T09:30:23.729 INFO:tasks.workunit.client.0.smithi114.stdout: create 192/35 file after the snapshot 2024-05-27T09:30:23.730 INFO:tasks.workunit.client.0.smithi114.stdout: create 192/36 file after the snapshot 2024-05-27T09:30:23.731 INFO:tasks.workunit.client.0.smithi114.stdout: create 192/37 file after the snapshot 2024-05-27T09:30:23.732 INFO:tasks.workunit.client.0.smithi114.stdout: create 192/38 file after the snapshot 2024-05-27T09:30:23.734 INFO:tasks.workunit.client.0.smithi114.stdout: create 192/39 file after the snapshot 2024-05-27T09:30:23.736 INFO:tasks.workunit.client.0.smithi114.stdout: create 193/.snap/snap-subdir-test 2024-05-27T09:30:23.781 INFO:tasks.workunit.client.0.smithi114.stdout: create 193/30 file after the snapshot 2024-05-27T09:30:23.784 INFO:tasks.workunit.client.0.smithi114.stdout: create 193/31 file after the snapshot 2024-05-27T09:30:23.786 INFO:tasks.workunit.client.0.smithi114.stdout: create 193/32 file after the snapshot 2024-05-27T09:30:23.789 INFO:tasks.workunit.client.0.smithi114.stdout: create 193/33 file after the snapshot 2024-05-27T09:30:23.792 INFO:tasks.workunit.client.0.smithi114.stdout: create 193/34 file after the snapshot 2024-05-27T09:30:23.795 INFO:tasks.workunit.client.0.smithi114.stdout: create 193/35 file after the snapshot 2024-05-27T09:30:23.798 INFO:tasks.workunit.client.0.smithi114.stdout: create 193/36 file after the snapshot 2024-05-27T09:30:23.800 INFO:tasks.workunit.client.0.smithi114.stdout: create 193/37 file after the snapshot 2024-05-27T09:30:23.803 INFO:tasks.workunit.client.0.smithi114.stdout: create 193/38 file after the snapshot 2024-05-27T09:30:23.806 INFO:tasks.workunit.client.0.smithi114.stdout: create 193/39 file after the snapshot 2024-05-27T09:30:23.809 INFO:tasks.workunit.client.0.smithi114.stdout: create 194/.snap/snap-subdir-test 2024-05-27T09:30:23.851 INFO:tasks.workunit.client.0.smithi114.stdout: create 194/30 file after the snapshot 2024-05-27T09:30:23.853 INFO:tasks.workunit.client.0.smithi114.stdout: create 194/31 file after the snapshot 2024-05-27T09:30:23.856 INFO:tasks.workunit.client.0.smithi114.stdout: create 194/32 file after the snapshot 2024-05-27T09:30:23.859 INFO:tasks.workunit.client.0.smithi114.stdout: create 194/33 file after the snapshot 2024-05-27T09:30:23.862 INFO:tasks.workunit.client.0.smithi114.stdout: create 194/34 file after the snapshot 2024-05-27T09:30:23.865 INFO:tasks.workunit.client.0.smithi114.stdout: create 194/35 file after the snapshot 2024-05-27T09:30:23.868 INFO:tasks.workunit.client.0.smithi114.stdout: create 194/36 file after the snapshot 2024-05-27T09:30:23.870 INFO:tasks.workunit.client.0.smithi114.stdout: create 194/37 file after the snapshot 2024-05-27T09:30:23.873 INFO:tasks.workunit.client.0.smithi114.stdout: create 194/38 file after the snapshot 2024-05-27T09:30:23.876 INFO:tasks.workunit.client.0.smithi114.stdout: create 194/39 file after the snapshot 2024-05-27T09:30:23.879 INFO:tasks.workunit.client.0.smithi114.stdout: create 195/.snap/snap-subdir-test 2024-05-27T09:30:23.929 INFO:tasks.workunit.client.0.smithi114.stdout: create 195/30 file after the snapshot 2024-05-27T09:30:23.934 INFO:tasks.workunit.client.0.smithi114.stdout: create 195/31 file after the snapshot 2024-05-27T09:30:23.934 INFO:tasks.workunit.client.0.smithi114.stdout: create 195/32 file after the snapshot 2024-05-27T09:30:23.934 INFO:tasks.workunit.client.0.smithi114.stdout: create 195/33 file after the snapshot 2024-05-27T09:30:23.936 INFO:tasks.workunit.client.0.smithi114.stdout: create 195/34 file after the snapshot 2024-05-27T09:30:23.938 INFO:tasks.workunit.client.0.smithi114.stdout: create 195/35 file after the snapshot 2024-05-27T09:30:23.939 INFO:tasks.workunit.client.0.smithi114.stdout: create 195/36 file after the snapshot 2024-05-27T09:30:23.941 INFO:tasks.workunit.client.0.smithi114.stdout: create 195/37 file after the snapshot 2024-05-27T09:30:23.944 INFO:tasks.workunit.client.0.smithi114.stdout: create 195/38 file after the snapshot 2024-05-27T09:30:23.947 INFO:tasks.workunit.client.0.smithi114.stdout: create 195/39 file after the snapshot 2024-05-27T09:30:23.950 INFO:tasks.workunit.client.0.smithi114.stdout: create 196/.snap/snap-subdir-test 2024-05-27T09:30:23.993 INFO:tasks.workunit.client.0.smithi114.stdout: create 196/30 file after the snapshot 2024-05-27T09:30:23.996 INFO:tasks.workunit.client.0.smithi114.stdout: create 196/31 file after the snapshot 2024-05-27T09:30:23.998 INFO:tasks.workunit.client.0.smithi114.stdout: create 196/32 file after the snapshot 2024-05-27T09:30:24.001 INFO:tasks.workunit.client.0.smithi114.stdout: create 196/33 file after the snapshot 2024-05-27T09:30:24.004 INFO:tasks.workunit.client.0.smithi114.stdout: create 196/34 file after the snapshot 2024-05-27T09:30:24.006 INFO:tasks.workunit.client.0.smithi114.stdout: create 196/35 file after the snapshot 2024-05-27T09:30:24.009 INFO:tasks.workunit.client.0.smithi114.stdout: create 196/36 file after the snapshot 2024-05-27T09:30:24.011 INFO:tasks.workunit.client.0.smithi114.stdout: create 196/37 file after the snapshot 2024-05-27T09:30:24.014 INFO:tasks.workunit.client.0.smithi114.stdout: create 196/38 file after the snapshot 2024-05-27T09:30:24.017 INFO:tasks.workunit.client.0.smithi114.stdout: create 196/39 file after the snapshot 2024-05-27T09:30:24.020 INFO:tasks.workunit.client.0.smithi114.stdout: create 197/.snap/snap-subdir-test 2024-05-27T09:30:24.063 INFO:tasks.workunit.client.0.smithi114.stdout: create 197/30 file after the snapshot 2024-05-27T09:30:24.066 INFO:tasks.workunit.client.0.smithi114.stdout: create 197/31 file after the snapshot 2024-05-27T09:30:24.068 INFO:tasks.workunit.client.0.smithi114.stdout: create 197/32 file after the snapshot 2024-05-27T09:30:24.070 INFO:tasks.workunit.client.0.smithi114.stdout: create 197/33 file after the snapshot 2024-05-27T09:30:24.073 INFO:tasks.workunit.client.0.smithi114.stdout: create 197/34 file after the snapshot 2024-05-27T09:30:24.076 INFO:tasks.workunit.client.0.smithi114.stdout: create 197/35 file after the snapshot 2024-05-27T09:30:24.079 INFO:tasks.workunit.client.0.smithi114.stdout: create 197/36 file after the snapshot 2024-05-27T09:30:24.082 INFO:tasks.workunit.client.0.smithi114.stdout: create 197/37 file after the snapshot 2024-05-27T09:30:24.085 INFO:tasks.workunit.client.0.smithi114.stdout: create 197/38 file after the snapshot 2024-05-27T09:30:24.088 INFO:tasks.workunit.client.0.smithi114.stdout: create 197/39 file after the snapshot 2024-05-27T09:30:24.090 INFO:tasks.workunit.client.0.smithi114.stdout: create 198/.snap/snap-subdir-test 2024-05-27T09:30:24.144 INFO:tasks.workunit.client.0.smithi114.stdout: create 198/30 file after the snapshot 2024-05-27T09:30:24.148 INFO:tasks.workunit.client.0.smithi114.stdout: create 198/31 file after the snapshot 2024-05-27T09:30:24.153 INFO:tasks.workunit.client.0.smithi114.stdout: create 198/32 file after the snapshot 2024-05-27T09:30:24.158 INFO:tasks.workunit.client.0.smithi114.stdout: create 198/33 file after the snapshot 2024-05-27T09:30:24.162 INFO:tasks.workunit.client.0.smithi114.stdout: create 198/34 file after the snapshot 2024-05-27T09:30:24.166 INFO:tasks.workunit.client.0.smithi114.stdout: create 198/35 file after the snapshot 2024-05-27T09:30:24.170 INFO:tasks.workunit.client.0.smithi114.stdout: create 198/36 file after the snapshot 2024-05-27T09:30:24.175 INFO:tasks.workunit.client.0.smithi114.stdout: create 198/37 file after the snapshot 2024-05-27T09:30:24.179 INFO:tasks.workunit.client.0.smithi114.stdout: create 198/38 file after the snapshot 2024-05-27T09:30:24.184 INFO:tasks.workunit.client.0.smithi114.stdout: create 198/39 file after the snapshot 2024-05-27T09:30:24.188 INFO:tasks.workunit.client.0.smithi114.stdout: create 199/.snap/snap-subdir-test 2024-05-27T09:30:24.238 INFO:tasks.workunit.client.0.smithi114.stdout: create 199/30 file after the snapshot 2024-05-27T09:30:24.241 INFO:tasks.workunit.client.0.smithi114.stdout: create 199/31 file after the snapshot 2024-05-27T09:30:24.244 INFO:tasks.workunit.client.0.smithi114.stdout: create 199/32 file after the snapshot 2024-05-27T09:30:24.247 INFO:tasks.workunit.client.0.smithi114.stdout: create 199/33 file after the snapshot 2024-05-27T09:30:24.250 INFO:tasks.workunit.client.0.smithi114.stdout: create 199/34 file after the snapshot 2024-05-27T09:30:24.253 INFO:tasks.workunit.client.0.smithi114.stdout: create 199/35 file after the snapshot 2024-05-27T09:30:24.256 INFO:tasks.workunit.client.0.smithi114.stdout: create 199/36 file after the snapshot 2024-05-27T09:30:24.259 INFO:tasks.workunit.client.0.smithi114.stdout: create 199/37 file after the snapshot 2024-05-27T09:30:24.262 INFO:tasks.workunit.client.0.smithi114.stdout: create 199/38 file after the snapshot 2024-05-27T09:30:24.265 INFO:tasks.workunit.client.0.smithi114.stdout: create 199/39 file after the snapshot 2024-05-27T09:30:24.268 INFO:tasks.workunit.client.0.smithi114.stdout: create 200/.snap/snap-subdir-test 2024-05-27T09:30:24.324 INFO:tasks.workunit.client.0.smithi114.stdout: create 200/30 file after the snapshot 2024-05-27T09:30:24.326 INFO:tasks.workunit.client.0.smithi114.stdout: create 200/31 file after the snapshot 2024-05-27T09:30:24.329 INFO:tasks.workunit.client.0.smithi114.stdout: create 200/32 file after the snapshot 2024-05-27T09:30:24.332 INFO:tasks.workunit.client.0.smithi114.stdout: create 200/33 file after the snapshot 2024-05-27T09:30:24.335 INFO:tasks.workunit.client.0.smithi114.stdout: create 200/34 file after the snapshot 2024-05-27T09:30:24.337 INFO:tasks.workunit.client.0.smithi114.stdout: create 200/35 file after the snapshot 2024-05-27T09:30:24.341 INFO:tasks.workunit.client.0.smithi114.stdout: create 200/36 file after the snapshot 2024-05-27T09:30:24.343 INFO:tasks.workunit.client.0.smithi114.stdout: create 200/37 file after the snapshot 2024-05-27T09:30:24.346 INFO:tasks.workunit.client.0.smithi114.stdout: create 200/38 file after the snapshot 2024-05-27T09:30:24.349 INFO:tasks.workunit.client.0.smithi114.stdout: create 200/39 file after the snapshot 2024-05-27T09:30:24.352 INFO:tasks.workunit.client.0.smithi114.stdout: create 201/.snap/snap-subdir-test 2024-05-27T09:30:24.404 INFO:tasks.workunit.client.0.smithi114.stdout: create 201/30 file after the snapshot 2024-05-27T09:30:24.405 INFO:tasks.workunit.client.0.smithi114.stdout: create 201/31 file after the snapshot 2024-05-27T09:30:24.407 INFO:tasks.workunit.client.0.smithi114.stdout: create 201/32 file after the snapshot 2024-05-27T09:30:24.409 INFO:tasks.workunit.client.0.smithi114.stdout: create 201/33 file after the snapshot 2024-05-27T09:30:24.411 INFO:tasks.workunit.client.0.smithi114.stdout: create 201/34 file after the snapshot 2024-05-27T09:30:24.414 INFO:tasks.workunit.client.0.smithi114.stdout: create 201/35 file after the snapshot 2024-05-27T09:30:24.417 INFO:tasks.workunit.client.0.smithi114.stdout: create 201/36 file after the snapshot 2024-05-27T09:30:24.420 INFO:tasks.workunit.client.0.smithi114.stdout: create 201/37 file after the snapshot 2024-05-27T09:30:24.423 INFO:tasks.workunit.client.0.smithi114.stdout: create 201/38 file after the snapshot 2024-05-27T09:30:24.426 INFO:tasks.workunit.client.0.smithi114.stdout: create 201/39 file after the snapshot 2024-05-27T09:30:24.428 INFO:tasks.workunit.client.0.smithi114.stdout: create 202/.snap/snap-subdir-test 2024-05-27T09:30:24.480 INFO:tasks.workunit.client.0.smithi114.stdout: create 202/30 file after the snapshot 2024-05-27T09:30:24.483 INFO:tasks.workunit.client.0.smithi114.stdout: create 202/31 file after the snapshot 2024-05-27T09:30:24.484 INFO:tasks.workunit.client.0.smithi114.stdout: create 202/32 file after the snapshot 2024-05-27T09:30:24.486 INFO:tasks.workunit.client.0.smithi114.stdout: create 202/33 file after the snapshot 2024-05-27T09:30:24.488 INFO:tasks.workunit.client.0.smithi114.stdout: create 202/34 file after the snapshot 2024-05-27T09:30:24.490 INFO:tasks.workunit.client.0.smithi114.stdout: create 202/35 file after the snapshot 2024-05-27T09:30:24.492 INFO:tasks.workunit.client.0.smithi114.stdout: create 202/36 file after the snapshot 2024-05-27T09:30:24.494 INFO:tasks.workunit.client.0.smithi114.stdout: create 202/37 file after the snapshot 2024-05-27T09:30:24.495 INFO:tasks.workunit.client.0.smithi114.stdout: create 202/38 file after the snapshot 2024-05-27T09:30:24.497 INFO:tasks.workunit.client.0.smithi114.stdout: create 202/39 file after the snapshot 2024-05-27T09:30:24.500 INFO:tasks.workunit.client.0.smithi114.stdout: create 203/.snap/snap-subdir-test 2024-05-27T09:30:24.550 INFO:tasks.workunit.client.0.smithi114.stdout: create 203/30 file after the snapshot 2024-05-27T09:30:24.553 INFO:tasks.workunit.client.0.smithi114.stdout: create 203/31 file after the snapshot 2024-05-27T09:30:24.555 INFO:tasks.workunit.client.0.smithi114.stdout: create 203/32 file after the snapshot 2024-05-27T09:30:24.557 INFO:tasks.workunit.client.0.smithi114.stdout: create 203/33 file after the snapshot 2024-05-27T09:30:24.559 INFO:tasks.workunit.client.0.smithi114.stdout: create 203/34 file after the snapshot 2024-05-27T09:30:24.561 INFO:tasks.workunit.client.0.smithi114.stdout: create 203/35 file after the snapshot 2024-05-27T09:30:24.563 INFO:tasks.workunit.client.0.smithi114.stdout: create 203/36 file after the snapshot 2024-05-27T09:30:24.565 INFO:tasks.workunit.client.0.smithi114.stdout: create 203/37 file after the snapshot 2024-05-27T09:30:24.567 INFO:tasks.workunit.client.0.smithi114.stdout: create 203/38 file after the snapshot 2024-05-27T09:30:24.569 INFO:tasks.workunit.client.0.smithi114.stdout: create 203/39 file after the snapshot 2024-05-27T09:30:24.572 INFO:tasks.workunit.client.0.smithi114.stdout: create 204/.snap/snap-subdir-test 2024-05-27T09:30:24.612 INFO:tasks.workunit.client.0.smithi114.stdout: create 204/30 file after the snapshot 2024-05-27T09:30:24.616 INFO:tasks.workunit.client.0.smithi114.stdout: create 204/31 file after the snapshot 2024-05-27T09:30:24.620 INFO:tasks.workunit.client.0.smithi114.stdout: create 204/32 file after the snapshot 2024-05-27T09:30:24.625 INFO:tasks.workunit.client.0.smithi114.stdout: create 204/33 file after the snapshot 2024-05-27T09:30:24.629 INFO:tasks.workunit.client.0.smithi114.stdout: create 204/34 file after the snapshot 2024-05-27T09:30:24.633 INFO:tasks.workunit.client.0.smithi114.stdout: create 204/35 file after the snapshot 2024-05-27T09:30:24.638 INFO:tasks.workunit.client.0.smithi114.stdout: create 204/36 file after the snapshot 2024-05-27T09:30:24.642 INFO:tasks.workunit.client.0.smithi114.stdout: create 204/37 file after the snapshot 2024-05-27T09:30:24.647 INFO:tasks.workunit.client.0.smithi114.stdout: create 204/38 file after the snapshot 2024-05-27T09:30:24.651 INFO:tasks.workunit.client.0.smithi114.stdout: create 204/39 file after the snapshot 2024-05-27T09:30:24.655 INFO:tasks.workunit.client.0.smithi114.stdout: create 205/.snap/snap-subdir-test 2024-05-27T09:30:24.709 INFO:tasks.workunit.client.0.smithi114.stdout: create 205/30 file after the snapshot 2024-05-27T09:30:24.712 INFO:tasks.workunit.client.0.smithi114.stdout: create 205/31 file after the snapshot 2024-05-27T09:30:24.715 INFO:tasks.workunit.client.0.smithi114.stdout: create 205/32 file after the snapshot 2024-05-27T09:30:24.718 INFO:tasks.workunit.client.0.smithi114.stdout: create 205/33 file after the snapshot 2024-05-27T09:30:24.721 INFO:tasks.workunit.client.0.smithi114.stdout: create 205/34 file after the snapshot 2024-05-27T09:30:24.724 INFO:tasks.workunit.client.0.smithi114.stdout: create 205/35 file after the snapshot 2024-05-27T09:30:24.727 INFO:tasks.workunit.client.0.smithi114.stdout: create 205/36 file after the snapshot 2024-05-27T09:30:24.730 INFO:tasks.workunit.client.0.smithi114.stdout: create 205/37 file after the snapshot 2024-05-27T09:30:24.733 INFO:tasks.workunit.client.0.smithi114.stdout: create 205/38 file after the snapshot 2024-05-27T09:30:24.736 INFO:tasks.workunit.client.0.smithi114.stdout: create 205/39 file after the snapshot 2024-05-27T09:30:24.739 INFO:tasks.workunit.client.0.smithi114.stdout: create 206/.snap/snap-subdir-test 2024-05-27T09:30:24.782 INFO:tasks.workunit.client.0.smithi114.stdout: create 206/30 file after the snapshot 2024-05-27T09:30:24.785 INFO:tasks.workunit.client.0.smithi114.stdout: create 206/31 file after the snapshot 2024-05-27T09:30:24.788 INFO:tasks.workunit.client.0.smithi114.stdout: create 206/32 file after the snapshot 2024-05-27T09:30:24.791 INFO:tasks.workunit.client.0.smithi114.stdout: create 206/33 file after the snapshot 2024-05-27T09:30:24.794 INFO:tasks.workunit.client.0.smithi114.stdout: create 206/34 file after the snapshot 2024-05-27T09:30:24.797 INFO:tasks.workunit.client.0.smithi114.stdout: create 206/35 file after the snapshot 2024-05-27T09:30:24.800 INFO:tasks.workunit.client.0.smithi114.stdout: create 206/36 file after the snapshot 2024-05-27T09:30:24.802 INFO:tasks.workunit.client.0.smithi114.stdout: create 206/37 file after the snapshot 2024-05-27T09:30:24.805 INFO:tasks.workunit.client.0.smithi114.stdout: create 206/38 file after the snapshot 2024-05-27T09:30:24.808 INFO:tasks.workunit.client.0.smithi114.stdout: create 206/39 file after the snapshot 2024-05-27T09:30:24.811 INFO:tasks.workunit.client.0.smithi114.stdout: create 207/.snap/snap-subdir-test 2024-05-27T09:30:24.857 INFO:tasks.workunit.client.0.smithi114.stdout: create 207/30 file after the snapshot 2024-05-27T09:30:24.860 INFO:tasks.workunit.client.0.smithi114.stdout: create 207/31 file after the snapshot 2024-05-27T09:30:24.863 INFO:tasks.workunit.client.0.smithi114.stdout: create 207/32 file after the snapshot 2024-05-27T09:30:24.866 INFO:tasks.workunit.client.0.smithi114.stdout: create 207/33 file after the snapshot 2024-05-27T09:30:24.869 INFO:tasks.workunit.client.0.smithi114.stdout: create 207/34 file after the snapshot 2024-05-27T09:30:24.872 INFO:tasks.workunit.client.0.smithi114.stdout: create 207/35 file after the snapshot 2024-05-27T09:30:24.875 INFO:tasks.workunit.client.0.smithi114.stdout: create 207/36 file after the snapshot 2024-05-27T09:30:24.878 INFO:tasks.workunit.client.0.smithi114.stdout: create 207/37 file after the snapshot 2024-05-27T09:30:24.881 INFO:tasks.workunit.client.0.smithi114.stdout: create 207/38 file after the snapshot 2024-05-27T09:30:24.884 INFO:tasks.workunit.client.0.smithi114.stdout: create 207/39 file after the snapshot 2024-05-27T09:30:24.887 INFO:tasks.workunit.client.0.smithi114.stdout: create 208/.snap/snap-subdir-test 2024-05-27T09:30:24.939 INFO:tasks.workunit.client.0.smithi114.stdout: create 208/30 file after the snapshot 2024-05-27T09:30:24.941 INFO:tasks.workunit.client.0.smithi114.stdout: create 208/31 file after the snapshot 2024-05-27T09:30:24.944 INFO:tasks.workunit.client.0.smithi114.stdout: create 208/32 file after the snapshot 2024-05-27T09:30:24.947 INFO:tasks.workunit.client.0.smithi114.stdout: create 208/33 file after the snapshot 2024-05-27T09:30:24.950 INFO:tasks.workunit.client.0.smithi114.stdout: create 208/34 file after the snapshot 2024-05-27T09:30:24.953 INFO:tasks.workunit.client.0.smithi114.stdout: create 208/35 file after the snapshot 2024-05-27T09:30:24.956 INFO:tasks.workunit.client.0.smithi114.stdout: create 208/36 file after the snapshot 2024-05-27T09:30:24.959 INFO:tasks.workunit.client.0.smithi114.stdout: create 208/37 file after the snapshot 2024-05-27T09:30:24.962 INFO:tasks.workunit.client.0.smithi114.stdout: create 208/38 file after the snapshot 2024-05-27T09:30:24.965 INFO:tasks.workunit.client.0.smithi114.stdout: create 208/39 file after the snapshot 2024-05-27T09:30:24.968 INFO:tasks.workunit.client.0.smithi114.stdout: create 209/.snap/snap-subdir-test 2024-05-27T09:30:25.015 INFO:tasks.workunit.client.0.smithi114.stdout: create 209/30 file after the snapshot 2024-05-27T09:30:25.017 INFO:tasks.workunit.client.0.smithi114.stdout: create 209/31 file after the snapshot 2024-05-27T09:30:25.020 INFO:tasks.workunit.client.0.smithi114.stdout: create 209/32 file after the snapshot 2024-05-27T09:30:25.023 INFO:tasks.workunit.client.0.smithi114.stdout: create 209/33 file after the snapshot 2024-05-27T09:30:25.026 INFO:tasks.workunit.client.0.smithi114.stdout: create 209/34 file after the snapshot 2024-05-27T09:30:25.028 INFO:tasks.workunit.client.0.smithi114.stdout: create 209/35 file after the snapshot 2024-05-27T09:30:25.031 INFO:tasks.workunit.client.0.smithi114.stdout: create 209/36 file after the snapshot 2024-05-27T09:30:25.034 INFO:tasks.workunit.client.0.smithi114.stdout: create 209/37 file after the snapshot 2024-05-27T09:30:25.037 INFO:tasks.workunit.client.0.smithi114.stdout: create 209/38 file after the snapshot 2024-05-27T09:30:25.040 INFO:tasks.workunit.client.0.smithi114.stdout: create 209/39 file after the snapshot 2024-05-27T09:30:25.043 INFO:tasks.workunit.client.0.smithi114.stdout: create 210/.snap/snap-subdir-test 2024-05-27T09:30:25.091 INFO:tasks.workunit.client.0.smithi114.stdout: create 210/30 file after the snapshot 2024-05-27T09:30:25.094 INFO:tasks.workunit.client.0.smithi114.stdout: create 210/31 file after the snapshot 2024-05-27T09:30:25.096 INFO:tasks.workunit.client.0.smithi114.stdout: create 210/32 file after the snapshot 2024-05-27T09:30:25.099 INFO:tasks.workunit.client.0.smithi114.stdout: create 210/33 file after the snapshot 2024-05-27T09:30:25.102 INFO:tasks.workunit.client.0.smithi114.stdout: create 210/34 file after the snapshot 2024-05-27T09:30:25.105 INFO:tasks.workunit.client.0.smithi114.stdout: create 210/35 file after the snapshot 2024-05-27T09:30:25.107 INFO:tasks.workunit.client.0.smithi114.stdout: create 210/36 file after the snapshot 2024-05-27T09:30:25.110 INFO:tasks.workunit.client.0.smithi114.stdout: create 210/37 file after the snapshot 2024-05-27T09:30:25.113 INFO:tasks.workunit.client.0.smithi114.stdout: create 210/38 file after the snapshot 2024-05-27T09:30:25.115 INFO:tasks.workunit.client.0.smithi114.stdout: create 210/39 file after the snapshot 2024-05-27T09:30:25.118 INFO:tasks.workunit.client.0.smithi114.stdout: create 211/.snap/snap-subdir-test 2024-05-27T09:30:25.160 INFO:tasks.workunit.client.0.smithi114.stdout: create 211/30 file after the snapshot 2024-05-27T09:30:25.161 INFO:tasks.workunit.client.0.smithi114.stdout: create 211/31 file after the snapshot 2024-05-27T09:30:25.163 INFO:tasks.workunit.client.0.smithi114.stdout: create 211/32 file after the snapshot 2024-05-27T09:30:25.165 INFO:tasks.workunit.client.0.smithi114.stdout: create 211/33 file after the snapshot 2024-05-27T09:30:25.166 INFO:tasks.workunit.client.0.smithi114.stdout: create 211/34 file after the snapshot 2024-05-27T09:30:25.168 INFO:tasks.workunit.client.0.smithi114.stdout: create 211/35 file after the snapshot 2024-05-27T09:30:25.170 INFO:tasks.workunit.client.0.smithi114.stdout: create 211/36 file after the snapshot 2024-05-27T09:30:25.172 INFO:tasks.workunit.client.0.smithi114.stdout: create 211/37 file after the snapshot 2024-05-27T09:30:25.175 INFO:tasks.workunit.client.0.smithi114.stdout: create 211/38 file after the snapshot 2024-05-27T09:30:25.178 INFO:tasks.workunit.client.0.smithi114.stdout: create 211/39 file after the snapshot 2024-05-27T09:30:25.181 INFO:tasks.workunit.client.0.smithi114.stdout: create 212/.snap/snap-subdir-test 2024-05-27T09:30:25.233 INFO:tasks.workunit.client.0.smithi114.stdout: create 212/30 file after the snapshot 2024-05-27T09:30:25.236 INFO:tasks.workunit.client.0.smithi114.stdout: create 212/31 file after the snapshot 2024-05-27T09:30:25.238 INFO:tasks.workunit.client.0.smithi114.stdout: create 212/32 file after the snapshot 2024-05-27T09:30:25.241 INFO:tasks.workunit.client.0.smithi114.stdout: create 212/33 file after the snapshot 2024-05-27T09:30:25.244 INFO:tasks.workunit.client.0.smithi114.stdout: create 212/34 file after the snapshot 2024-05-27T09:30:25.246 INFO:tasks.workunit.client.0.smithi114.stdout: create 212/35 file after the snapshot 2024-05-27T09:30:25.249 INFO:tasks.workunit.client.0.smithi114.stdout: create 212/36 file after the snapshot 2024-05-27T09:30:25.251 INFO:tasks.workunit.client.0.smithi114.stdout: create 212/37 file after the snapshot 2024-05-27T09:30:25.254 INFO:tasks.workunit.client.0.smithi114.stdout: create 212/38 file after the snapshot 2024-05-27T09:30:25.256 INFO:tasks.workunit.client.0.smithi114.stdout: create 212/39 file after the snapshot 2024-05-27T09:30:25.259 INFO:tasks.workunit.client.0.smithi114.stdout: create 213/.snap/snap-subdir-test 2024-05-27T09:30:25.303 INFO:tasks.workunit.client.0.smithi114.stdout: create 213/30 file after the snapshot 2024-05-27T09:30:25.306 INFO:tasks.workunit.client.0.smithi114.stdout: create 213/31 file after the snapshot 2024-05-27T09:30:25.309 INFO:tasks.workunit.client.0.smithi114.stdout: create 213/32 file after the snapshot 2024-05-27T09:30:25.311 INFO:tasks.workunit.client.0.smithi114.stdout: create 213/33 file after the snapshot 2024-05-27T09:30:25.314 INFO:tasks.workunit.client.0.smithi114.stdout: create 213/34 file after the snapshot 2024-05-27T09:30:25.317 INFO:tasks.workunit.client.0.smithi114.stdout: create 213/35 file after the snapshot 2024-05-27T09:30:25.320 INFO:tasks.workunit.client.0.smithi114.stdout: create 213/36 file after the snapshot 2024-05-27T09:30:25.322 INFO:tasks.workunit.client.0.smithi114.stdout: create 213/37 file after the snapshot 2024-05-27T09:30:25.325 INFO:tasks.workunit.client.0.smithi114.stdout: create 213/38 file after the snapshot 2024-05-27T09:30:25.328 INFO:tasks.workunit.client.0.smithi114.stdout: create 213/39 file after the snapshot 2024-05-27T09:30:25.331 INFO:tasks.workunit.client.0.smithi114.stdout: create 214/.snap/snap-subdir-test 2024-05-27T09:30:25.365 INFO:tasks.workunit.client.0.smithi114.stdout: create 214/30 file after the snapshot 2024-05-27T09:30:25.369 INFO:tasks.workunit.client.0.smithi114.stdout: create 214/31 file after the snapshot 2024-05-27T09:30:25.374 INFO:tasks.workunit.client.0.smithi114.stdout: create 214/32 file after the snapshot 2024-05-27T09:30:25.378 INFO:tasks.workunit.client.0.smithi114.stdout: create 214/33 file after the snapshot 2024-05-27T09:30:25.382 INFO:tasks.workunit.client.0.smithi114.stdout: create 214/34 file after the snapshot 2024-05-27T09:30:25.387 INFO:tasks.workunit.client.0.smithi114.stdout: create 214/35 file after the snapshot 2024-05-27T09:30:25.391 INFO:tasks.workunit.client.0.smithi114.stdout: create 214/36 file after the snapshot 2024-05-27T09:30:25.396 INFO:tasks.workunit.client.0.smithi114.stdout: create 214/37 file after the snapshot 2024-05-27T09:30:25.400 INFO:tasks.workunit.client.0.smithi114.stdout: create 214/38 file after the snapshot 2024-05-27T09:30:25.404 INFO:tasks.workunit.client.0.smithi114.stdout: create 214/39 file after the snapshot 2024-05-27T09:30:25.409 INFO:tasks.workunit.client.0.smithi114.stdout: create 215/.snap/snap-subdir-test 2024-05-27T09:30:25.450 INFO:tasks.workunit.client.0.smithi114.stdout: create 215/30 file after the snapshot 2024-05-27T09:30:25.453 INFO:tasks.workunit.client.0.smithi114.stdout: create 215/31 file after the snapshot 2024-05-27T09:30:25.455 INFO:tasks.workunit.client.0.smithi114.stdout: create 215/32 file after the snapshot 2024-05-27T09:30:25.456 INFO:tasks.workunit.client.0.smithi114.stdout: create 215/33 file after the snapshot 2024-05-27T09:30:25.459 INFO:tasks.workunit.client.0.smithi114.stdout: create 215/34 file after the snapshot 2024-05-27T09:30:25.461 INFO:tasks.workunit.client.0.smithi114.stdout: create 215/35 file after the snapshot 2024-05-27T09:30:25.463 INFO:tasks.workunit.client.0.smithi114.stdout: create 215/36 file after the snapshot 2024-05-27T09:30:25.465 INFO:tasks.workunit.client.0.smithi114.stdout: create 215/37 file after the snapshot 2024-05-27T09:30:25.466 INFO:tasks.workunit.client.0.smithi114.stdout: create 215/38 file after the snapshot 2024-05-27T09:30:25.469 INFO:tasks.workunit.client.0.smithi114.stdout: create 215/39 file after the snapshot 2024-05-27T09:30:25.472 INFO:tasks.workunit.client.0.smithi114.stdout: create 216/.snap/snap-subdir-test 2024-05-27T09:30:25.514 INFO:tasks.workunit.client.0.smithi114.stdout: create 216/30 file after the snapshot 2024-05-27T09:30:25.516 INFO:tasks.workunit.client.0.smithi114.stdout: create 216/31 file after the snapshot 2024-05-27T09:30:25.518 INFO:tasks.workunit.client.0.smithi114.stdout: create 216/32 file after the snapshot 2024-05-27T09:30:25.521 INFO:tasks.workunit.client.0.smithi114.stdout: create 216/33 file after the snapshot 2024-05-27T09:30:25.524 INFO:tasks.workunit.client.0.smithi114.stdout: create 216/34 file after the snapshot 2024-05-27T09:30:25.526 INFO:tasks.workunit.client.0.smithi114.stdout: create 216/35 file after the snapshot 2024-05-27T09:30:25.529 INFO:tasks.workunit.client.0.smithi114.stdout: create 216/36 file after the snapshot 2024-05-27T09:30:25.532 INFO:tasks.workunit.client.0.smithi114.stdout: create 216/37 file after the snapshot 2024-05-27T09:30:25.535 INFO:tasks.workunit.client.0.smithi114.stdout: create 216/38 file after the snapshot 2024-05-27T09:30:25.538 INFO:tasks.workunit.client.0.smithi114.stdout: create 216/39 file after the snapshot 2024-05-27T09:30:25.541 INFO:tasks.workunit.client.0.smithi114.stdout: create 217/.snap/snap-subdir-test 2024-05-27T09:30:25.577 INFO:tasks.workunit.client.0.smithi114.stdout: create 217/30 file after the snapshot 2024-05-27T09:30:25.580 INFO:tasks.workunit.client.0.smithi114.stdout: create 217/31 file after the snapshot 2024-05-27T09:30:25.582 INFO:tasks.workunit.client.0.smithi114.stdout: create 217/32 file after the snapshot 2024-05-27T09:30:25.585 INFO:tasks.workunit.client.0.smithi114.stdout: create 217/33 file after the snapshot 2024-05-27T09:30:25.588 INFO:tasks.workunit.client.0.smithi114.stdout: create 217/34 file after the snapshot 2024-05-27T09:30:25.590 INFO:tasks.workunit.client.0.smithi114.stdout: create 217/35 file after the snapshot 2024-05-27T09:30:25.593 INFO:tasks.workunit.client.0.smithi114.stdout: create 217/36 file after the snapshot 2024-05-27T09:30:25.596 INFO:tasks.workunit.client.0.smithi114.stdout: create 217/37 file after the snapshot 2024-05-27T09:30:25.598 INFO:tasks.workunit.client.0.smithi114.stdout: create 217/38 file after the snapshot 2024-05-27T09:30:25.601 INFO:tasks.workunit.client.0.smithi114.stdout: create 217/39 file after the snapshot 2024-05-27T09:30:25.604 INFO:tasks.workunit.client.0.smithi114.stdout: create 218/.snap/snap-subdir-test 2024-05-27T09:30:25.645 INFO:tasks.workunit.client.0.smithi114.stdout: create 218/30 file after the snapshot 2024-05-27T09:30:25.648 INFO:tasks.workunit.client.0.smithi114.stdout: create 218/31 file after the snapshot 2024-05-27T09:30:25.651 INFO:tasks.workunit.client.0.smithi114.stdout: create 218/32 file after the snapshot 2024-05-27T09:30:25.653 INFO:tasks.workunit.client.0.smithi114.stdout: create 218/33 file after the snapshot 2024-05-27T09:30:25.654 INFO:tasks.workunit.client.0.smithi114.stdout: create 218/34 file after the snapshot 2024-05-27T09:30:25.656 INFO:tasks.workunit.client.0.smithi114.stdout: create 218/35 file after the snapshot 2024-05-27T09:30:25.658 INFO:tasks.workunit.client.0.smithi114.stdout: create 218/36 file after the snapshot 2024-05-27T09:30:25.660 INFO:tasks.workunit.client.0.smithi114.stdout: create 218/37 file after the snapshot 2024-05-27T09:30:25.663 INFO:tasks.workunit.client.0.smithi114.stdout: create 218/38 file after the snapshot 2024-05-27T09:30:25.666 INFO:tasks.workunit.client.0.smithi114.stdout: create 218/39 file after the snapshot 2024-05-27T09:30:25.669 INFO:tasks.workunit.client.0.smithi114.stdout: create 219/.snap/snap-subdir-test 2024-05-27T09:30:25.712 INFO:tasks.workunit.client.0.smithi114.stdout: create 219/30 file after the snapshot 2024-05-27T09:30:25.714 INFO:tasks.workunit.client.0.smithi114.stdout: create 219/31 file after the snapshot 2024-05-27T09:30:25.716 INFO:tasks.workunit.client.0.smithi114.stdout: create 219/32 file after the snapshot 2024-05-27T09:30:25.718 INFO:tasks.workunit.client.0.smithi114.stdout: create 219/33 file after the snapshot 2024-05-27T09:30:25.720 INFO:tasks.workunit.client.0.smithi114.stdout: create 219/34 file after the snapshot 2024-05-27T09:30:25.722 INFO:tasks.workunit.client.0.smithi114.stdout: create 219/35 file after the snapshot 2024-05-27T09:30:25.724 INFO:tasks.workunit.client.0.smithi114.stdout: create 219/36 file after the snapshot 2024-05-27T09:30:25.726 INFO:tasks.workunit.client.0.smithi114.stdout: create 219/37 file after the snapshot 2024-05-27T09:30:25.728 INFO:tasks.workunit.client.0.smithi114.stdout: create 219/38 file after the snapshot 2024-05-27T09:30:25.730 INFO:tasks.workunit.client.0.smithi114.stdout: create 219/39 file after the snapshot 2024-05-27T09:30:25.733 INFO:tasks.workunit.client.0.smithi114.stdout: create 220/.snap/snap-subdir-test 2024-05-27T09:30:25.778 INFO:tasks.workunit.client.0.smithi114.stdout: create 220/30 file after the snapshot 2024-05-27T09:30:25.779 INFO:tasks.workunit.client.0.smithi114.stdout: create 220/31 file after the snapshot 2024-05-27T09:30:25.781 INFO:tasks.workunit.client.0.smithi114.stdout: create 220/32 file after the snapshot 2024-05-27T09:30:25.783 INFO:tasks.workunit.client.0.smithi114.stdout: create 220/33 file after the snapshot 2024-05-27T09:30:25.784 INFO:tasks.workunit.client.0.smithi114.stdout: create 220/34 file after the snapshot 2024-05-27T09:30:25.786 INFO:tasks.workunit.client.0.smithi114.stdout: create 220/35 file after the snapshot 2024-05-27T09:30:25.789 INFO:tasks.workunit.client.0.smithi114.stdout: create 220/36 file after the snapshot 2024-05-27T09:30:25.792 INFO:tasks.workunit.client.0.smithi114.stdout: create 220/37 file after the snapshot 2024-05-27T09:30:25.794 INFO:tasks.workunit.client.0.smithi114.stdout: create 220/38 file after the snapshot 2024-05-27T09:30:25.797 INFO:tasks.workunit.client.0.smithi114.stdout: create 220/39 file after the snapshot 2024-05-27T09:30:25.800 INFO:tasks.workunit.client.0.smithi114.stdout: create 221/.snap/snap-subdir-test 2024-05-27T09:30:25.851 INFO:tasks.workunit.client.0.smithi114.stdout: create 221/30 file after the snapshot 2024-05-27T09:30:25.855 INFO:tasks.workunit.client.0.smithi114.stdout: create 221/31 file after the snapshot 2024-05-27T09:30:25.860 INFO:tasks.workunit.client.0.smithi114.stdout: create 221/32 file after the snapshot 2024-05-27T09:30:25.864 INFO:tasks.workunit.client.0.smithi114.stdout: create 221/33 file after the snapshot 2024-05-27T09:30:25.869 INFO:tasks.workunit.client.0.smithi114.stdout: create 221/34 file after the snapshot 2024-05-27T09:30:25.873 INFO:tasks.workunit.client.0.smithi114.stdout: create 221/35 file after the snapshot 2024-05-27T09:30:25.877 INFO:tasks.workunit.client.0.smithi114.stdout: create 221/36 file after the snapshot 2024-05-27T09:30:25.880 INFO:tasks.workunit.client.0.smithi114.stdout: create 221/37 file after the snapshot 2024-05-27T09:30:25.883 INFO:tasks.workunit.client.0.smithi114.stdout: create 221/38 file after the snapshot 2024-05-27T09:30:25.885 INFO:tasks.workunit.client.0.smithi114.stdout: create 221/39 file after the snapshot 2024-05-27T09:30:25.888 INFO:tasks.workunit.client.0.smithi114.stdout: create 222/.snap/snap-subdir-test 2024-05-27T09:30:25.926 INFO:tasks.workunit.client.0.smithi114.stdout: create 222/30 file after the snapshot 2024-05-27T09:30:25.929 INFO:tasks.workunit.client.0.smithi114.stdout: create 222/31 file after the snapshot 2024-05-27T09:30:25.932 INFO:tasks.workunit.client.0.smithi114.stdout: create 222/32 file after the snapshot 2024-05-27T09:30:25.935 INFO:tasks.workunit.client.0.smithi114.stdout: create 222/33 file after the snapshot 2024-05-27T09:30:25.938 INFO:tasks.workunit.client.0.smithi114.stdout: create 222/34 file after the snapshot 2024-05-27T09:30:25.941 INFO:tasks.workunit.client.0.smithi114.stdout: create 222/35 file after the snapshot 2024-05-27T09:30:25.943 INFO:tasks.workunit.client.0.smithi114.stdout: create 222/36 file after the snapshot 2024-05-27T09:30:25.946 INFO:tasks.workunit.client.0.smithi114.stdout: create 222/37 file after the snapshot 2024-05-27T09:30:25.949 INFO:tasks.workunit.client.0.smithi114.stdout: create 222/38 file after the snapshot 2024-05-27T09:30:25.952 INFO:tasks.workunit.client.0.smithi114.stdout: create 222/39 file after the snapshot 2024-05-27T09:30:25.955 INFO:tasks.workunit.client.0.smithi114.stdout: create 223/.snap/snap-subdir-test 2024-05-27T09:30:25.998 INFO:tasks.workunit.client.0.smithi114.stdout: create 223/30 file after the snapshot 2024-05-27T09:30:26.000 INFO:tasks.workunit.client.0.smithi114.stdout: create 223/31 file after the snapshot 2024-05-27T09:30:26.001 INFO:tasks.workunit.client.0.smithi114.stdout: create 223/32 file after the snapshot 2024-05-27T09:30:26.003 INFO:tasks.workunit.client.0.smithi114.stdout: create 223/33 file after the snapshot 2024-05-27T09:30:26.005 INFO:tasks.workunit.client.0.smithi114.stdout: create 223/34 file after the snapshot 2024-05-27T09:30:26.007 INFO:tasks.workunit.client.0.smithi114.stdout: create 223/35 file after the snapshot 2024-05-27T09:30:26.009 INFO:tasks.workunit.client.0.smithi114.stdout: create 223/36 file after the snapshot 2024-05-27T09:30:26.011 INFO:tasks.workunit.client.0.smithi114.stdout: create 223/37 file after the snapshot 2024-05-27T09:30:26.013 INFO:tasks.workunit.client.0.smithi114.stdout: create 223/38 file after the snapshot 2024-05-27T09:30:26.015 INFO:tasks.workunit.client.0.smithi114.stdout: create 223/39 file after the snapshot 2024-05-27T09:30:26.018 INFO:tasks.workunit.client.0.smithi114.stdout: create 224/.snap/snap-subdir-test 2024-05-27T09:30:26.071 INFO:tasks.workunit.client.0.smithi114.stdout: create 224/30 file after the snapshot 2024-05-27T09:30:26.073 INFO:tasks.workunit.client.0.smithi114.stdout: create 224/31 file after the snapshot 2024-05-27T09:30:26.075 INFO:tasks.workunit.client.0.smithi114.stdout: create 224/32 file after the snapshot 2024-05-27T09:30:26.078 INFO:tasks.workunit.client.0.smithi114.stdout: create 224/33 file after the snapshot 2024-05-27T09:30:26.081 INFO:tasks.workunit.client.0.smithi114.stdout: create 224/34 file after the snapshot 2024-05-27T09:30:26.083 INFO:tasks.workunit.client.0.smithi114.stdout: create 224/35 file after the snapshot 2024-05-27T09:30:26.086 INFO:tasks.workunit.client.0.smithi114.stdout: create 224/36 file after the snapshot 2024-05-27T09:30:26.089 INFO:tasks.workunit.client.0.smithi114.stdout: create 224/37 file after the snapshot 2024-05-27T09:30:26.092 INFO:tasks.workunit.client.0.smithi114.stdout: create 224/38 file after the snapshot 2024-05-27T09:30:26.095 INFO:tasks.workunit.client.0.smithi114.stdout: create 224/39 file after the snapshot 2024-05-27T09:30:26.097 INFO:tasks.workunit.client.0.smithi114.stdout: create 225/.snap/snap-subdir-test 2024-05-27T09:30:26.154 INFO:tasks.workunit.client.0.smithi114.stdout: create 225/30 file after the snapshot 2024-05-27T09:30:26.156 INFO:tasks.workunit.client.0.smithi114.stdout: create 225/31 file after the snapshot 2024-05-27T09:30:26.158 INFO:tasks.workunit.client.0.smithi114.stdout: create 225/32 file after the snapshot 2024-05-27T09:30:26.160 INFO:tasks.workunit.client.0.smithi114.stdout: create 225/33 file after the snapshot 2024-05-27T09:30:26.162 INFO:tasks.workunit.client.0.smithi114.stdout: create 225/34 file after the snapshot 2024-05-27T09:30:26.164 INFO:tasks.workunit.client.0.smithi114.stdout: create 225/35 file after the snapshot 2024-05-27T09:30:26.166 INFO:tasks.workunit.client.0.smithi114.stdout: create 225/36 file after the snapshot 2024-05-27T09:30:26.168 INFO:tasks.workunit.client.0.smithi114.stdout: create 225/37 file after the snapshot 2024-05-27T09:30:26.170 INFO:tasks.workunit.client.0.smithi114.stdout: create 225/38 file after the snapshot 2024-05-27T09:30:26.172 INFO:tasks.workunit.client.0.smithi114.stdout: create 225/39 file after the snapshot 2024-05-27T09:30:26.174 INFO:tasks.workunit.client.0.smithi114.stdout: create 226/.snap/snap-subdir-test 2024-05-27T09:30:26.220 INFO:tasks.workunit.client.0.smithi114.stdout: create 226/30 file after the snapshot 2024-05-27T09:30:26.222 INFO:tasks.workunit.client.0.smithi114.stdout: create 226/31 file after the snapshot 2024-05-27T09:30:26.223 INFO:tasks.workunit.client.0.smithi114.stdout: create 226/32 file after the snapshot 2024-05-27T09:30:26.225 INFO:tasks.workunit.client.0.smithi114.stdout: create 226/33 file after the snapshot 2024-05-27T09:30:26.226 INFO:tasks.workunit.client.0.smithi114.stdout: create 226/34 file after the snapshot 2024-05-27T09:30:26.228 INFO:tasks.workunit.client.0.smithi114.stdout: create 226/35 file after the snapshot 2024-05-27T09:30:26.230 INFO:tasks.workunit.client.0.smithi114.stdout: create 226/36 file after the snapshot 2024-05-27T09:30:26.231 INFO:tasks.workunit.client.0.smithi114.stdout: create 226/37 file after the snapshot 2024-05-27T09:30:26.234 INFO:tasks.workunit.client.0.smithi114.stdout: create 226/38 file after the snapshot 2024-05-27T09:30:26.236 INFO:tasks.workunit.client.0.smithi114.stdout: create 226/39 file after the snapshot 2024-05-27T09:30:26.239 INFO:tasks.workunit.client.0.smithi114.stdout: create 227/.snap/snap-subdir-test 2024-05-27T09:30:26.292 INFO:tasks.workunit.client.0.smithi114.stdout: create 227/30 file after the snapshot 2024-05-27T09:30:26.295 INFO:tasks.workunit.client.0.smithi114.stdout: create 227/31 file after the snapshot 2024-05-27T09:30:26.297 INFO:tasks.workunit.client.0.smithi114.stdout: create 227/32 file after the snapshot 2024-05-27T09:30:26.300 INFO:tasks.workunit.client.0.smithi114.stdout: create 227/33 file after the snapshot 2024-05-27T09:30:26.302 INFO:tasks.workunit.client.0.smithi114.stdout: create 227/34 file after the snapshot 2024-05-27T09:30:26.305 INFO:tasks.workunit.client.0.smithi114.stdout: create 227/35 file after the snapshot 2024-05-27T09:30:26.307 INFO:tasks.workunit.client.0.smithi114.stdout: create 227/36 file after the snapshot 2024-05-27T09:30:26.310 INFO:tasks.workunit.client.0.smithi114.stdout: create 227/37 file after the snapshot 2024-05-27T09:30:26.313 INFO:tasks.workunit.client.0.smithi114.stdout: create 227/38 file after the snapshot 2024-05-27T09:30:26.316 INFO:tasks.workunit.client.0.smithi114.stdout: create 227/39 file after the snapshot 2024-05-27T09:30:26.318 INFO:tasks.workunit.client.0.smithi114.stdout: create 228/.snap/snap-subdir-test 2024-05-27T09:30:26.369 INFO:tasks.workunit.client.0.smithi114.stdout: create 228/30 file after the snapshot 2024-05-27T09:30:26.372 INFO:tasks.workunit.client.0.smithi114.stdout: create 228/31 file after the snapshot 2024-05-27T09:30:26.375 INFO:tasks.workunit.client.0.smithi114.stdout: create 228/32 file after the snapshot 2024-05-27T09:30:26.377 INFO:tasks.workunit.client.0.smithi114.stdout: create 228/33 file after the snapshot 2024-05-27T09:30:26.380 INFO:tasks.workunit.client.0.smithi114.stdout: create 228/34 file after the snapshot 2024-05-27T09:30:26.382 INFO:tasks.workunit.client.0.smithi114.stdout: create 228/35 file after the snapshot 2024-05-27T09:30:26.385 INFO:tasks.workunit.client.0.smithi114.stdout: create 228/36 file after the snapshot 2024-05-27T09:30:26.388 INFO:tasks.workunit.client.0.smithi114.stdout: create 228/37 file after the snapshot 2024-05-27T09:30:26.391 INFO:tasks.workunit.client.0.smithi114.stdout: create 228/38 file after the snapshot 2024-05-27T09:30:26.394 INFO:tasks.workunit.client.0.smithi114.stdout: create 228/39 file after the snapshot 2024-05-27T09:30:26.396 INFO:tasks.workunit.client.0.smithi114.stdout: create 229/.snap/snap-subdir-test 2024-05-27T09:30:26.449 INFO:tasks.workunit.client.0.smithi114.stdout: create 229/30 file after the snapshot 2024-05-27T09:30:26.452 INFO:tasks.workunit.client.0.smithi114.stdout: create 229/31 file after the snapshot 2024-05-27T09:30:26.455 INFO:tasks.workunit.client.0.smithi114.stdout: create 229/32 file after the snapshot 2024-05-27T09:30:26.458 INFO:tasks.workunit.client.0.smithi114.stdout: create 229/33 file after the snapshot 2024-05-27T09:30:26.461 INFO:tasks.workunit.client.0.smithi114.stdout: create 229/34 file after the snapshot 2024-05-27T09:30:26.464 INFO:tasks.workunit.client.0.smithi114.stdout: create 229/35 file after the snapshot 2024-05-27T09:30:26.467 INFO:tasks.workunit.client.0.smithi114.stdout: create 229/36 file after the snapshot 2024-05-27T09:30:26.470 INFO:tasks.workunit.client.0.smithi114.stdout: create 229/37 file after the snapshot 2024-05-27T09:30:26.473 INFO:tasks.workunit.client.0.smithi114.stdout: create 229/38 file after the snapshot 2024-05-27T09:30:26.475 INFO:tasks.workunit.client.0.smithi114.stdout: create 229/39 file after the snapshot 2024-05-27T09:30:26.478 INFO:tasks.workunit.client.0.smithi114.stdout: create 230/.snap/snap-subdir-test 2024-05-27T09:30:26.533 INFO:tasks.workunit.client.0.smithi114.stdout: create 230/30 file after the snapshot 2024-05-27T09:30:26.534 INFO:tasks.workunit.client.0.smithi114.stdout: create 230/31 file after the snapshot 2024-05-27T09:30:26.536 INFO:tasks.workunit.client.0.smithi114.stdout: create 230/32 file after the snapshot 2024-05-27T09:30:26.538 INFO:tasks.workunit.client.0.smithi114.stdout: create 230/33 file after the snapshot 2024-05-27T09:30:26.539 INFO:tasks.workunit.client.0.smithi114.stdout: create 230/34 file after the snapshot 2024-05-27T09:30:26.542 INFO:tasks.workunit.client.0.smithi114.stdout: create 230/35 file after the snapshot 2024-05-27T09:30:26.543 INFO:tasks.workunit.client.0.smithi114.stdout: create 230/36 file after the snapshot 2024-05-27T09:30:26.546 INFO:tasks.workunit.client.0.smithi114.stdout: create 230/37 file after the snapshot 2024-05-27T09:30:26.548 INFO:tasks.workunit.client.0.smithi114.stdout: create 230/38 file after the snapshot 2024-05-27T09:30:26.551 INFO:tasks.workunit.client.0.smithi114.stdout: create 230/39 file after the snapshot 2024-05-27T09:30:26.554 INFO:tasks.workunit.client.0.smithi114.stdout: create 231/.snap/snap-subdir-test 2024-05-27T09:30:26.608 INFO:tasks.workunit.client.0.smithi114.stdout: create 231/30 file after the snapshot 2024-05-27T09:30:26.611 INFO:tasks.workunit.client.0.smithi114.stdout: create 231/31 file after the snapshot 2024-05-27T09:30:26.614 INFO:tasks.workunit.client.0.smithi114.stdout: create 231/32 file after the snapshot 2024-05-27T09:30:26.617 INFO:tasks.workunit.client.0.smithi114.stdout: create 231/33 file after the snapshot 2024-05-27T09:30:26.620 INFO:tasks.workunit.client.0.smithi114.stdout: create 231/34 file after the snapshot 2024-05-27T09:30:26.623 INFO:tasks.workunit.client.0.smithi114.stdout: create 231/35 file after the snapshot 2024-05-27T09:30:26.625 INFO:tasks.workunit.client.0.smithi114.stdout: create 231/36 file after the snapshot 2024-05-27T09:30:26.628 INFO:tasks.workunit.client.0.smithi114.stdout: create 231/37 file after the snapshot 2024-05-27T09:30:26.631 INFO:tasks.workunit.client.0.smithi114.stdout: create 231/38 file after the snapshot 2024-05-27T09:30:26.634 INFO:tasks.workunit.client.0.smithi114.stdout: create 231/39 file after the snapshot 2024-05-27T09:30:26.636 INFO:tasks.workunit.client.0.smithi114.stdout: create 232/.snap/snap-subdir-test 2024-05-27T09:30:26.694 INFO:tasks.workunit.client.0.smithi114.stdout: create 232/30 file after the snapshot 2024-05-27T09:30:26.697 INFO:tasks.workunit.client.0.smithi114.stdout: create 232/31 file after the snapshot 2024-05-27T09:30:26.700 INFO:tasks.workunit.client.0.smithi114.stdout: create 232/32 file after the snapshot 2024-05-27T09:30:26.703 INFO:tasks.workunit.client.0.smithi114.stdout: create 232/33 file after the snapshot 2024-05-27T09:30:26.705 INFO:tasks.workunit.client.0.smithi114.stdout: create 232/34 file after the snapshot 2024-05-27T09:30:26.708 INFO:tasks.workunit.client.0.smithi114.stdout: create 232/35 file after the snapshot 2024-05-27T09:30:26.711 INFO:tasks.workunit.client.0.smithi114.stdout: create 232/36 file after the snapshot 2024-05-27T09:30:26.714 INFO:tasks.workunit.client.0.smithi114.stdout: create 232/37 file after the snapshot 2024-05-27T09:30:26.716 INFO:tasks.workunit.client.0.smithi114.stdout: create 232/38 file after the snapshot 2024-05-27T09:30:26.719 INFO:tasks.workunit.client.0.smithi114.stdout: create 232/39 file after the snapshot 2024-05-27T09:30:26.722 INFO:tasks.workunit.client.0.smithi114.stdout: create 233/.snap/snap-subdir-test 2024-05-27T09:30:26.779 INFO:tasks.workunit.client.0.smithi114.stdout: create 233/30 file after the snapshot 2024-05-27T09:30:26.783 INFO:tasks.workunit.client.0.smithi114.stdout: create 233/31 file after the snapshot 2024-05-27T09:30:26.786 INFO:tasks.workunit.client.0.smithi114.stdout: create 233/32 file after the snapshot 2024-05-27T09:30:26.788 INFO:tasks.workunit.client.0.smithi114.stdout: create 233/33 file after the snapshot 2024-05-27T09:30:26.793 INFO:tasks.workunit.client.0.smithi114.stdout: create 233/34 file after the snapshot 2024-05-27T09:30:26.794 INFO:tasks.workunit.client.0.smithi114.stdout: create 233/35 file after the snapshot 2024-05-27T09:30:26.797 INFO:tasks.workunit.client.0.smithi114.stdout: create 233/36 file after the snapshot 2024-05-27T09:30:26.800 INFO:tasks.workunit.client.0.smithi114.stdout: create 233/37 file after the snapshot 2024-05-27T09:30:26.802 INFO:tasks.workunit.client.0.smithi114.stdout: create 233/38 file after the snapshot 2024-05-27T09:30:26.805 INFO:tasks.workunit.client.0.smithi114.stdout: create 233/39 file after the snapshot 2024-05-27T09:30:26.808 INFO:tasks.workunit.client.0.smithi114.stdout: create 234/.snap/snap-subdir-test 2024-05-27T09:30:26.847 INFO:tasks.workunit.client.0.smithi114.stdout: create 234/30 file after the snapshot 2024-05-27T09:30:26.851 INFO:tasks.workunit.client.0.smithi114.stdout: create 234/31 file after the snapshot 2024-05-27T09:30:26.856 INFO:tasks.workunit.client.0.smithi114.stdout: create 234/32 file after the snapshot 2024-05-27T09:30:26.860 INFO:tasks.workunit.client.0.smithi114.stdout: create 234/33 file after the snapshot 2024-05-27T09:30:26.864 INFO:tasks.workunit.client.0.smithi114.stdout: create 234/34 file after the snapshot 2024-05-27T09:30:26.869 INFO:tasks.workunit.client.0.smithi114.stdout: create 234/35 file after the snapshot 2024-05-27T09:30:26.873 INFO:tasks.workunit.client.0.smithi114.stdout: create 234/36 file after the snapshot 2024-05-27T09:30:26.877 INFO:tasks.workunit.client.0.smithi114.stdout: create 234/37 file after the snapshot 2024-05-27T09:30:26.882 INFO:tasks.workunit.client.0.smithi114.stdout: create 234/38 file after the snapshot 2024-05-27T09:30:26.886 INFO:tasks.workunit.client.0.smithi114.stdout: create 234/39 file after the snapshot 2024-05-27T09:30:26.890 INFO:tasks.workunit.client.0.smithi114.stdout: create 235/.snap/snap-subdir-test 2024-05-27T09:30:26.934 INFO:tasks.workunit.client.0.smithi114.stdout: create 235/30 file after the snapshot 2024-05-27T09:30:26.935 INFO:tasks.workunit.client.0.smithi114.stdout: create 235/31 file after the snapshot 2024-05-27T09:30:26.937 INFO:tasks.workunit.client.0.smithi114.stdout: create 235/32 file after the snapshot 2024-05-27T09:30:26.939 INFO:tasks.workunit.client.0.smithi114.stdout: create 235/33 file after the snapshot 2024-05-27T09:30:26.941 INFO:tasks.workunit.client.0.smithi114.stdout: create 235/34 file after the snapshot 2024-05-27T09:30:26.943 INFO:tasks.workunit.client.0.smithi114.stdout: create 235/35 file after the snapshot 2024-05-27T09:30:26.945 INFO:tasks.workunit.client.0.smithi114.stdout: create 235/36 file after the snapshot 2024-05-27T09:30:26.947 INFO:tasks.workunit.client.0.smithi114.stdout: create 235/37 file after the snapshot 2024-05-27T09:30:26.949 INFO:tasks.workunit.client.0.smithi114.stdout: create 235/38 file after the snapshot 2024-05-27T09:30:26.951 INFO:tasks.workunit.client.0.smithi114.stdout: create 235/39 file after the snapshot 2024-05-27T09:30:26.954 INFO:tasks.workunit.client.0.smithi114.stdout: create 236/.snap/snap-subdir-test 2024-05-27T09:30:26.997 INFO:tasks.workunit.client.0.smithi114.stdout: create 236/30 file after the snapshot 2024-05-27T09:30:26.999 INFO:tasks.workunit.client.0.smithi114.stdout: create 236/31 file after the snapshot 2024-05-27T09:30:27.002 INFO:tasks.workunit.client.0.smithi114.stdout: create 236/32 file after the snapshot 2024-05-27T09:30:27.005 INFO:tasks.workunit.client.0.smithi114.stdout: create 236/33 file after the snapshot 2024-05-27T09:30:27.007 INFO:tasks.workunit.client.0.smithi114.stdout: create 236/34 file after the snapshot 2024-05-27T09:30:27.010 INFO:tasks.workunit.client.0.smithi114.stdout: create 236/35 file after the snapshot 2024-05-27T09:30:27.013 INFO:tasks.workunit.client.0.smithi114.stdout: create 236/36 file after the snapshot 2024-05-27T09:30:27.016 INFO:tasks.workunit.client.0.smithi114.stdout: create 236/37 file after the snapshot 2024-05-27T09:30:27.019 INFO:tasks.workunit.client.0.smithi114.stdout: create 236/38 file after the snapshot 2024-05-27T09:30:27.022 INFO:tasks.workunit.client.0.smithi114.stdout: create 236/39 file after the snapshot 2024-05-27T09:30:27.024 INFO:tasks.workunit.client.0.smithi114.stdout: create 237/.snap/snap-subdir-test 2024-05-27T09:30:27.063 INFO:tasks.workunit.client.0.smithi114.stdout: create 237/30 file after the snapshot 2024-05-27T09:30:27.065 INFO:tasks.workunit.client.0.smithi114.stdout: create 237/31 file after the snapshot 2024-05-27T09:30:27.068 INFO:tasks.workunit.client.0.smithi114.stdout: create 237/32 file after the snapshot 2024-05-27T09:30:27.071 INFO:tasks.workunit.client.0.smithi114.stdout: create 237/33 file after the snapshot 2024-05-27T09:30:27.073 INFO:tasks.workunit.client.0.smithi114.stdout: create 237/34 file after the snapshot 2024-05-27T09:30:27.076 INFO:tasks.workunit.client.0.smithi114.stdout: create 237/35 file after the snapshot 2024-05-27T09:30:27.079 INFO:tasks.workunit.client.0.smithi114.stdout: create 237/36 file after the snapshot 2024-05-27T09:30:27.082 INFO:tasks.workunit.client.0.smithi114.stdout: create 237/37 file after the snapshot 2024-05-27T09:30:27.085 INFO:tasks.workunit.client.0.smithi114.stdout: create 237/38 file after the snapshot 2024-05-27T09:30:27.087 INFO:tasks.workunit.client.0.smithi114.stdout: create 237/39 file after the snapshot 2024-05-27T09:30:27.090 INFO:tasks.workunit.client.0.smithi114.stdout: create 238/.snap/snap-subdir-test 2024-05-27T09:30:27.135 INFO:tasks.workunit.client.0.smithi114.stdout: create 238/30 file after the snapshot 2024-05-27T09:30:27.138 INFO:tasks.workunit.client.0.smithi114.stdout: create 238/31 file after the snapshot 2024-05-27T09:30:27.140 INFO:tasks.workunit.client.0.smithi114.stdout: create 238/32 file after the snapshot 2024-05-27T09:30:27.142 INFO:tasks.workunit.client.0.smithi114.stdout: create 238/33 file after the snapshot 2024-05-27T09:30:27.143 INFO:tasks.workunit.client.0.smithi114.stdout: create 238/34 file after the snapshot 2024-05-27T09:30:27.145 INFO:tasks.workunit.client.0.smithi114.stdout: create 238/35 file after the snapshot 2024-05-27T09:30:27.146 INFO:tasks.workunit.client.0.smithi114.stdout: create 238/36 file after the snapshot 2024-05-27T09:30:27.148 INFO:tasks.workunit.client.0.smithi114.stdout: create 238/37 file after the snapshot 2024-05-27T09:30:27.149 INFO:tasks.workunit.client.0.smithi114.stdout: create 238/38 file after the snapshot 2024-05-27T09:30:27.151 INFO:tasks.workunit.client.0.smithi114.stdout: create 238/39 file after the snapshot 2024-05-27T09:30:27.152 INFO:tasks.workunit.client.0.smithi114.stdout: create 239/.snap/snap-subdir-test 2024-05-27T09:30:27.196 INFO:tasks.workunit.client.0.smithi114.stdout: create 239/30 file after the snapshot 2024-05-27T09:30:27.199 INFO:tasks.workunit.client.0.smithi114.stdout: create 239/31 file after the snapshot 2024-05-27T09:30:27.202 INFO:tasks.workunit.client.0.smithi114.stdout: create 239/32 file after the snapshot 2024-05-27T09:30:27.205 INFO:tasks.workunit.client.0.smithi114.stdout: create 239/33 file after the snapshot 2024-05-27T09:30:27.208 INFO:tasks.workunit.client.0.smithi114.stdout: create 239/34 file after the snapshot 2024-05-27T09:30:27.211 INFO:tasks.workunit.client.0.smithi114.stdout: create 239/35 file after the snapshot 2024-05-27T09:30:27.214 INFO:tasks.workunit.client.0.smithi114.stdout: create 239/36 file after the snapshot 2024-05-27T09:30:27.217 INFO:tasks.workunit.client.0.smithi114.stdout: create 239/37 file after the snapshot 2024-05-27T09:30:27.219 INFO:tasks.workunit.client.0.smithi114.stdout: create 239/38 file after the snapshot 2024-05-27T09:30:27.222 INFO:tasks.workunit.client.0.smithi114.stdout: create 239/39 file after the snapshot 2024-05-27T09:30:27.225 INFO:tasks.workunit.client.0.smithi114.stdout: create 240/.snap/snap-subdir-test 2024-05-27T09:30:27.268 INFO:tasks.workunit.client.0.smithi114.stdout: create 240/30 file after the snapshot 2024-05-27T09:30:27.269 INFO:tasks.workunit.client.0.smithi114.stdout: create 240/31 file after the snapshot 2024-05-27T09:30:27.271 INFO:tasks.workunit.client.0.smithi114.stdout: create 240/32 file after the snapshot 2024-05-27T09:30:27.273 INFO:tasks.workunit.client.0.smithi114.stdout: create 240/33 file after the snapshot 2024-05-27T09:30:27.276 INFO:tasks.workunit.client.0.smithi114.stdout: create 240/34 file after the snapshot 2024-05-27T09:30:27.278 INFO:tasks.workunit.client.0.smithi114.stdout: create 240/35 file after the snapshot 2024-05-27T09:30:27.280 INFO:tasks.workunit.client.0.smithi114.stdout: create 240/36 file after the snapshot 2024-05-27T09:30:27.282 INFO:tasks.workunit.client.0.smithi114.stdout: create 240/37 file after the snapshot 2024-05-27T09:30:27.284 INFO:tasks.workunit.client.0.smithi114.stdout: create 240/38 file after the snapshot 2024-05-27T09:30:27.286 INFO:tasks.workunit.client.0.smithi114.stdout: create 240/39 file after the snapshot 2024-05-27T09:30:27.288 INFO:tasks.workunit.client.0.smithi114.stdout: create 241/.snap/snap-subdir-test 2024-05-27T09:30:27.330 INFO:tasks.workunit.client.0.smithi114.stdout: create 241/30 file after the snapshot 2024-05-27T09:30:27.332 INFO:tasks.workunit.client.0.smithi114.stdout: create 241/31 file after the snapshot 2024-05-27T09:30:27.335 INFO:tasks.workunit.client.0.smithi114.stdout: create 241/32 file after the snapshot 2024-05-27T09:30:27.338 INFO:tasks.workunit.client.0.smithi114.stdout: create 241/33 file after the snapshot 2024-05-27T09:30:27.341 INFO:tasks.workunit.client.0.smithi114.stdout: create 241/34 file after the snapshot 2024-05-27T09:30:27.343 INFO:tasks.workunit.client.0.smithi114.stdout: create 241/35 file after the snapshot 2024-05-27T09:30:27.346 INFO:tasks.workunit.client.0.smithi114.stdout: create 241/36 file after the snapshot 2024-05-27T09:30:27.349 INFO:tasks.workunit.client.0.smithi114.stdout: create 241/37 file after the snapshot 2024-05-27T09:30:27.352 INFO:tasks.workunit.client.0.smithi114.stdout: create 241/38 file after the snapshot 2024-05-27T09:30:27.354 INFO:tasks.workunit.client.0.smithi114.stdout: create 241/39 file after the snapshot 2024-05-27T09:30:27.357 INFO:tasks.workunit.client.0.smithi114.stdout: create 242/.snap/snap-subdir-test 2024-05-27T09:30:27.396 INFO:tasks.workunit.client.0.smithi114.stdout: create 242/30 file after the snapshot 2024-05-27T09:30:27.399 INFO:tasks.workunit.client.0.smithi114.stdout: create 242/31 file after the snapshot 2024-05-27T09:30:27.401 INFO:tasks.workunit.client.0.smithi114.stdout: create 242/32 file after the snapshot 2024-05-27T09:30:27.405 INFO:tasks.workunit.client.0.smithi114.stdout: create 242/33 file after the snapshot 2024-05-27T09:30:27.407 INFO:tasks.workunit.client.0.smithi114.stdout: create 242/34 file after the snapshot 2024-05-27T09:30:27.410 INFO:tasks.workunit.client.0.smithi114.stdout: create 242/35 file after the snapshot 2024-05-27T09:30:27.414 INFO:tasks.workunit.client.0.smithi114.stdout: create 242/36 file after the snapshot 2024-05-27T09:30:27.417 INFO:tasks.workunit.client.0.smithi114.stdout: create 242/37 file after the snapshot 2024-05-27T09:30:27.420 INFO:tasks.workunit.client.0.smithi114.stdout: create 242/38 file after the snapshot 2024-05-27T09:30:27.423 INFO:tasks.workunit.client.0.smithi114.stdout: create 242/39 file after the snapshot 2024-05-27T09:30:27.426 INFO:tasks.workunit.client.0.smithi114.stdout: create 243/.snap/snap-subdir-test 2024-05-27T09:30:27.468 INFO:tasks.workunit.client.0.smithi114.stdout: create 243/30 file after the snapshot 2024-05-27T09:30:27.470 INFO:tasks.workunit.client.0.smithi114.stdout: create 243/31 file after the snapshot 2024-05-27T09:30:27.473 INFO:tasks.workunit.client.0.smithi114.stdout: create 243/32 file after the snapshot 2024-05-27T09:30:27.476 INFO:tasks.workunit.client.0.smithi114.stdout: create 243/33 file after the snapshot 2024-05-27T09:30:27.479 INFO:tasks.workunit.client.0.smithi114.stdout: create 243/34 file after the snapshot 2024-05-27T09:30:27.481 INFO:tasks.workunit.client.0.smithi114.stdout: create 243/35 file after the snapshot 2024-05-27T09:30:27.484 INFO:tasks.workunit.client.0.smithi114.stdout: create 243/36 file after the snapshot 2024-05-27T09:30:27.487 INFO:tasks.workunit.client.0.smithi114.stdout: create 243/37 file after the snapshot 2024-05-27T09:30:27.489 INFO:tasks.workunit.client.0.smithi114.stdout: create 243/38 file after the snapshot 2024-05-27T09:30:27.492 INFO:tasks.workunit.client.0.smithi114.stdout: create 243/39 file after the snapshot 2024-05-27T09:30:27.495 INFO:tasks.workunit.client.0.smithi114.stdout: create 244/.snap/snap-subdir-test 2024-05-27T09:30:27.537 INFO:tasks.workunit.client.0.smithi114.stdout: create 244/30 file after the snapshot 2024-05-27T09:30:27.540 INFO:tasks.workunit.client.0.smithi114.stdout: create 244/31 file after the snapshot 2024-05-27T09:30:27.543 INFO:tasks.workunit.client.0.smithi114.stdout: create 244/32 file after the snapshot 2024-05-27T09:30:27.546 INFO:tasks.workunit.client.0.smithi114.stdout: create 244/33 file after the snapshot 2024-05-27T09:30:27.549 INFO:tasks.workunit.client.0.smithi114.stdout: create 244/34 file after the snapshot 2024-05-27T09:30:27.552 INFO:tasks.workunit.client.0.smithi114.stdout: create 244/35 file after the snapshot 2024-05-27T09:30:27.555 INFO:tasks.workunit.client.0.smithi114.stdout: create 244/36 file after the snapshot 2024-05-27T09:30:27.558 INFO:tasks.workunit.client.0.smithi114.stdout: create 244/37 file after the snapshot 2024-05-27T09:30:27.560 INFO:tasks.workunit.client.0.smithi114.stdout: create 244/38 file after the snapshot 2024-05-27T09:30:27.563 INFO:tasks.workunit.client.0.smithi114.stdout: create 244/39 file after the snapshot 2024-05-27T09:30:27.566 INFO:tasks.workunit.client.0.smithi114.stdout: create 245/.snap/snap-subdir-test 2024-05-27T09:30:27.602 INFO:tasks.workunit.client.0.smithi114.stdout: create 245/30 file after the snapshot 2024-05-27T09:30:27.604 INFO:tasks.workunit.client.0.smithi114.stdout: create 245/31 file after the snapshot 2024-05-27T09:30:27.606 INFO:tasks.workunit.client.0.smithi114.stdout: create 245/32 file after the snapshot 2024-05-27T09:30:27.609 INFO:tasks.workunit.client.0.smithi114.stdout: create 245/33 file after the snapshot 2024-05-27T09:30:27.609 INFO:tasks.workunit.client.0.smithi114.stdout: create 245/34 file after the snapshot 2024-05-27T09:30:27.610 INFO:tasks.workunit.client.0.smithi114.stdout: create 245/35 file after the snapshot 2024-05-27T09:30:27.612 INFO:tasks.workunit.client.0.smithi114.stdout: create 245/36 file after the snapshot 2024-05-27T09:30:27.613 INFO:tasks.workunit.client.0.smithi114.stdout: create 245/37 file after the snapshot 2024-05-27T09:30:27.616 INFO:tasks.workunit.client.0.smithi114.stdout: create 245/38 file after the snapshot 2024-05-27T09:30:27.619 INFO:tasks.workunit.client.0.smithi114.stdout: create 245/39 file after the snapshot 2024-05-27T09:30:27.621 INFO:tasks.workunit.client.0.smithi114.stdout: create 246/.snap/snap-subdir-test 2024-05-27T09:30:27.664 INFO:tasks.workunit.client.0.smithi114.stdout: create 246/30 file after the snapshot 2024-05-27T09:30:27.667 INFO:tasks.workunit.client.0.smithi114.stdout: create 246/31 file after the snapshot 2024-05-27T09:30:27.670 INFO:tasks.workunit.client.0.smithi114.stdout: create 246/32 file after the snapshot 2024-05-27T09:30:27.673 INFO:tasks.workunit.client.0.smithi114.stdout: create 246/33 file after the snapshot 2024-05-27T09:30:27.676 INFO:tasks.workunit.client.0.smithi114.stdout: create 246/34 file after the snapshot 2024-05-27T09:30:27.679 INFO:tasks.workunit.client.0.smithi114.stdout: create 246/35 file after the snapshot 2024-05-27T09:30:27.682 INFO:tasks.workunit.client.0.smithi114.stdout: create 246/36 file after the snapshot 2024-05-27T09:30:27.685 INFO:tasks.workunit.client.0.smithi114.stdout: create 246/37 file after the snapshot 2024-05-27T09:30:27.688 INFO:tasks.workunit.client.0.smithi114.stdout: create 246/38 file after the snapshot 2024-05-27T09:30:27.691 INFO:tasks.workunit.client.0.smithi114.stdout: create 246/39 file after the snapshot 2024-05-27T09:30:27.693 INFO:tasks.workunit.client.0.smithi114.stdout: create 247/.snap/snap-subdir-test 2024-05-27T09:30:27.736 INFO:tasks.workunit.client.0.smithi114.stdout: create 247/30 file after the snapshot 2024-05-27T09:30:27.738 INFO:tasks.workunit.client.0.smithi114.stdout: create 247/31 file after the snapshot 2024-05-27T09:30:27.741 INFO:tasks.workunit.client.0.smithi114.stdout: create 247/32 file after the snapshot 2024-05-27T09:30:27.743 INFO:tasks.workunit.client.0.smithi114.stdout: create 247/33 file after the snapshot 2024-05-27T09:30:27.746 INFO:tasks.workunit.client.0.smithi114.stdout: create 247/34 file after the snapshot 2024-05-27T09:30:27.748 INFO:tasks.workunit.client.0.smithi114.stdout: create 247/35 file after the snapshot 2024-05-27T09:30:27.751 INFO:tasks.workunit.client.0.smithi114.stdout: create 247/36 file after the snapshot 2024-05-27T09:30:27.753 INFO:tasks.workunit.client.0.smithi114.stdout: create 247/37 file after the snapshot 2024-05-27T09:30:27.756 INFO:tasks.workunit.client.0.smithi114.stdout: create 247/38 file after the snapshot 2024-05-27T09:30:27.759 INFO:tasks.workunit.client.0.smithi114.stdout: create 247/39 file after the snapshot 2024-05-27T09:30:27.761 INFO:tasks.workunit.client.0.smithi114.stdout: create 248/.snap/snap-subdir-test 2024-05-27T09:30:27.799 INFO:tasks.workunit.client.0.smithi114.stdout: create 248/30 file after the snapshot 2024-05-27T09:30:27.802 INFO:tasks.workunit.client.0.smithi114.stdout: create 248/31 file after the snapshot 2024-05-27T09:30:27.804 INFO:tasks.workunit.client.0.smithi114.stdout: create 248/32 file after the snapshot 2024-05-27T09:30:27.807 INFO:tasks.workunit.client.0.smithi114.stdout: create 248/33 file after the snapshot 2024-05-27T09:30:27.809 INFO:tasks.workunit.client.0.smithi114.stdout: create 248/34 file after the snapshot 2024-05-27T09:30:27.811 INFO:tasks.workunit.client.0.smithi114.stdout: create 248/35 file after the snapshot 2024-05-27T09:30:27.814 INFO:tasks.workunit.client.0.smithi114.stdout: create 248/36 file after the snapshot 2024-05-27T09:30:27.817 INFO:tasks.workunit.client.0.smithi114.stdout: create 248/37 file after the snapshot 2024-05-27T09:30:27.820 INFO:tasks.workunit.client.0.smithi114.stdout: create 248/38 file after the snapshot 2024-05-27T09:30:27.822 INFO:tasks.workunit.client.0.smithi114.stdout: create 248/39 file after the snapshot 2024-05-27T09:30:27.825 INFO:tasks.workunit.client.0.smithi114.stdout: create 249/.snap/snap-subdir-test 2024-05-27T09:30:27.868 INFO:tasks.workunit.client.0.smithi114.stdout: create 249/30 file after the snapshot 2024-05-27T09:30:27.870 INFO:tasks.workunit.client.0.smithi114.stdout: create 249/31 file after the snapshot 2024-05-27T09:30:27.872 INFO:tasks.workunit.client.0.smithi114.stdout: create 249/32 file after the snapshot 2024-05-27T09:30:27.874 INFO:tasks.workunit.client.0.smithi114.stdout: create 249/33 file after the snapshot 2024-05-27T09:30:27.876 INFO:tasks.workunit.client.0.smithi114.stdout: create 249/34 file after the snapshot 2024-05-27T09:30:27.878 INFO:tasks.workunit.client.0.smithi114.stdout: create 249/35 file after the snapshot 2024-05-27T09:30:27.880 INFO:tasks.workunit.client.0.smithi114.stdout: create 249/36 file after the snapshot 2024-05-27T09:30:27.882 INFO:tasks.workunit.client.0.smithi114.stdout: create 249/37 file after the snapshot 2024-05-27T09:30:27.884 INFO:tasks.workunit.client.0.smithi114.stdout: create 249/38 file after the snapshot 2024-05-27T09:30:27.886 INFO:tasks.workunit.client.0.smithi114.stdout: create 249/39 file after the snapshot 2024-05-27T09:30:27.888 INFO:tasks.workunit.client.0.smithi114.stdout: create 250/.snap/snap-subdir-test 2024-05-27T09:30:27.930 INFO:tasks.workunit.client.0.smithi114.stdout: create 250/30 file after the snapshot 2024-05-27T09:30:27.933 INFO:tasks.workunit.client.0.smithi114.stdout: create 250/31 file after the snapshot 2024-05-27T09:30:27.935 INFO:tasks.workunit.client.0.smithi114.stdout: create 250/32 file after the snapshot 2024-05-27T09:30:27.938 INFO:tasks.workunit.client.0.smithi114.stdout: create 250/33 file after the snapshot 2024-05-27T09:30:27.941 INFO:tasks.workunit.client.0.smithi114.stdout: create 250/34 file after the snapshot 2024-05-27T09:30:27.944 INFO:tasks.workunit.client.0.smithi114.stdout: create 250/35 file after the snapshot 2024-05-27T09:30:27.947 INFO:tasks.workunit.client.0.smithi114.stdout: create 250/36 file after the snapshot 2024-05-27T09:30:27.949 INFO:tasks.workunit.client.0.smithi114.stdout: create 250/37 file after the snapshot 2024-05-27T09:30:27.952 INFO:tasks.workunit.client.0.smithi114.stdout: create 250/38 file after the snapshot 2024-05-27T09:30:27.955 INFO:tasks.workunit.client.0.smithi114.stdout: create 250/39 file after the snapshot 2024-05-27T09:30:27.958 INFO:tasks.workunit.client.0.smithi114.stdout: create 251/.snap/snap-subdir-test 2024-05-27T09:30:27.994 INFO:tasks.workunit.client.0.smithi114.stdout: create 251/30 file after the snapshot 2024-05-27T09:30:27.995 INFO:tasks.workunit.client.0.smithi114.stdout: create 251/31 file after the snapshot 2024-05-27T09:30:27.997 INFO:tasks.workunit.client.0.smithi114.stdout: create 251/32 file after the snapshot 2024-05-27T09:30:27.998 INFO:tasks.workunit.client.0.smithi114.stdout: create 251/33 file after the snapshot 2024-05-27T09:30:28.000 INFO:tasks.workunit.client.0.smithi114.stdout: create 251/34 file after the snapshot 2024-05-27T09:30:28.001 INFO:tasks.workunit.client.0.smithi114.stdout: create 251/35 file after the snapshot 2024-05-27T09:30:28.003 INFO:tasks.workunit.client.0.smithi114.stdout: create 251/36 file after the snapshot 2024-05-27T09:30:28.004 INFO:tasks.workunit.client.0.smithi114.stdout: create 251/37 file after the snapshot 2024-05-27T09:30:28.006 INFO:tasks.workunit.client.0.smithi114.stdout: create 251/38 file after the snapshot 2024-05-27T09:30:28.008 INFO:tasks.workunit.client.0.smithi114.stdout: create 251/39 file after the snapshot 2024-05-27T09:30:28.009 INFO:tasks.workunit.client.0.smithi114.stdout: create 252/.snap/snap-subdir-test 2024-05-27T09:30:28.063 INFO:tasks.workunit.client.0.smithi114.stdout: create 252/30 file after the snapshot 2024-05-27T09:30:28.066 INFO:tasks.workunit.client.0.smithi114.stdout: create 252/31 file after the snapshot 2024-05-27T09:30:28.068 INFO:tasks.workunit.client.0.smithi114.stdout: create 252/32 file after the snapshot 2024-05-27T09:30:28.071 INFO:tasks.workunit.client.0.smithi114.stdout: create 252/33 file after the snapshot 2024-05-27T09:30:28.074 INFO:tasks.workunit.client.0.smithi114.stdout: create 252/34 file after the snapshot 2024-05-27T09:30:28.077 INFO:tasks.workunit.client.0.smithi114.stdout: create 252/35 file after the snapshot 2024-05-27T09:30:28.079 INFO:tasks.workunit.client.0.smithi114.stdout: create 252/36 file after the snapshot 2024-05-27T09:30:28.082 INFO:tasks.workunit.client.0.smithi114.stdout: create 252/37 file after the snapshot 2024-05-27T09:30:28.085 INFO:tasks.workunit.client.0.smithi114.stdout: create 252/38 file after the snapshot 2024-05-27T09:30:28.088 INFO:tasks.workunit.client.0.smithi114.stdout: create 252/39 file after the snapshot 2024-05-27T09:30:28.091 INFO:tasks.workunit.client.0.smithi114.stdout: create 253/.snap/snap-subdir-test 2024-05-27T09:30:28.152 INFO:tasks.workunit.client.0.smithi114.stdout: create 253/30 file after the snapshot 2024-05-27T09:30:28.155 INFO:tasks.workunit.client.0.smithi114.stdout: create 253/31 file after the snapshot 2024-05-27T09:30:28.157 INFO:tasks.workunit.client.0.smithi114.stdout: create 253/32 file after the snapshot 2024-05-27T09:30:28.160 INFO:tasks.workunit.client.0.smithi114.stdout: create 253/33 file after the snapshot 2024-05-27T09:30:28.163 INFO:tasks.workunit.client.0.smithi114.stdout: create 253/34 file after the snapshot 2024-05-27T09:30:28.166 INFO:tasks.workunit.client.0.smithi114.stdout: create 253/35 file after the snapshot 2024-05-27T09:30:28.168 INFO:tasks.workunit.client.0.smithi114.stdout: create 253/36 file after the snapshot 2024-05-27T09:30:28.171 INFO:tasks.workunit.client.0.smithi114.stdout: create 253/37 file after the snapshot 2024-05-27T09:30:28.174 INFO:tasks.workunit.client.0.smithi114.stdout: create 253/38 file after the snapshot 2024-05-27T09:30:28.176 INFO:tasks.workunit.client.0.smithi114.stdout: create 253/39 file after the snapshot 2024-05-27T09:30:28.179 INFO:tasks.workunit.client.0.smithi114.stdout: create 254/.snap/snap-subdir-test 2024-05-27T09:30:28.239 INFO:tasks.workunit.client.0.smithi114.stdout: create 254/30 file after the snapshot 2024-05-27T09:30:28.241 INFO:tasks.workunit.client.0.smithi114.stdout: create 254/31 file after the snapshot 2024-05-27T09:30:28.243 INFO:tasks.workunit.client.0.smithi114.stdout: create 254/32 file after the snapshot 2024-05-27T09:30:28.244 INFO:tasks.workunit.client.0.smithi114.stdout: create 254/33 file after the snapshot 2024-05-27T09:30:28.246 INFO:tasks.workunit.client.0.smithi114.stdout: create 254/34 file after the snapshot 2024-05-27T09:30:28.248 INFO:tasks.workunit.client.0.smithi114.stdout: create 254/35 file after the snapshot 2024-05-27T09:30:28.250 INFO:tasks.workunit.client.0.smithi114.stdout: create 254/36 file after the snapshot 2024-05-27T09:30:28.252 INFO:tasks.workunit.client.0.smithi114.stdout: create 254/37 file after the snapshot 2024-05-27T09:30:28.255 INFO:tasks.workunit.client.0.smithi114.stdout: create 254/38 file after the snapshot 2024-05-27T09:30:28.258 INFO:tasks.workunit.client.0.smithi114.stdout: create 254/39 file after the snapshot 2024-05-27T09:30:28.260 INFO:tasks.workunit.client.0.smithi114.stdout: create 255/.snap/snap-subdir-test 2024-05-27T09:30:28.302 INFO:tasks.workunit.client.0.smithi114.stdout: create 255/30 file after the snapshot 2024-05-27T09:30:28.304 INFO:tasks.workunit.client.0.smithi114.stdout: create 255/31 file after the snapshot 2024-05-27T09:30:28.307 INFO:tasks.workunit.client.0.smithi114.stdout: create 255/32 file after the snapshot 2024-05-27T09:30:28.310 INFO:tasks.workunit.client.0.smithi114.stdout: create 255/33 file after the snapshot 2024-05-27T09:30:28.313 INFO:tasks.workunit.client.0.smithi114.stdout: create 255/34 file after the snapshot 2024-05-27T09:30:28.315 INFO:tasks.workunit.client.0.smithi114.stdout: create 255/35 file after the snapshot 2024-05-27T09:30:28.318 INFO:tasks.workunit.client.0.smithi114.stdout: create 255/36 file after the snapshot 2024-05-27T09:30:28.321 INFO:tasks.workunit.client.0.smithi114.stdout: create 255/37 file after the snapshot 2024-05-27T09:30:28.324 INFO:tasks.workunit.client.0.smithi114.stdout: create 255/38 file after the snapshot 2024-05-27T09:30:28.327 INFO:tasks.workunit.client.0.smithi114.stdout: create 255/39 file after the snapshot 2024-05-27T09:30:28.329 INFO:tasks.workunit.client.0.smithi114.stdout: create 256/.snap/snap-subdir-test 2024-05-27T09:30:28.376 INFO:tasks.workunit.client.0.smithi114.stdout: create 256/30 file after the snapshot 2024-05-27T09:30:28.380 INFO:tasks.workunit.client.0.smithi114.stdout: create 256/31 file after the snapshot 2024-05-27T09:30:28.384 INFO:tasks.workunit.client.0.smithi114.stdout: create 256/32 file after the snapshot 2024-05-27T09:30:28.388 INFO:tasks.workunit.client.0.smithi114.stdout: create 256/33 file after the snapshot 2024-05-27T09:30:28.393 INFO:tasks.workunit.client.0.smithi114.stdout: create 256/34 file after the snapshot 2024-05-27T09:30:28.397 INFO:tasks.workunit.client.0.smithi114.stdout: create 256/35 file after the snapshot 2024-05-27T09:30:28.401 INFO:tasks.workunit.client.0.smithi114.stdout: create 256/36 file after the snapshot 2024-05-27T09:30:28.406 INFO:tasks.workunit.client.0.smithi114.stdout: create 256/37 file after the snapshot 2024-05-27T09:30:28.410 INFO:tasks.workunit.client.0.smithi114.stdout: create 256/38 file after the snapshot 2024-05-27T09:30:28.415 INFO:tasks.workunit.client.0.smithi114.stdout: create 256/39 file after the snapshot 2024-05-27T09:30:28.419 INFO:tasks.workunit.client.0.smithi114.stdout: create 257/.snap/snap-subdir-test 2024-05-27T09:30:28.462 INFO:tasks.workunit.client.0.smithi114.stdout: create 257/30 file after the snapshot 2024-05-27T09:30:28.465 INFO:tasks.workunit.client.0.smithi114.stdout: create 257/31 file after the snapshot 2024-05-27T09:30:28.468 INFO:tasks.workunit.client.0.smithi114.stdout: create 257/32 file after the snapshot 2024-05-27T09:30:28.470 INFO:tasks.workunit.client.0.smithi114.stdout: create 257/33 file after the snapshot 2024-05-27T09:30:28.473 INFO:tasks.workunit.client.0.smithi114.stdout: create 257/34 file after the snapshot 2024-05-27T09:30:28.476 INFO:tasks.workunit.client.0.smithi114.stdout: create 257/35 file after the snapshot 2024-05-27T09:30:28.478 INFO:tasks.workunit.client.0.smithi114.stdout: create 257/36 file after the snapshot 2024-05-27T09:30:28.481 INFO:tasks.workunit.client.0.smithi114.stdout: create 257/37 file after the snapshot 2024-05-27T09:30:28.484 INFO:tasks.workunit.client.0.smithi114.stdout: create 257/38 file after the snapshot 2024-05-27T09:30:28.487 INFO:tasks.workunit.client.0.smithi114.stdout: create 257/39 file after the snapshot 2024-05-27T09:30:28.490 INFO:tasks.workunit.client.0.smithi114.stdout: create 258/.snap/snap-subdir-test 2024-05-27T09:30:28.533 INFO:tasks.workunit.client.0.smithi114.stdout: create 258/30 file after the snapshot 2024-05-27T09:30:28.536 INFO:tasks.workunit.client.0.smithi114.stdout: create 258/31 file after the snapshot 2024-05-27T09:30:28.539 INFO:tasks.workunit.client.0.smithi114.stdout: create 258/32 file after the snapshot 2024-05-27T09:30:28.542 INFO:tasks.workunit.client.0.smithi114.stdout: create 258/33 file after the snapshot 2024-05-27T09:30:28.545 INFO:tasks.workunit.client.0.smithi114.stdout: create 258/34 file after the snapshot 2024-05-27T09:30:28.548 INFO:tasks.workunit.client.0.smithi114.stdout: create 258/35 file after the snapshot 2024-05-27T09:30:28.551 INFO:tasks.workunit.client.0.smithi114.stdout: create 258/36 file after the snapshot 2024-05-27T09:30:28.554 INFO:tasks.workunit.client.0.smithi114.stdout: create 258/37 file after the snapshot 2024-05-27T09:30:28.557 INFO:tasks.workunit.client.0.smithi114.stdout: create 258/38 file after the snapshot 2024-05-27T09:30:28.560 INFO:tasks.workunit.client.0.smithi114.stdout: create 258/39 file after the snapshot 2024-05-27T09:30:28.563 INFO:tasks.workunit.client.0.smithi114.stdout: create 259/.snap/snap-subdir-test 2024-05-27T09:30:28.612 INFO:tasks.workunit.client.0.smithi114.stdout: create 259/30 file after the snapshot 2024-05-27T09:30:28.623 INFO:tasks.workunit.client.0.smithi114.stdout: create 259/31 file after the snapshot 2024-05-27T09:30:28.625 INFO:tasks.workunit.client.0.smithi114.stdout: create 259/32 file after the snapshot 2024-05-27T09:30:28.627 INFO:tasks.workunit.client.0.smithi114.stdout: create 259/33 file after the snapshot 2024-05-27T09:30:28.629 INFO:tasks.workunit.client.0.smithi114.stdout: create 259/34 file after the snapshot 2024-05-27T09:30:28.630 INFO:tasks.workunit.client.0.smithi114.stdout: create 259/35 file after the snapshot 2024-05-27T09:30:28.632 INFO:tasks.workunit.client.0.smithi114.stdout: create 259/36 file after the snapshot 2024-05-27T09:30:28.634 INFO:tasks.workunit.client.0.smithi114.stdout: create 259/37 file after the snapshot 2024-05-27T09:30:28.636 INFO:tasks.workunit.client.0.smithi114.stdout: create 259/38 file after the snapshot 2024-05-27T09:30:28.638 INFO:tasks.workunit.client.0.smithi114.stdout: create 259/39 file after the snapshot 2024-05-27T09:30:28.640 INFO:tasks.workunit.client.0.smithi114.stdout: create 260/.snap/snap-subdir-test 2024-05-27T09:30:28.682 INFO:tasks.workunit.client.0.smithi114.stdout: create 260/30 file after the snapshot 2024-05-27T09:30:28.684 INFO:tasks.workunit.client.0.smithi114.stdout: create 260/31 file after the snapshot 2024-05-27T09:30:28.686 INFO:tasks.workunit.client.0.smithi114.stdout: create 260/32 file after the snapshot 2024-05-27T09:30:28.687 INFO:tasks.workunit.client.0.smithi114.stdout: create 260/33 file after the snapshot 2024-05-27T09:30:28.689 INFO:tasks.workunit.client.0.smithi114.stdout: create 260/34 file after the snapshot 2024-05-27T09:30:28.692 INFO:tasks.workunit.client.0.smithi114.stdout: create 260/35 file after the snapshot 2024-05-27T09:30:28.695 INFO:tasks.workunit.client.0.smithi114.stdout: create 260/36 file after the snapshot 2024-05-27T09:30:28.698 INFO:tasks.workunit.client.0.smithi114.stdout: create 260/37 file after the snapshot 2024-05-27T09:30:28.700 INFO:tasks.workunit.client.0.smithi114.stdout: create 260/38 file after the snapshot 2024-05-27T09:30:28.703 INFO:tasks.workunit.client.0.smithi114.stdout: create 260/39 file after the snapshot 2024-05-27T09:30:28.706 INFO:tasks.workunit.client.0.smithi114.stdout: create 261/.snap/snap-subdir-test 2024-05-27T09:30:28.762 INFO:tasks.workunit.client.0.smithi114.stdout: create 261/30 file after the snapshot 2024-05-27T09:30:28.765 INFO:tasks.workunit.client.0.smithi114.stdout: create 261/31 file after the snapshot 2024-05-27T09:30:28.768 INFO:tasks.workunit.client.0.smithi114.stdout: create 261/32 file after the snapshot 2024-05-27T09:30:28.771 INFO:tasks.workunit.client.0.smithi114.stdout: create 261/33 file after the snapshot 2024-05-27T09:30:28.774 INFO:tasks.workunit.client.0.smithi114.stdout: create 261/34 file after the snapshot 2024-05-27T09:30:28.777 INFO:tasks.workunit.client.0.smithi114.stdout: create 261/35 file after the snapshot 2024-05-27T09:30:28.780 INFO:tasks.workunit.client.0.smithi114.stdout: create 261/36 file after the snapshot 2024-05-27T09:30:28.782 INFO:tasks.workunit.client.0.smithi114.stdout: create 261/37 file after the snapshot 2024-05-27T09:30:28.785 INFO:tasks.workunit.client.0.smithi114.stdout: create 261/38 file after the snapshot 2024-05-27T09:30:28.788 INFO:tasks.workunit.client.0.smithi114.stdout: create 261/39 file after the snapshot 2024-05-27T09:30:28.791 INFO:tasks.workunit.client.0.smithi114.stdout: create 262/.snap/snap-subdir-test 2024-05-27T09:30:28.835 INFO:tasks.workunit.client.0.smithi114.stdout: create 262/30 file after the snapshot 2024-05-27T09:30:28.837 INFO:tasks.workunit.client.0.smithi114.stdout: create 262/31 file after the snapshot 2024-05-27T09:30:28.840 INFO:tasks.workunit.client.0.smithi114.stdout: create 262/32 file after the snapshot 2024-05-27T09:30:28.842 INFO:tasks.workunit.client.0.smithi114.stdout: create 262/33 file after the snapshot 2024-05-27T09:30:28.845 INFO:tasks.workunit.client.0.smithi114.stdout: create 262/34 file after the snapshot 2024-05-27T09:30:28.848 INFO:tasks.workunit.client.0.smithi114.stdout: create 262/35 file after the snapshot 2024-05-27T09:30:28.850 INFO:tasks.workunit.client.0.smithi114.stdout: create 262/36 file after the snapshot 2024-05-27T09:30:28.853 INFO:tasks.workunit.client.0.smithi114.stdout: create 262/37 file after the snapshot 2024-05-27T09:30:28.856 INFO:tasks.workunit.client.0.smithi114.stdout: create 262/38 file after the snapshot 2024-05-27T09:30:28.859 INFO:tasks.workunit.client.0.smithi114.stdout: create 262/39 file after the snapshot 2024-05-27T09:30:28.862 INFO:tasks.workunit.client.0.smithi114.stdout: create 263/.snap/snap-subdir-test 2024-05-27T09:30:28.908 INFO:tasks.workunit.client.0.smithi114.stdout: create 263/30 file after the snapshot 2024-05-27T09:30:28.911 INFO:tasks.workunit.client.0.smithi114.stdout: create 263/31 file after the snapshot 2024-05-27T09:30:28.914 INFO:tasks.workunit.client.0.smithi114.stdout: create 263/32 file after the snapshot 2024-05-27T09:30:28.917 INFO:tasks.workunit.client.0.smithi114.stdout: create 263/33 file after the snapshot 2024-05-27T09:30:28.920 INFO:tasks.workunit.client.0.smithi114.stdout: create 263/34 file after the snapshot 2024-05-27T09:30:28.924 INFO:tasks.workunit.client.0.smithi114.stdout: create 263/35 file after the snapshot 2024-05-27T09:30:28.926 INFO:tasks.workunit.client.0.smithi114.stdout: create 263/36 file after the snapshot 2024-05-27T09:30:28.929 INFO:tasks.workunit.client.0.smithi114.stdout: create 263/37 file after the snapshot 2024-05-27T09:30:28.932 INFO:tasks.workunit.client.0.smithi114.stdout: create 263/38 file after the snapshot 2024-05-27T09:30:28.935 INFO:tasks.workunit.client.0.smithi114.stdout: create 263/39 file after the snapshot 2024-05-27T09:30:28.938 INFO:tasks.workunit.client.0.smithi114.stdout: create 264/.snap/snap-subdir-test 2024-05-27T09:30:28.992 INFO:tasks.workunit.client.0.smithi114.stdout: create 264/30 file after the snapshot 2024-05-27T09:30:28.996 INFO:tasks.workunit.client.0.smithi114.stdout: create 264/31 file after the snapshot 2024-05-27T09:30:29.000 INFO:tasks.workunit.client.0.smithi114.stdout: create 264/32 file after the snapshot 2024-05-27T09:30:29.005 INFO:tasks.workunit.client.0.smithi114.stdout: create 264/33 file after the snapshot 2024-05-27T09:30:29.009 INFO:tasks.workunit.client.0.smithi114.stdout: create 264/34 file after the snapshot 2024-05-27T09:30:29.013 INFO:tasks.workunit.client.0.smithi114.stdout: create 264/35 file after the snapshot 2024-05-27T09:30:29.018 INFO:tasks.workunit.client.0.smithi114.stdout: create 264/36 file after the snapshot 2024-05-27T09:30:29.022 INFO:tasks.workunit.client.0.smithi114.stdout: create 264/37 file after the snapshot 2024-05-27T09:30:29.027 INFO:tasks.workunit.client.0.smithi114.stdout: create 264/38 file after the snapshot 2024-05-27T09:30:29.031 INFO:tasks.workunit.client.0.smithi114.stdout: create 264/39 file after the snapshot 2024-05-27T09:30:29.037 INFO:tasks.workunit.client.0.smithi114.stdout: create 265/.snap/snap-subdir-test 2024-05-27T09:30:29.088 INFO:tasks.workunit.client.0.smithi114.stdout: create 265/30 file after the snapshot 2024-05-27T09:30:29.091 INFO:tasks.workunit.client.0.smithi114.stdout: create 265/31 file after the snapshot 2024-05-27T09:30:29.096 INFO:tasks.workunit.client.0.smithi114.stdout: create 265/32 file after the snapshot 2024-05-27T09:30:29.100 INFO:tasks.workunit.client.0.smithi114.stdout: create 265/33 file after the snapshot 2024-05-27T09:30:29.104 INFO:tasks.workunit.client.0.smithi114.stdout: create 265/34 file after the snapshot 2024-05-27T09:30:29.109 INFO:tasks.workunit.client.0.smithi114.stdout: create 265/35 file after the snapshot 2024-05-27T09:30:29.113 INFO:tasks.workunit.client.0.smithi114.stdout: create 265/36 file after the snapshot 2024-05-27T09:30:29.117 INFO:tasks.workunit.client.0.smithi114.stdout: create 265/37 file after the snapshot 2024-05-27T09:30:29.122 INFO:tasks.workunit.client.0.smithi114.stdout: create 265/38 file after the snapshot 2024-05-27T09:30:29.126 INFO:tasks.workunit.client.0.smithi114.stdout: create 265/39 file after the snapshot 2024-05-27T09:30:29.130 INFO:tasks.workunit.client.0.smithi114.stdout: create 266/.snap/snap-subdir-test 2024-05-27T09:30:29.176 INFO:tasks.workunit.client.0.smithi114.stdout: create 266/30 file after the snapshot 2024-05-27T09:30:29.178 INFO:tasks.workunit.client.0.smithi114.stdout: create 266/31 file after the snapshot 2024-05-27T09:30:29.181 INFO:tasks.workunit.client.0.smithi114.stdout: create 266/32 file after the snapshot 2024-05-27T09:30:29.184 INFO:tasks.workunit.client.0.smithi114.stdout: create 266/33 file after the snapshot 2024-05-27T09:30:29.187 INFO:tasks.workunit.client.0.smithi114.stdout: create 266/34 file after the snapshot 2024-05-27T09:30:29.190 INFO:tasks.workunit.client.0.smithi114.stdout: create 266/35 file after the snapshot 2024-05-27T09:30:29.193 INFO:tasks.workunit.client.0.smithi114.stdout: create 266/36 file after the snapshot 2024-05-27T09:30:29.196 INFO:tasks.workunit.client.0.smithi114.stdout: create 266/37 file after the snapshot 2024-05-27T09:30:29.198 INFO:tasks.workunit.client.0.smithi114.stdout: create 266/38 file after the snapshot 2024-05-27T09:30:29.201 INFO:tasks.workunit.client.0.smithi114.stdout: create 266/39 file after the snapshot 2024-05-27T09:30:29.204 INFO:tasks.workunit.client.0.smithi114.stdout: create 267/.snap/snap-subdir-test 2024-05-27T09:30:29.264 INFO:tasks.workunit.client.0.smithi114.stdout: create 267/30 file after the snapshot 2024-05-27T09:30:29.266 INFO:tasks.workunit.client.0.smithi114.stdout: create 267/31 file after the snapshot 2024-05-27T09:30:29.269 INFO:tasks.workunit.client.0.smithi114.stdout: create 267/32 file after the snapshot 2024-05-27T09:30:29.271 INFO:tasks.workunit.client.0.smithi114.stdout: create 267/33 file after the snapshot 2024-05-27T09:30:29.274 INFO:tasks.workunit.client.0.smithi114.stdout: create 267/34 file after the snapshot 2024-05-27T09:30:29.277 INFO:tasks.workunit.client.0.smithi114.stdout: create 267/35 file after the snapshot 2024-05-27T09:30:29.279 INFO:tasks.workunit.client.0.smithi114.stdout: create 267/36 file after the snapshot 2024-05-27T09:30:29.282 INFO:tasks.workunit.client.0.smithi114.stdout: create 267/37 file after the snapshot 2024-05-27T09:30:29.285 INFO:tasks.workunit.client.0.smithi114.stdout: create 267/38 file after the snapshot 2024-05-27T09:30:29.288 INFO:tasks.workunit.client.0.smithi114.stdout: create 267/39 file after the snapshot 2024-05-27T09:30:29.291 INFO:tasks.workunit.client.0.smithi114.stdout: create 268/.snap/snap-subdir-test 2024-05-27T09:30:29.337 INFO:tasks.workunit.client.0.smithi114.stdout: create 268/30 file after the snapshot 2024-05-27T09:30:29.339 INFO:tasks.workunit.client.0.smithi114.stdout: create 268/31 file after the snapshot 2024-05-27T09:30:29.340 INFO:tasks.workunit.client.0.smithi114.stdout: create 268/32 file after the snapshot 2024-05-27T09:30:29.342 INFO:tasks.workunit.client.0.smithi114.stdout: create 268/33 file after the snapshot 2024-05-27T09:30:29.343 INFO:tasks.workunit.client.0.smithi114.stdout: create 268/34 file after the snapshot 2024-05-27T09:30:29.345 INFO:tasks.workunit.client.0.smithi114.stdout: create 268/35 file after the snapshot 2024-05-27T09:30:29.347 INFO:tasks.workunit.client.0.smithi114.stdout: create 268/36 file after the snapshot 2024-05-27T09:30:29.349 INFO:tasks.workunit.client.0.smithi114.stdout: create 268/37 file after the snapshot 2024-05-27T09:30:29.351 INFO:tasks.workunit.client.0.smithi114.stdout: create 268/38 file after the snapshot 2024-05-27T09:30:29.353 INFO:tasks.workunit.client.0.smithi114.stdout: create 268/39 file after the snapshot 2024-05-27T09:30:29.355 INFO:tasks.workunit.client.0.smithi114.stdout: create 269/.snap/snap-subdir-test 2024-05-27T09:30:29.405 INFO:tasks.workunit.client.0.smithi114.stdout: create 269/30 file after the snapshot 2024-05-27T09:30:29.407 INFO:tasks.workunit.client.0.smithi114.stdout: create 269/31 file after the snapshot 2024-05-27T09:30:29.409 INFO:tasks.workunit.client.0.smithi114.stdout: create 269/32 file after the snapshot 2024-05-27T09:30:29.411 INFO:tasks.workunit.client.0.smithi114.stdout: create 269/33 file after the snapshot 2024-05-27T09:30:29.413 INFO:tasks.workunit.client.0.smithi114.stdout: create 269/34 file after the snapshot 2024-05-27T09:30:29.416 INFO:tasks.workunit.client.0.smithi114.stdout: create 269/35 file after the snapshot 2024-05-27T09:30:29.419 INFO:tasks.workunit.client.0.smithi114.stdout: create 269/36 file after the snapshot 2024-05-27T09:30:29.421 INFO:tasks.workunit.client.0.smithi114.stdout: create 269/37 file after the snapshot 2024-05-27T09:30:29.424 INFO:tasks.workunit.client.0.smithi114.stdout: create 269/38 file after the snapshot 2024-05-27T09:30:29.427 INFO:tasks.workunit.client.0.smithi114.stdout: create 269/39 file after the snapshot 2024-05-27T09:30:29.429 INFO:tasks.workunit.client.0.smithi114.stdout: create 270/.snap/snap-subdir-test 2024-05-27T09:30:29.493 INFO:tasks.workunit.client.0.smithi114.stdout: create 270/30 file after the snapshot 2024-05-27T09:30:29.495 INFO:tasks.workunit.client.0.smithi114.stdout: create 270/31 file after the snapshot 2024-05-27T09:30:29.498 INFO:tasks.workunit.client.0.smithi114.stdout: create 270/32 file after the snapshot 2024-05-27T09:30:29.501 INFO:tasks.workunit.client.0.smithi114.stdout: create 270/33 file after the snapshot 2024-05-27T09:30:29.503 INFO:tasks.workunit.client.0.smithi114.stdout: create 270/34 file after the snapshot 2024-05-27T09:30:29.506 INFO:tasks.workunit.client.0.smithi114.stdout: create 270/35 file after the snapshot 2024-05-27T09:30:29.509 INFO:tasks.workunit.client.0.smithi114.stdout: create 270/36 file after the snapshot 2024-05-27T09:30:29.512 INFO:tasks.workunit.client.0.smithi114.stdout: create 270/37 file after the snapshot 2024-05-27T09:30:29.514 INFO:tasks.workunit.client.0.smithi114.stdout: create 270/38 file after the snapshot 2024-05-27T09:30:29.517 INFO:tasks.workunit.client.0.smithi114.stdout: create 270/39 file after the snapshot 2024-05-27T09:30:29.520 INFO:tasks.workunit.client.0.smithi114.stdout: create 271/.snap/snap-subdir-test 2024-05-27T09:30:29.577 INFO:tasks.workunit.client.0.smithi114.stdout: create 271/30 file after the snapshot 2024-05-27T09:30:29.580 INFO:tasks.workunit.client.0.smithi114.stdout: create 271/31 file after the snapshot 2024-05-27T09:30:29.583 INFO:tasks.workunit.client.0.smithi114.stdout: create 271/32 file after the snapshot 2024-05-27T09:30:29.586 INFO:tasks.workunit.client.0.smithi114.stdout: create 271/33 file after the snapshot 2024-05-27T09:30:29.590 INFO:tasks.workunit.client.0.smithi114.stdout: create 271/34 file after the snapshot 2024-05-27T09:30:29.593 INFO:tasks.workunit.client.0.smithi114.stdout: create 271/35 file after the snapshot 2024-05-27T09:30:29.597 INFO:tasks.workunit.client.0.smithi114.stdout: create 271/36 file after the snapshot 2024-05-27T09:30:29.601 INFO:tasks.workunit.client.0.smithi114.stdout: create 271/37 file after the snapshot 2024-05-27T09:30:29.605 INFO:tasks.workunit.client.0.smithi114.stdout: create 271/38 file after the snapshot 2024-05-27T09:30:29.609 INFO:tasks.workunit.client.0.smithi114.stdout: create 271/39 file after the snapshot 2024-05-27T09:30:29.612 INFO:tasks.workunit.client.0.smithi114.stdout: create 272/.snap/snap-subdir-test 2024-05-27T09:30:29.663 INFO:tasks.workunit.client.0.smithi114.stdout: create 272/30 file after the snapshot 2024-05-27T09:30:29.665 INFO:tasks.workunit.client.0.smithi114.stdout: create 272/31 file after the snapshot 2024-05-27T09:30:29.668 INFO:tasks.workunit.client.0.smithi114.stdout: create 272/32 file after the snapshot 2024-05-27T09:30:29.671 INFO:tasks.workunit.client.0.smithi114.stdout: create 272/33 file after the snapshot 2024-05-27T09:30:29.675 INFO:tasks.workunit.client.0.smithi114.stdout: create 272/34 file after the snapshot 2024-05-27T09:30:29.678 INFO:tasks.workunit.client.0.smithi114.stdout: create 272/35 file after the snapshot 2024-05-27T09:30:29.681 INFO:tasks.workunit.client.0.smithi114.stdout: create 272/36 file after the snapshot 2024-05-27T09:30:29.684 INFO:tasks.workunit.client.0.smithi114.stdout: create 272/37 file after the snapshot 2024-05-27T09:30:29.687 INFO:tasks.workunit.client.0.smithi114.stdout: create 272/38 file after the snapshot 2024-05-27T09:30:29.690 INFO:tasks.workunit.client.0.smithi114.stdout: create 272/39 file after the snapshot 2024-05-27T09:30:29.693 INFO:tasks.workunit.client.0.smithi114.stdout: create 273/.snap/snap-subdir-test 2024-05-27T09:30:29.740 INFO:tasks.workunit.client.0.smithi114.stdout: create 273/30 file after the snapshot 2024-05-27T09:30:29.742 INFO:tasks.workunit.client.0.smithi114.stdout: create 273/31 file after the snapshot 2024-05-27T09:30:29.744 INFO:tasks.workunit.client.0.smithi114.stdout: create 273/32 file after the snapshot 2024-05-27T09:30:29.746 INFO:tasks.workunit.client.0.smithi114.stdout: create 273/33 file after the snapshot 2024-05-27T09:30:29.748 INFO:tasks.workunit.client.0.smithi114.stdout: create 273/34 file after the snapshot 2024-05-27T09:30:29.750 INFO:tasks.workunit.client.0.smithi114.stdout: create 273/35 file after the snapshot 2024-05-27T09:30:29.752 INFO:tasks.workunit.client.0.smithi114.stdout: create 273/36 file after the snapshot 2024-05-27T09:30:29.754 INFO:tasks.workunit.client.0.smithi114.stdout: create 273/37 file after the snapshot 2024-05-27T09:30:29.756 INFO:tasks.workunit.client.0.smithi114.stdout: create 273/38 file after the snapshot 2024-05-27T09:30:29.758 INFO:tasks.workunit.client.0.smithi114.stdout: create 273/39 file after the snapshot 2024-05-27T09:30:29.760 INFO:tasks.workunit.client.0.smithi114.stdout: create 274/.snap/snap-subdir-test 2024-05-27T09:30:29.813 INFO:tasks.workunit.client.0.smithi114.stdout: create 274/30 file after the snapshot 2024-05-27T09:30:29.815 INFO:tasks.workunit.client.0.smithi114.stdout: create 274/31 file after the snapshot 2024-05-27T09:30:29.817 INFO:tasks.workunit.client.0.smithi114.stdout: create 274/32 file after the snapshot 2024-05-27T09:30:29.819 INFO:tasks.workunit.client.0.smithi114.stdout: create 274/33 file after the snapshot 2024-05-27T09:30:29.820 INFO:tasks.workunit.client.0.smithi114.stdout: create 274/34 file after the snapshot 2024-05-27T09:30:29.822 INFO:tasks.workunit.client.0.smithi114.stdout: create 274/35 file after the snapshot 2024-05-27T09:30:29.824 INFO:tasks.workunit.client.0.smithi114.stdout: create 274/36 file after the snapshot 2024-05-27T09:30:29.826 INFO:tasks.workunit.client.0.smithi114.stdout: create 274/37 file after the snapshot 2024-05-27T09:30:29.828 INFO:tasks.workunit.client.0.smithi114.stdout: create 274/38 file after the snapshot 2024-05-27T09:30:29.831 INFO:tasks.workunit.client.0.smithi114.stdout: create 274/39 file after the snapshot 2024-05-27T09:30:29.834 INFO:tasks.workunit.client.0.smithi114.stdout: create 275/.snap/snap-subdir-test 2024-05-27T09:30:29.879 INFO:tasks.workunit.client.0.smithi114.stdout: create 275/30 file after the snapshot 2024-05-27T09:30:29.881 INFO:tasks.workunit.client.0.smithi114.stdout: create 275/31 file after the snapshot 2024-05-27T09:30:29.883 INFO:tasks.workunit.client.0.smithi114.stdout: create 275/32 file after the snapshot 2024-05-27T09:30:29.885 INFO:tasks.workunit.client.0.smithi114.stdout: create 275/33 file after the snapshot 2024-05-27T09:30:29.888 INFO:tasks.workunit.client.0.smithi114.stdout: create 275/34 file after the snapshot 2024-05-27T09:30:29.891 INFO:tasks.workunit.client.0.smithi114.stdout: create 275/35 file after the snapshot 2024-05-27T09:30:29.894 INFO:tasks.workunit.client.0.smithi114.stdout: create 275/36 file after the snapshot 2024-05-27T09:30:29.897 INFO:tasks.workunit.client.0.smithi114.stdout: create 275/37 file after the snapshot 2024-05-27T09:30:29.899 INFO:tasks.workunit.client.0.smithi114.stdout: create 275/38 file after the snapshot 2024-05-27T09:30:29.902 INFO:tasks.workunit.client.0.smithi114.stdout: create 275/39 file after the snapshot 2024-05-27T09:30:29.905 INFO:tasks.workunit.client.0.smithi114.stdout: create 276/.snap/snap-subdir-test 2024-05-27T09:30:29.962 INFO:tasks.workunit.client.0.smithi114.stdout: create 276/30 file after the snapshot 2024-05-27T09:30:29.964 INFO:tasks.workunit.client.0.smithi114.stdout: create 276/31 file after the snapshot 2024-05-27T09:30:29.968 INFO:tasks.workunit.client.0.smithi114.stdout: create 276/32 file after the snapshot 2024-05-27T09:30:29.972 INFO:tasks.workunit.client.0.smithi114.stdout: create 276/33 file after the snapshot 2024-05-27T09:30:29.975 INFO:tasks.workunit.client.0.smithi114.stdout: create 276/34 file after the snapshot 2024-05-27T09:30:29.978 INFO:tasks.workunit.client.0.smithi114.stdout: create 276/35 file after the snapshot 2024-05-27T09:30:29.981 INFO:tasks.workunit.client.0.smithi114.stdout: create 276/36 file after the snapshot 2024-05-27T09:30:29.983 INFO:tasks.workunit.client.0.smithi114.stdout: create 276/37 file after the snapshot 2024-05-27T09:30:29.986 INFO:tasks.workunit.client.0.smithi114.stdout: create 276/38 file after the snapshot 2024-05-27T09:30:29.989 INFO:tasks.workunit.client.0.smithi114.stdout: create 276/39 file after the snapshot 2024-05-27T09:30:29.992 INFO:tasks.workunit.client.0.smithi114.stdout: create 277/.snap/snap-subdir-test 2024-05-27T09:30:30.049 INFO:tasks.workunit.client.0.smithi114.stdout: create 277/30 file after the snapshot 2024-05-27T09:30:30.053 INFO:tasks.workunit.client.0.smithi114.stdout: create 277/31 file after the snapshot 2024-05-27T09:30:30.057 INFO:tasks.workunit.client.0.smithi114.stdout: create 277/32 file after the snapshot 2024-05-27T09:30:30.058 INFO:tasks.workunit.client.0.smithi114.stdout: create 277/33 file after the snapshot 2024-05-27T09:30:30.061 INFO:tasks.workunit.client.0.smithi114.stdout: create 277/34 file after the snapshot 2024-05-27T09:30:30.064 INFO:tasks.workunit.client.0.smithi114.stdout: create 277/35 file after the snapshot 2024-05-27T09:30:30.066 INFO:tasks.workunit.client.0.smithi114.stdout: create 277/36 file after the snapshot 2024-05-27T09:30:30.069 INFO:tasks.workunit.client.0.smithi114.stdout: create 277/37 file after the snapshot 2024-05-27T09:30:30.072 INFO:tasks.workunit.client.0.smithi114.stdout: create 277/38 file after the snapshot 2024-05-27T09:30:30.075 INFO:tasks.workunit.client.0.smithi114.stdout: create 277/39 file after the snapshot 2024-05-27T09:30:30.078 INFO:tasks.workunit.client.0.smithi114.stdout: create 278/.snap/snap-subdir-test 2024-05-27T09:30:30.129 INFO:tasks.workunit.client.0.smithi114.stdout: create 278/30 file after the snapshot 2024-05-27T09:30:30.132 INFO:tasks.workunit.client.0.smithi114.stdout: create 278/31 file after the snapshot 2024-05-27T09:30:30.134 INFO:tasks.workunit.client.0.smithi114.stdout: create 278/32 file after the snapshot 2024-05-27T09:30:30.137 INFO:tasks.workunit.client.0.smithi114.stdout: create 278/33 file after the snapshot 2024-05-27T09:30:30.140 INFO:tasks.workunit.client.0.smithi114.stdout: create 278/34 file after the snapshot 2024-05-27T09:30:30.143 INFO:tasks.workunit.client.0.smithi114.stdout: create 278/35 file after the snapshot 2024-05-27T09:30:30.146 INFO:tasks.workunit.client.0.smithi114.stdout: create 278/36 file after the snapshot 2024-05-27T09:30:30.148 INFO:tasks.workunit.client.0.smithi114.stdout: create 278/37 file after the snapshot 2024-05-27T09:30:30.151 INFO:tasks.workunit.client.0.smithi114.stdout: create 278/38 file after the snapshot 2024-05-27T09:30:30.154 INFO:tasks.workunit.client.0.smithi114.stdout: create 278/39 file after the snapshot 2024-05-27T09:30:30.157 INFO:tasks.workunit.client.0.smithi114.stdout: create 279/.snap/snap-subdir-test 2024-05-27T09:30:30.202 INFO:tasks.workunit.client.0.smithi114.stdout: create 279/30 file after the snapshot 2024-05-27T09:30:30.204 INFO:tasks.workunit.client.0.smithi114.stdout: create 279/31 file after the snapshot 2024-05-27T09:30:30.207 INFO:tasks.workunit.client.0.smithi114.stdout: create 279/32 file after the snapshot 2024-05-27T09:30:30.210 INFO:tasks.workunit.client.0.smithi114.stdout: create 279/33 file after the snapshot 2024-05-27T09:30:30.213 INFO:tasks.workunit.client.0.smithi114.stdout: create 279/34 file after the snapshot 2024-05-27T09:30:30.215 INFO:tasks.workunit.client.0.smithi114.stdout: create 279/35 file after the snapshot 2024-05-27T09:30:30.218 INFO:tasks.workunit.client.0.smithi114.stdout: create 279/36 file after the snapshot 2024-05-27T09:30:30.220 INFO:tasks.workunit.client.0.smithi114.stdout: create 279/37 file after the snapshot 2024-05-27T09:30:30.223 INFO:tasks.workunit.client.0.smithi114.stdout: create 279/38 file after the snapshot 2024-05-27T09:30:30.226 INFO:tasks.workunit.client.0.smithi114.stdout: create 279/39 file after the snapshot 2024-05-27T09:30:30.229 INFO:tasks.workunit.client.0.smithi114.stdout: create 280/.snap/snap-subdir-test 2024-05-27T09:30:30.277 INFO:tasks.workunit.client.0.smithi114.stdout: create 280/30 file after the snapshot 2024-05-27T09:30:30.279 INFO:tasks.workunit.client.0.smithi114.stdout: create 280/31 file after the snapshot 2024-05-27T09:30:30.280 INFO:tasks.workunit.client.0.smithi114.stdout: create 280/32 file after the snapshot 2024-05-27T09:30:30.282 INFO:tasks.workunit.client.0.smithi114.stdout: create 280/33 file after the snapshot 2024-05-27T09:30:30.284 INFO:tasks.workunit.client.0.smithi114.stdout: create 280/34 file after the snapshot 2024-05-27T09:30:30.285 INFO:tasks.workunit.client.0.smithi114.stdout: create 280/35 file after the snapshot 2024-05-27T09:30:30.287 INFO:tasks.workunit.client.0.smithi114.stdout: create 280/36 file after the snapshot 2024-05-27T09:30:30.289 INFO:tasks.workunit.client.0.smithi114.stdout: create 280/37 file after the snapshot 2024-05-27T09:30:30.290 INFO:tasks.workunit.client.0.smithi114.stdout: create 280/38 file after the snapshot 2024-05-27T09:30:30.292 INFO:tasks.workunit.client.0.smithi114.stdout: create 280/39 file after the snapshot 2024-05-27T09:30:30.295 INFO:tasks.workunit.client.0.smithi114.stdout: create 281/.snap/snap-subdir-test 2024-05-27T09:30:30.338 INFO:tasks.workunit.client.0.smithi114.stdout: create 281/30 file after the snapshot 2024-05-27T09:30:30.340 INFO:tasks.workunit.client.0.smithi114.stdout: create 281/31 file after the snapshot 2024-05-27T09:30:30.341 INFO:tasks.workunit.client.0.smithi114.stdout: create 281/32 file after the snapshot 2024-05-27T09:30:30.343 INFO:tasks.workunit.client.0.smithi114.stdout: create 281/33 file after the snapshot 2024-05-27T09:30:30.344 INFO:tasks.workunit.client.0.smithi114.stdout: create 281/34 file after the snapshot 2024-05-27T09:30:30.346 INFO:tasks.workunit.client.0.smithi114.stdout: create 281/35 file after the snapshot 2024-05-27T09:30:30.347 INFO:tasks.workunit.client.0.smithi114.stdout: create 281/36 file after the snapshot 2024-05-27T09:30:30.349 INFO:tasks.workunit.client.0.smithi114.stdout: create 281/37 file after the snapshot 2024-05-27T09:30:30.350 INFO:tasks.workunit.client.0.smithi114.stdout: create 281/38 file after the snapshot 2024-05-27T09:30:30.352 INFO:tasks.workunit.client.0.smithi114.stdout: create 281/39 file after the snapshot 2024-05-27T09:30:30.354 INFO:tasks.workunit.client.0.smithi114.stdout: create 282/.snap/snap-subdir-test 2024-05-27T09:30:30.393 INFO:tasks.workunit.client.0.smithi114.stdout: create 282/30 file after the snapshot 2024-05-27T09:30:30.395 INFO:tasks.workunit.client.0.smithi114.stdout: create 282/31 file after the snapshot 2024-05-27T09:30:30.396 INFO:tasks.workunit.client.0.smithi114.stdout: create 282/32 file after the snapshot 2024-05-27T09:30:30.398 INFO:tasks.workunit.client.0.smithi114.stdout: create 282/33 file after the snapshot 2024-05-27T09:30:30.400 INFO:tasks.workunit.client.0.smithi114.stdout: create 282/34 file after the snapshot 2024-05-27T09:30:30.402 INFO:tasks.workunit.client.0.smithi114.stdout: create 282/35 file after the snapshot 2024-05-27T09:30:30.405 INFO:tasks.workunit.client.0.smithi114.stdout: create 282/36 file after the snapshot 2024-05-27T09:30:30.408 INFO:tasks.workunit.client.0.smithi114.stdout: create 282/37 file after the snapshot 2024-05-27T09:30:30.411 INFO:tasks.workunit.client.0.smithi114.stdout: create 282/38 file after the snapshot 2024-05-27T09:30:30.413 INFO:tasks.workunit.client.0.smithi114.stdout: create 282/39 file after the snapshot 2024-05-27T09:30:30.416 INFO:tasks.workunit.client.0.smithi114.stdout: create 283/.snap/snap-subdir-test 2024-05-27T09:30:30.467 INFO:tasks.workunit.client.0.smithi114.stdout: create 283/30 file after the snapshot 2024-05-27T09:30:30.471 INFO:tasks.workunit.client.0.smithi114.stdout: create 283/31 file after the snapshot 2024-05-27T09:30:30.475 INFO:tasks.workunit.client.0.smithi114.stdout: create 283/32 file after the snapshot 2024-05-27T09:30:30.479 INFO:tasks.workunit.client.0.smithi114.stdout: create 283/33 file after the snapshot 2024-05-27T09:30:30.484 INFO:tasks.workunit.client.0.smithi114.stdout: create 283/34 file after the snapshot 2024-05-27T09:30:30.489 INFO:tasks.workunit.client.0.smithi114.stdout: create 283/35 file after the snapshot 2024-05-27T09:30:30.493 INFO:tasks.workunit.client.0.smithi114.stdout: create 283/36 file after the snapshot 2024-05-27T09:30:30.497 INFO:tasks.workunit.client.0.smithi114.stdout: create 283/37 file after the snapshot 2024-05-27T09:30:30.502 INFO:tasks.workunit.client.0.smithi114.stdout: create 283/38 file after the snapshot 2024-05-27T09:30:30.506 INFO:tasks.workunit.client.0.smithi114.stdout: create 283/39 file after the snapshot 2024-05-27T09:30:30.511 INFO:tasks.workunit.client.0.smithi114.stdout: create 284/.snap/snap-subdir-test 2024-05-27T09:30:30.558 INFO:tasks.workunit.client.0.smithi114.stdout: create 284/30 file after the snapshot 2024-05-27T09:30:30.562 INFO:tasks.workunit.client.0.smithi114.stdout: create 284/31 file after the snapshot 2024-05-27T09:30:30.565 INFO:tasks.workunit.client.0.smithi114.stdout: create 284/32 file after the snapshot 2024-05-27T09:30:30.569 INFO:tasks.workunit.client.0.smithi114.stdout: create 284/33 file after the snapshot 2024-05-27T09:30:30.572 INFO:tasks.workunit.client.0.smithi114.stdout: create 284/34 file after the snapshot 2024-05-27T09:30:30.576 INFO:tasks.workunit.client.0.smithi114.stdout: create 284/35 file after the snapshot 2024-05-27T09:30:30.579 INFO:tasks.workunit.client.0.smithi114.stdout: create 284/36 file after the snapshot 2024-05-27T09:30:30.583 INFO:tasks.workunit.client.0.smithi114.stdout: create 284/37 file after the snapshot 2024-05-27T09:30:30.586 INFO:tasks.workunit.client.0.smithi114.stdout: create 284/38 file after the snapshot 2024-05-27T09:30:30.589 INFO:tasks.workunit.client.0.smithi114.stdout: create 284/39 file after the snapshot 2024-05-27T09:30:30.593 INFO:tasks.workunit.client.0.smithi114.stdout: create 285/.snap/snap-subdir-test 2024-05-27T09:30:30.646 INFO:tasks.workunit.client.0.smithi114.stdout: create 285/30 file after the snapshot 2024-05-27T09:30:30.650 INFO:tasks.workunit.client.0.smithi114.stdout: create 285/31 file after the snapshot 2024-05-27T09:30:30.654 INFO:tasks.workunit.client.0.smithi114.stdout: create 285/32 file after the snapshot 2024-05-27T09:30:30.658 INFO:tasks.workunit.client.0.smithi114.stdout: create 285/33 file after the snapshot 2024-05-27T09:30:30.662 INFO:tasks.workunit.client.0.smithi114.stdout: create 285/34 file after the snapshot 2024-05-27T09:30:30.667 INFO:tasks.workunit.client.0.smithi114.stdout: create 285/35 file after the snapshot 2024-05-27T09:30:30.671 INFO:tasks.workunit.client.0.smithi114.stdout: create 285/36 file after the snapshot 2024-05-27T09:30:30.675 INFO:tasks.workunit.client.0.smithi114.stdout: create 285/37 file after the snapshot 2024-05-27T09:30:30.679 INFO:tasks.workunit.client.0.smithi114.stdout: create 285/38 file after the snapshot 2024-05-27T09:30:30.683 INFO:tasks.workunit.client.0.smithi114.stdout: create 285/39 file after the snapshot 2024-05-27T09:30:30.687 INFO:tasks.workunit.client.0.smithi114.stdout: create 286/.snap/snap-subdir-test 2024-05-27T09:30:30.742 INFO:tasks.workunit.client.0.smithi114.stdout: create 286/30 file after the snapshot 2024-05-27T09:30:30.744 INFO:tasks.workunit.client.0.smithi114.stdout: create 286/31 file after the snapshot 2024-05-27T09:30:30.747 INFO:tasks.workunit.client.0.smithi114.stdout: create 286/32 file after the snapshot 2024-05-27T09:30:30.750 INFO:tasks.workunit.client.0.smithi114.stdout: create 286/33 file after the snapshot 2024-05-27T09:30:30.753 INFO:tasks.workunit.client.0.smithi114.stdout: create 286/34 file after the snapshot 2024-05-27T09:30:30.756 INFO:tasks.workunit.client.0.smithi114.stdout: create 286/35 file after the snapshot 2024-05-27T09:30:30.758 INFO:tasks.workunit.client.0.smithi114.stdout: create 286/36 file after the snapshot 2024-05-27T09:30:30.761 INFO:tasks.workunit.client.0.smithi114.stdout: create 286/37 file after the snapshot 2024-05-27T09:30:30.765 INFO:tasks.workunit.client.0.smithi114.stdout: create 286/38 file after the snapshot 2024-05-27T09:30:30.768 INFO:tasks.workunit.client.0.smithi114.stdout: create 286/39 file after the snapshot 2024-05-27T09:30:30.771 INFO:tasks.workunit.client.0.smithi114.stdout: create 287/.snap/snap-subdir-test 2024-05-27T09:30:30.836 INFO:tasks.workunit.client.0.smithi114.stdout: create 287/30 file after the snapshot 2024-05-27T09:30:30.838 INFO:tasks.workunit.client.0.smithi114.stdout: create 287/31 file after the snapshot 2024-05-27T09:30:30.841 INFO:tasks.workunit.client.0.smithi114.stdout: create 287/32 file after the snapshot 2024-05-27T09:30:30.844 INFO:tasks.workunit.client.0.smithi114.stdout: create 287/33 file after the snapshot 2024-05-27T09:30:30.847 INFO:tasks.workunit.client.0.smithi114.stdout: create 287/34 file after the snapshot 2024-05-27T09:30:30.850 INFO:tasks.workunit.client.0.smithi114.stdout: create 287/35 file after the snapshot 2024-05-27T09:30:30.853 INFO:tasks.workunit.client.0.smithi114.stdout: create 287/36 file after the snapshot 2024-05-27T09:30:30.855 INFO:tasks.workunit.client.0.smithi114.stdout: create 287/37 file after the snapshot 2024-05-27T09:30:30.858 INFO:tasks.workunit.client.0.smithi114.stdout: create 287/38 file after the snapshot 2024-05-27T09:30:30.861 INFO:tasks.workunit.client.0.smithi114.stdout: create 287/39 file after the snapshot 2024-05-27T09:30:30.864 INFO:tasks.workunit.client.0.smithi114.stdout: create 288/.snap/snap-subdir-test 2024-05-27T09:30:30.914 INFO:tasks.workunit.client.0.smithi114.stdout: create 288/30 file after the snapshot 2024-05-27T09:30:30.918 INFO:tasks.workunit.client.0.smithi114.stdout: create 288/31 file after the snapshot 2024-05-27T09:30:30.922 INFO:tasks.workunit.client.0.smithi114.stdout: create 288/32 file after the snapshot 2024-05-27T09:30:30.927 INFO:tasks.workunit.client.0.smithi114.stdout: create 288/33 file after the snapshot 2024-05-27T09:30:30.932 INFO:tasks.workunit.client.0.smithi114.stdout: create 288/34 file after the snapshot 2024-05-27T09:30:30.936 INFO:tasks.workunit.client.0.smithi114.stdout: create 288/35 file after the snapshot 2024-05-27T09:30:30.941 INFO:tasks.workunit.client.0.smithi114.stdout: create 288/36 file after the snapshot 2024-05-27T09:30:30.945 INFO:tasks.workunit.client.0.smithi114.stdout: create 288/37 file after the snapshot 2024-05-27T09:30:30.949 INFO:tasks.workunit.client.0.smithi114.stdout: create 288/38 file after the snapshot 2024-05-27T09:30:30.954 INFO:tasks.workunit.client.0.smithi114.stdout: create 288/39 file after the snapshot 2024-05-27T09:30:30.958 INFO:tasks.workunit.client.0.smithi114.stdout: create 289/.snap/snap-subdir-test 2024-05-27T09:30:31.018 INFO:tasks.workunit.client.0.smithi114.stdout: create 289/30 file after the snapshot 2024-05-27T09:30:31.020 INFO:tasks.workunit.client.0.smithi114.stdout: create 289/31 file after the snapshot 2024-05-27T09:30:31.023 INFO:tasks.workunit.client.0.smithi114.stdout: create 289/32 file after the snapshot 2024-05-27T09:30:31.026 INFO:tasks.workunit.client.0.smithi114.stdout: create 289/33 file after the snapshot 2024-05-27T09:30:31.028 INFO:tasks.workunit.client.0.smithi114.stdout: create 289/34 file after the snapshot 2024-05-27T09:30:31.031 INFO:tasks.workunit.client.0.smithi114.stdout: create 289/35 file after the snapshot 2024-05-27T09:30:31.034 INFO:tasks.workunit.client.0.smithi114.stdout: create 289/36 file after the snapshot 2024-05-27T09:30:31.037 INFO:tasks.workunit.client.0.smithi114.stdout: create 289/37 file after the snapshot 2024-05-27T09:30:31.040 INFO:tasks.workunit.client.0.smithi114.stdout: create 289/38 file after the snapshot 2024-05-27T09:30:31.043 INFO:tasks.workunit.client.0.smithi114.stdout: create 289/39 file after the snapshot 2024-05-27T09:30:31.046 INFO:tasks.workunit.client.0.smithi114.stdout: create 290/.snap/snap-subdir-test 2024-05-27T09:30:31.124 INFO:tasks.workunit.client.0.smithi114.stdout: create 290/30 file after the snapshot 2024-05-27T09:30:31.127 INFO:tasks.workunit.client.0.smithi114.stdout: create 290/31 file after the snapshot 2024-05-27T09:30:31.130 INFO:tasks.workunit.client.0.smithi114.stdout: create 290/32 file after the snapshot 2024-05-27T09:30:31.133 INFO:tasks.workunit.client.0.smithi114.stdout: create 290/33 file after the snapshot 2024-05-27T09:30:31.136 INFO:tasks.workunit.client.0.smithi114.stdout: create 290/34 file after the snapshot 2024-05-27T09:30:31.139 INFO:tasks.workunit.client.0.smithi114.stdout: create 290/35 file after the snapshot 2024-05-27T09:30:31.142 INFO:tasks.workunit.client.0.smithi114.stdout: create 290/36 file after the snapshot 2024-05-27T09:30:31.146 INFO:tasks.workunit.client.0.smithi114.stdout: create 290/37 file after the snapshot 2024-05-27T09:30:31.151 INFO:tasks.workunit.client.0.smithi114.stdout: create 290/38 file after the snapshot 2024-05-27T09:30:31.155 INFO:tasks.workunit.client.0.smithi114.stdout: create 290/39 file after the snapshot 2024-05-27T09:30:31.159 INFO:tasks.workunit.client.0.smithi114.stdout: create 291/.snap/snap-subdir-test 2024-05-27T09:30:31.228 INFO:tasks.workunit.client.0.smithi114.stdout: create 291/30 file after the snapshot 2024-05-27T09:30:31.230 INFO:tasks.workunit.client.0.smithi114.stdout: create 291/31 file after the snapshot 2024-05-27T09:30:31.233 INFO:tasks.workunit.client.0.smithi114.stdout: create 291/32 file after the snapshot 2024-05-27T09:30:31.237 INFO:tasks.workunit.client.0.smithi114.stdout: create 291/33 file after the snapshot 2024-05-27T09:30:31.241 INFO:tasks.workunit.client.0.smithi114.stdout: create 291/34 file after the snapshot 2024-05-27T09:30:31.245 INFO:tasks.workunit.client.0.smithi114.stdout: create 291/35 file after the snapshot 2024-05-27T09:30:31.249 INFO:tasks.workunit.client.0.smithi114.stdout: create 291/36 file after the snapshot 2024-05-27T09:30:31.253 INFO:tasks.workunit.client.0.smithi114.stdout: create 291/37 file after the snapshot 2024-05-27T09:30:31.257 INFO:tasks.workunit.client.0.smithi114.stdout: create 291/38 file after the snapshot 2024-05-27T09:30:31.261 INFO:tasks.workunit.client.0.smithi114.stdout: create 291/39 file after the snapshot 2024-05-27T09:30:31.265 INFO:tasks.workunit.client.0.smithi114.stdout: create 292/.snap/snap-subdir-test 2024-05-27T09:30:31.335 INFO:tasks.workunit.client.0.smithi114.stdout: create 292/30 file after the snapshot 2024-05-27T09:30:31.338 INFO:tasks.workunit.client.0.smithi114.stdout: create 292/31 file after the snapshot 2024-05-27T09:30:31.341 INFO:tasks.workunit.client.0.smithi114.stdout: create 292/32 file after the snapshot 2024-05-27T09:30:31.343 INFO:tasks.workunit.client.0.smithi114.stdout: create 292/33 file after the snapshot 2024-05-27T09:30:31.346 INFO:tasks.workunit.client.0.smithi114.stdout: create 292/34 file after the snapshot 2024-05-27T09:30:31.349 INFO:tasks.workunit.client.0.smithi114.stdout: create 292/35 file after the snapshot 2024-05-27T09:30:31.352 INFO:tasks.workunit.client.0.smithi114.stdout: create 292/36 file after the snapshot 2024-05-27T09:30:31.355 INFO:tasks.workunit.client.0.smithi114.stdout: create 292/37 file after the snapshot 2024-05-27T09:30:31.358 INFO:tasks.workunit.client.0.smithi114.stdout: create 292/38 file after the snapshot 2024-05-27T09:30:31.361 INFO:tasks.workunit.client.0.smithi114.stdout: create 292/39 file after the snapshot 2024-05-27T09:30:31.363 INFO:tasks.workunit.client.0.smithi114.stdout: create 293/.snap/snap-subdir-test 2024-05-27T09:30:31.417 INFO:tasks.workunit.client.0.smithi114.stdout: create 293/30 file after the snapshot 2024-05-27T09:30:31.418 INFO:tasks.workunit.client.0.smithi114.stdout: create 293/31 file after the snapshot 2024-05-27T09:30:31.420 INFO:tasks.workunit.client.0.smithi114.stdout: create 293/32 file after the snapshot 2024-05-27T09:30:31.422 INFO:tasks.workunit.client.0.smithi114.stdout: create 293/33 file after the snapshot 2024-05-27T09:30:31.424 INFO:tasks.workunit.client.0.smithi114.stdout: create 293/34 file after the snapshot 2024-05-27T09:30:31.426 INFO:tasks.workunit.client.0.smithi114.stdout: create 293/35 file after the snapshot 2024-05-27T09:30:31.428 INFO:tasks.workunit.client.0.smithi114.stdout: create 293/36 file after the snapshot 2024-05-27T09:30:31.431 INFO:tasks.workunit.client.0.smithi114.stdout: create 293/37 file after the snapshot 2024-05-27T09:30:31.434 INFO:tasks.workunit.client.0.smithi114.stdout: create 293/38 file after the snapshot 2024-05-27T09:30:31.437 INFO:tasks.workunit.client.0.smithi114.stdout: create 293/39 file after the snapshot 2024-05-27T09:30:31.440 INFO:tasks.workunit.client.0.smithi114.stdout: create 294/.snap/snap-subdir-test 2024-05-27T09:30:31.492 INFO:tasks.workunit.client.0.smithi114.stdout: create 294/30 file after the snapshot 2024-05-27T09:30:31.493 INFO:tasks.workunit.client.0.smithi114.stdout: create 294/31 file after the snapshot 2024-05-27T09:30:31.495 INFO:tasks.workunit.client.0.smithi114.stdout: create 294/32 file after the snapshot 2024-05-27T09:30:31.497 INFO:tasks.workunit.client.0.smithi114.stdout: create 294/33 file after the snapshot 2024-05-27T09:30:31.499 INFO:tasks.workunit.client.0.smithi114.stdout: create 294/34 file after the snapshot 2024-05-27T09:30:31.502 INFO:tasks.workunit.client.0.smithi114.stdout: create 294/35 file after the snapshot 2024-05-27T09:30:31.505 INFO:tasks.workunit.client.0.smithi114.stdout: create 294/36 file after the snapshot 2024-05-27T09:30:31.508 INFO:tasks.workunit.client.0.smithi114.stdout: create 294/37 file after the snapshot 2024-05-27T09:30:31.510 INFO:tasks.workunit.client.0.smithi114.stdout: create 294/38 file after the snapshot 2024-05-27T09:30:31.513 INFO:tasks.workunit.client.0.smithi114.stdout: create 294/39 file after the snapshot 2024-05-27T09:30:31.516 INFO:tasks.workunit.client.0.smithi114.stdout: create 295/.snap/snap-subdir-test 2024-05-27T09:30:31.579 INFO:tasks.workunit.client.0.smithi114.stdout: create 295/30 file after the snapshot 2024-05-27T09:30:31.582 INFO:tasks.workunit.client.0.smithi114.stdout: create 295/31 file after the snapshot 2024-05-27T09:30:31.585 INFO:tasks.workunit.client.0.smithi114.stdout: create 295/32 file after the snapshot 2024-05-27T09:30:31.588 INFO:tasks.workunit.client.0.smithi114.stdout: create 295/33 file after the snapshot 2024-05-27T09:30:31.591 INFO:tasks.workunit.client.0.smithi114.stdout: create 295/34 file after the snapshot 2024-05-27T09:30:31.594 INFO:tasks.workunit.client.0.smithi114.stdout: create 295/35 file after the snapshot 2024-05-27T09:30:31.597 INFO:tasks.workunit.client.0.smithi114.stdout: create 295/36 file after the snapshot 2024-05-27T09:30:31.600 INFO:tasks.workunit.client.0.smithi114.stdout: create 295/37 file after the snapshot 2024-05-27T09:30:31.602 INFO:tasks.workunit.client.0.smithi114.stdout: create 295/38 file after the snapshot 2024-05-27T09:30:31.605 INFO:tasks.workunit.client.0.smithi114.stdout: create 295/39 file after the snapshot 2024-05-27T09:30:31.608 INFO:tasks.workunit.client.0.smithi114.stdout: create 296/.snap/snap-subdir-test 2024-05-27T09:30:31.664 INFO:tasks.workunit.client.0.smithi114.stdout: create 296/30 file after the snapshot 2024-05-27T09:30:31.666 INFO:tasks.workunit.client.0.smithi114.stdout: create 296/31 file after the snapshot 2024-05-27T09:30:31.667 INFO:tasks.workunit.client.0.smithi114.stdout: create 296/32 file after the snapshot 2024-05-27T09:30:31.669 INFO:tasks.workunit.client.0.smithi114.stdout: create 296/33 file after the snapshot 2024-05-27T09:30:31.671 INFO:tasks.workunit.client.0.smithi114.stdout: create 296/34 file after the snapshot 2024-05-27T09:30:31.672 INFO:tasks.workunit.client.0.smithi114.stdout: create 296/35 file after the snapshot 2024-05-27T09:30:31.675 INFO:tasks.workunit.client.0.smithi114.stdout: create 296/36 file after the snapshot 2024-05-27T09:30:31.677 INFO:tasks.workunit.client.0.smithi114.stdout: create 296/37 file after the snapshot 2024-05-27T09:30:31.680 INFO:tasks.workunit.client.0.smithi114.stdout: create 296/38 file after the snapshot 2024-05-27T09:30:31.683 INFO:tasks.workunit.client.0.smithi114.stdout: create 296/39 file after the snapshot 2024-05-27T09:30:31.685 INFO:tasks.workunit.client.0.smithi114.stdout: create 297/.snap/snap-subdir-test 2024-05-27T09:30:31.755 INFO:tasks.workunit.client.0.smithi114.stdout: create 297/30 file after the snapshot 2024-05-27T09:30:31.758 INFO:tasks.workunit.client.0.smithi114.stdout: create 297/31 file after the snapshot 2024-05-27T09:30:31.761 INFO:tasks.workunit.client.0.smithi114.stdout: create 297/32 file after the snapshot 2024-05-27T09:30:31.763 INFO:tasks.workunit.client.0.smithi114.stdout: create 297/33 file after the snapshot 2024-05-27T09:30:31.766 INFO:tasks.workunit.client.0.smithi114.stdout: create 297/34 file after the snapshot 2024-05-27T09:30:31.769 INFO:tasks.workunit.client.0.smithi114.stdout: create 297/35 file after the snapshot 2024-05-27T09:30:31.772 INFO:tasks.workunit.client.0.smithi114.stdout: create 297/36 file after the snapshot 2024-05-27T09:30:31.775 INFO:tasks.workunit.client.0.smithi114.stdout: create 297/37 file after the snapshot 2024-05-27T09:30:31.778 INFO:tasks.workunit.client.0.smithi114.stdout: create 297/38 file after the snapshot 2024-05-27T09:30:31.781 INFO:tasks.workunit.client.0.smithi114.stdout: create 297/39 file after the snapshot 2024-05-27T09:30:31.783 INFO:tasks.workunit.client.0.smithi114.stdout: create 298/.snap/snap-subdir-test 2024-05-27T09:30:31.846 INFO:tasks.workunit.client.0.smithi114.stdout: create 298/30 file after the snapshot 2024-05-27T09:30:31.847 INFO:tasks.workunit.client.0.smithi114.stdout: create 298/31 file after the snapshot 2024-05-27T09:30:31.849 INFO:tasks.workunit.client.0.smithi114.stdout: create 298/32 file after the snapshot 2024-05-27T09:30:31.850 INFO:tasks.workunit.client.0.smithi114.stdout: create 298/33 file after the snapshot 2024-05-27T09:30:31.852 INFO:tasks.workunit.client.0.smithi114.stdout: create 298/34 file after the snapshot 2024-05-27T09:30:31.854 INFO:tasks.workunit.client.0.smithi114.stdout: create 298/35 file after the snapshot 2024-05-27T09:30:31.855 INFO:tasks.workunit.client.0.smithi114.stdout: create 298/36 file after the snapshot 2024-05-27T09:30:31.858 INFO:tasks.workunit.client.0.smithi114.stdout: create 298/37 file after the snapshot 2024-05-27T09:30:31.862 INFO:tasks.workunit.client.0.smithi114.stdout: create 298/38 file after the snapshot 2024-05-27T09:30:31.865 INFO:tasks.workunit.client.0.smithi114.stdout: create 298/39 file after the snapshot 2024-05-27T09:30:31.868 INFO:tasks.workunit.client.0.smithi114.stdout: create 299/.snap/snap-subdir-test 2024-05-27T09:30:31.928 INFO:tasks.workunit.client.0.smithi114.stdout: create 299/30 file after the snapshot 2024-05-27T09:30:31.930 INFO:tasks.workunit.client.0.smithi114.stdout: create 299/31 file after the snapshot 2024-05-27T09:30:31.934 INFO:tasks.workunit.client.0.smithi114.stdout: create 299/32 file after the snapshot 2024-05-27T09:30:31.936 INFO:tasks.workunit.client.0.smithi114.stdout: create 299/33 file after the snapshot 2024-05-27T09:30:31.939 INFO:tasks.workunit.client.0.smithi114.stdout: create 299/34 file after the snapshot 2024-05-27T09:30:31.942 INFO:tasks.workunit.client.0.smithi114.stdout: create 299/35 file after the snapshot 2024-05-27T09:30:31.945 INFO:tasks.workunit.client.0.smithi114.stdout: create 299/36 file after the snapshot 2024-05-27T09:30:31.947 INFO:tasks.workunit.client.0.smithi114.stdout: create 299/37 file after the snapshot 2024-05-27T09:30:31.950 INFO:tasks.workunit.client.0.smithi114.stdout: create 299/38 file after the snapshot 2024-05-27T09:30:31.953 INFO:tasks.workunit.client.0.smithi114.stdout: create 299/39 file after the snapshot 2024-05-27T09:30:31.955 INFO:tasks.workunit.client.0.smithi114.stdout:Take second snapshot .snap/test2 2024-05-27T09:30:32.005 INFO:tasks.workunit.client.0.smithi114.stdout:Copy content of .snap/test1 to copyofsnap1 ... 2024-05-27T09:30:32.476 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1' -> 'copyofsnap1/test1' 2024-05-27T09:30:32.477 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/100' -> 'copyofsnap1/test1/100' 2024-05-27T09:30:32.477 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/100/10' -> 'copyofsnap1/test1/100/10' 2024-05-27T09:30:32.477 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/100/11' -> 'copyofsnap1/test1/100/11' 2024-05-27T09:30:32.477 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/100/12' -> 'copyofsnap1/test1/100/12' 2024-05-27T09:30:32.477 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/100/13' -> 'copyofsnap1/test1/100/13' 2024-05-27T09:30:32.477 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/100/14' -> 'copyofsnap1/test1/100/14' 2024-05-27T09:30:32.477 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/100/15' -> 'copyofsnap1/test1/100/15' 2024-05-27T09:30:32.477 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/100/16' -> 'copyofsnap1/test1/100/16' 2024-05-27T09:30:32.477 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/100/17' -> 'copyofsnap1/test1/100/17' 2024-05-27T09:30:32.477 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/100/18' -> 'copyofsnap1/test1/100/18' 2024-05-27T09:30:32.477 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/100/19' -> 'copyofsnap1/test1/100/19' 2024-05-27T09:30:32.478 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/100/20' -> 'copyofsnap1/test1/100/20' 2024-05-27T09:30:32.478 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/101' -> 'copyofsnap1/test1/101' 2024-05-27T09:30:32.478 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/101/10' -> 'copyofsnap1/test1/101/10' 2024-05-27T09:30:32.478 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/101/11' -> 'copyofsnap1/test1/101/11' 2024-05-27T09:30:32.478 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/101/12' -> 'copyofsnap1/test1/101/12' 2024-05-27T09:30:32.478 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/101/13' -> 'copyofsnap1/test1/101/13' 2024-05-27T09:30:32.478 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/101/14' -> 'copyofsnap1/test1/101/14' 2024-05-27T09:30:32.478 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/101/15' -> 'copyofsnap1/test1/101/15' 2024-05-27T09:30:32.478 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/101/16' -> 'copyofsnap1/test1/101/16' 2024-05-27T09:30:32.478 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/101/17' -> 'copyofsnap1/test1/101/17' 2024-05-27T09:30:32.478 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/101/18' -> 'copyofsnap1/test1/101/18' 2024-05-27T09:30:32.478 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/101/19' -> 'copyofsnap1/test1/101/19' 2024-05-27T09:30:32.478 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/101/20' -> 'copyofsnap1/test1/101/20' 2024-05-27T09:30:32.479 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/102' -> 'copyofsnap1/test1/102' 2024-05-27T09:30:32.479 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/102/10' -> 'copyofsnap1/test1/102/10' 2024-05-27T09:30:32.479 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/102/11' -> 'copyofsnap1/test1/102/11' 2024-05-27T09:30:32.479 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/102/12' -> 'copyofsnap1/test1/102/12' 2024-05-27T09:30:32.479 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/102/13' -> 'copyofsnap1/test1/102/13' 2024-05-27T09:30:32.479 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/102/14' -> 'copyofsnap1/test1/102/14' 2024-05-27T09:30:32.479 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/102/15' -> 'copyofsnap1/test1/102/15' 2024-05-27T09:30:32.479 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/102/16' -> 'copyofsnap1/test1/102/16' 2024-05-27T09:30:32.479 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/102/17' -> 'copyofsnap1/test1/102/17' 2024-05-27T09:30:32.479 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/102/18' -> 'copyofsnap1/test1/102/18' 2024-05-27T09:30:32.479 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/102/19' -> 'copyofsnap1/test1/102/19' 2024-05-27T09:30:32.479 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/102/20' -> 'copyofsnap1/test1/102/20' 2024-05-27T09:30:32.480 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/103' -> 'copyofsnap1/test1/103' 2024-05-27T09:30:32.480 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/103/10' -> 'copyofsnap1/test1/103/10' 2024-05-27T09:30:32.480 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/103/11' -> 'copyofsnap1/test1/103/11' 2024-05-27T09:30:32.480 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/103/12' -> 'copyofsnap1/test1/103/12' 2024-05-27T09:30:32.480 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/103/13' -> 'copyofsnap1/test1/103/13' 2024-05-27T09:30:32.480 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/103/14' -> 'copyofsnap1/test1/103/14' 2024-05-27T09:30:32.480 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/103/15' -> 'copyofsnap1/test1/103/15' 2024-05-27T09:30:32.480 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/103/16' -> 'copyofsnap1/test1/103/16' 2024-05-27T09:30:32.480 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/103/17' -> 'copyofsnap1/test1/103/17' 2024-05-27T09:30:32.480 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/103/18' -> 'copyofsnap1/test1/103/18' 2024-05-27T09:30:32.480 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/103/19' -> 'copyofsnap1/test1/103/19' 2024-05-27T09:30:32.480 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/103/20' -> 'copyofsnap1/test1/103/20' 2024-05-27T09:30:32.480 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/104' -> 'copyofsnap1/test1/104' 2024-05-27T09:30:32.481 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/104/10' -> 'copyofsnap1/test1/104/10' 2024-05-27T09:30:32.481 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/104/11' -> 'copyofsnap1/test1/104/11' 2024-05-27T09:30:32.481 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/104/12' -> 'copyofsnap1/test1/104/12' 2024-05-27T09:30:32.481 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/104/13' -> 'copyofsnap1/test1/104/13' 2024-05-27T09:30:32.481 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/104/14' -> 'copyofsnap1/test1/104/14' 2024-05-27T09:30:32.481 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/104/15' -> 'copyofsnap1/test1/104/15' 2024-05-27T09:30:32.481 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/104/16' -> 'copyofsnap1/test1/104/16' 2024-05-27T09:30:32.481 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/104/17' -> 'copyofsnap1/test1/104/17' 2024-05-27T09:30:32.481 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/104/18' -> 'copyofsnap1/test1/104/18' 2024-05-27T09:30:32.481 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/104/19' -> 'copyofsnap1/test1/104/19' 2024-05-27T09:30:32.481 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/104/20' -> 'copyofsnap1/test1/104/20' 2024-05-27T09:30:32.481 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/105' -> 'copyofsnap1/test1/105' 2024-05-27T09:30:32.481 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/105/10' -> 'copyofsnap1/test1/105/10' 2024-05-27T09:30:32.482 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/105/11' -> 'copyofsnap1/test1/105/11' 2024-05-27T09:30:32.482 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/105/12' -> 'copyofsnap1/test1/105/12' 2024-05-27T09:30:32.482 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/105/13' -> 'copyofsnap1/test1/105/13' 2024-05-27T09:30:32.482 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/105/14' -> 'copyofsnap1/test1/105/14' 2024-05-27T09:30:32.482 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/105/15' -> 'copyofsnap1/test1/105/15' 2024-05-27T09:30:32.482 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/105/16' -> 'copyofsnap1/test1/105/16' 2024-05-27T09:30:32.482 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/105/17' -> 'copyofsnap1/test1/105/17' 2024-05-27T09:30:32.482 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/105/18' -> 'copyofsnap1/test1/105/18' 2024-05-27T09:30:32.482 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/105/19' -> 'copyofsnap1/test1/105/19' 2024-05-27T09:30:32.483 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/105/20' -> 'copyofsnap1/test1/105/20' 2024-05-27T09:30:32.483 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/106' -> 'copyofsnap1/test1/106' 2024-05-27T09:30:32.483 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/106/10' -> 'copyofsnap1/test1/106/10' 2024-05-27T09:30:32.483 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/106/11' -> 'copyofsnap1/test1/106/11' 2024-05-27T09:30:32.483 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/106/12' -> 'copyofsnap1/test1/106/12' 2024-05-27T09:30:32.483 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/106/13' -> 'copyofsnap1/test1/106/13' 2024-05-27T09:30:32.483 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/106/14' -> 'copyofsnap1/test1/106/14' 2024-05-27T09:30:32.483 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/106/15' -> 'copyofsnap1/test1/106/15' 2024-05-27T09:30:32.483 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/106/16' -> 'copyofsnap1/test1/106/16' 2024-05-27T09:30:32.881 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/106/17' -> 'copyofsnap1/test1/106/17' 2024-05-27T09:30:32.882 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/106/18' -> 'copyofsnap1/test1/106/18' 2024-05-27T09:30:32.882 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/106/19' -> 'copyofsnap1/test1/106/19' 2024-05-27T09:30:32.882 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/106/20' -> 'copyofsnap1/test1/106/20' 2024-05-27T09:30:32.882 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/107' -> 'copyofsnap1/test1/107' 2024-05-27T09:30:32.882 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/107/10' -> 'copyofsnap1/test1/107/10' 2024-05-27T09:30:32.882 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/107/11' -> 'copyofsnap1/test1/107/11' 2024-05-27T09:30:32.882 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/107/12' -> 'copyofsnap1/test1/107/12' 2024-05-27T09:30:32.882 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/107/13' -> 'copyofsnap1/test1/107/13' 2024-05-27T09:30:32.882 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/107/14' -> 'copyofsnap1/test1/107/14' 2024-05-27T09:30:32.882 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/107/15' -> 'copyofsnap1/test1/107/15' 2024-05-27T09:30:32.882 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/107/16' -> 'copyofsnap1/test1/107/16' 2024-05-27T09:30:32.883 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/107/17' -> 'copyofsnap1/test1/107/17' 2024-05-27T09:30:32.883 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/107/18' -> 'copyofsnap1/test1/107/18' 2024-05-27T09:30:32.883 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/107/19' -> 'copyofsnap1/test1/107/19' 2024-05-27T09:30:32.883 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/107/20' -> 'copyofsnap1/test1/107/20' 2024-05-27T09:30:32.883 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/108' -> 'copyofsnap1/test1/108' 2024-05-27T09:30:32.883 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/108/10' -> 'copyofsnap1/test1/108/10' 2024-05-27T09:30:32.883 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/108/11' -> 'copyofsnap1/test1/108/11' 2024-05-27T09:30:32.883 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/108/12' -> 'copyofsnap1/test1/108/12' 2024-05-27T09:30:32.883 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/108/13' -> 'copyofsnap1/test1/108/13' 2024-05-27T09:30:32.883 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/108/14' -> 'copyofsnap1/test1/108/14' 2024-05-27T09:30:32.883 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/108/15' -> 'copyofsnap1/test1/108/15' 2024-05-27T09:30:32.883 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/108/16' -> 'copyofsnap1/test1/108/16' 2024-05-27T09:30:32.883 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/108/17' -> 'copyofsnap1/test1/108/17' 2024-05-27T09:30:32.884 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/108/18' -> 'copyofsnap1/test1/108/18' 2024-05-27T09:30:32.884 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/108/19' -> 'copyofsnap1/test1/108/19' 2024-05-27T09:30:32.884 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/108/20' -> 'copyofsnap1/test1/108/20' 2024-05-27T09:30:32.884 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/109' -> 'copyofsnap1/test1/109' 2024-05-27T09:30:32.884 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/109/10' -> 'copyofsnap1/test1/109/10' 2024-05-27T09:30:32.884 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/109/11' -> 'copyofsnap1/test1/109/11' 2024-05-27T09:30:32.884 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/109/12' -> 'copyofsnap1/test1/109/12' 2024-05-27T09:30:32.884 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/109/13' -> 'copyofsnap1/test1/109/13' 2024-05-27T09:30:32.884 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/109/14' -> 'copyofsnap1/test1/109/14' 2024-05-27T09:30:32.884 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/109/15' -> 'copyofsnap1/test1/109/15' 2024-05-27T09:30:32.884 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/109/16' -> 'copyofsnap1/test1/109/16' 2024-05-27T09:30:32.884 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/109/17' -> 'copyofsnap1/test1/109/17' 2024-05-27T09:30:32.884 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/109/18' -> 'copyofsnap1/test1/109/18' 2024-05-27T09:30:32.885 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/109/19' -> 'copyofsnap1/test1/109/19' 2024-05-27T09:30:32.885 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/109/20' -> 'copyofsnap1/test1/109/20' 2024-05-27T09:30:32.885 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/110' -> 'copyofsnap1/test1/110' 2024-05-27T09:30:32.885 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/110/10' -> 'copyofsnap1/test1/110/10' 2024-05-27T09:30:32.885 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/110/11' -> 'copyofsnap1/test1/110/11' 2024-05-27T09:30:32.885 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/110/12' -> 'copyofsnap1/test1/110/12' 2024-05-27T09:30:32.885 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/110/13' -> 'copyofsnap1/test1/110/13' 2024-05-27T09:30:32.885 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/110/14' -> 'copyofsnap1/test1/110/14' 2024-05-27T09:30:32.885 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/110/15' -> 'copyofsnap1/test1/110/15' 2024-05-27T09:30:32.885 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/110/16' -> 'copyofsnap1/test1/110/16' 2024-05-27T09:30:32.885 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/110/17' -> 'copyofsnap1/test1/110/17' 2024-05-27T09:30:32.885 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/110/18' -> 'copyofsnap1/test1/110/18' 2024-05-27T09:30:32.885 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/110/19' -> 'copyofsnap1/test1/110/19' 2024-05-27T09:30:32.886 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/110/20' -> 'copyofsnap1/test1/110/20' 2024-05-27T09:30:32.886 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/111' -> 'copyofsnap1/test1/111' 2024-05-27T09:30:32.886 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/111/10' -> 'copyofsnap1/test1/111/10' 2024-05-27T09:30:32.886 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/111/11' -> 'copyofsnap1/test1/111/11' 2024-05-27T09:30:32.886 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/111/12' -> 'copyofsnap1/test1/111/12' 2024-05-27T09:30:32.886 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/111/13' -> 'copyofsnap1/test1/111/13' 2024-05-27T09:30:32.886 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/111/14' -> 'copyofsnap1/test1/111/14' 2024-05-27T09:30:32.886 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/111/15' -> 'copyofsnap1/test1/111/15' 2024-05-27T09:30:32.886 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/111/16' -> 'copyofsnap1/test1/111/16' 2024-05-27T09:30:32.886 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/111/17' -> 'copyofsnap1/test1/111/17' 2024-05-27T09:30:32.886 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/111/18' -> 'copyofsnap1/test1/111/18' 2024-05-27T09:30:32.886 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/111/19' -> 'copyofsnap1/test1/111/19' 2024-05-27T09:30:32.886 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/111/20' -> 'copyofsnap1/test1/111/20' 2024-05-27T09:30:32.886 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/112' -> 'copyofsnap1/test1/112' 2024-05-27T09:30:32.887 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/112/10' -> 'copyofsnap1/test1/112/10' 2024-05-27T09:30:32.887 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/112/11' -> 'copyofsnap1/test1/112/11' 2024-05-27T09:30:32.887 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/112/12' -> 'copyofsnap1/test1/112/12' 2024-05-27T09:30:32.887 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/112/13' -> 'copyofsnap1/test1/112/13' 2024-05-27T09:30:32.887 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/112/14' -> 'copyofsnap1/test1/112/14' 2024-05-27T09:30:32.887 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/112/15' -> 'copyofsnap1/test1/112/15' 2024-05-27T09:30:32.887 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/112/16' -> 'copyofsnap1/test1/112/16' 2024-05-27T09:30:32.887 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/112/17' -> 'copyofsnap1/test1/112/17' 2024-05-27T09:30:32.887 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/112/18' -> 'copyofsnap1/test1/112/18' 2024-05-27T09:30:32.887 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/112/19' -> 'copyofsnap1/test1/112/19' 2024-05-27T09:30:32.887 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/112/20' -> 'copyofsnap1/test1/112/20' 2024-05-27T09:30:32.887 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/113' -> 'copyofsnap1/test1/113' 2024-05-27T09:30:32.887 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/113/10' -> 'copyofsnap1/test1/113/10' 2024-05-27T09:30:32.887 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/113/11' -> 'copyofsnap1/test1/113/11' 2024-05-27T09:30:32.888 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/113/12' -> 'copyofsnap1/test1/113/12' 2024-05-27T09:30:32.888 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/113/13' -> 'copyofsnap1/test1/113/13' 2024-05-27T09:30:33.260 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/113/14' -> 'copyofsnap1/test1/113/14' 2024-05-27T09:30:33.260 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/113/15' -> 'copyofsnap1/test1/113/15' 2024-05-27T09:30:33.260 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/113/16' -> 'copyofsnap1/test1/113/16' 2024-05-27T09:30:33.260 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/113/17' -> 'copyofsnap1/test1/113/17' 2024-05-27T09:30:33.260 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/113/18' -> 'copyofsnap1/test1/113/18' 2024-05-27T09:30:33.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/113/19' -> 'copyofsnap1/test1/113/19' 2024-05-27T09:30:33.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/113/20' -> 'copyofsnap1/test1/113/20' 2024-05-27T09:30:33.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/114' -> 'copyofsnap1/test1/114' 2024-05-27T09:30:33.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/114/10' -> 'copyofsnap1/test1/114/10' 2024-05-27T09:30:33.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/114/11' -> 'copyofsnap1/test1/114/11' 2024-05-27T09:30:33.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/114/12' -> 'copyofsnap1/test1/114/12' 2024-05-27T09:30:33.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/114/13' -> 'copyofsnap1/test1/114/13' 2024-05-27T09:30:33.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/114/14' -> 'copyofsnap1/test1/114/14' 2024-05-27T09:30:33.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/114/15' -> 'copyofsnap1/test1/114/15' 2024-05-27T09:30:33.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/114/16' -> 'copyofsnap1/test1/114/16' 2024-05-27T09:30:33.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/114/17' -> 'copyofsnap1/test1/114/17' 2024-05-27T09:30:33.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/114/18' -> 'copyofsnap1/test1/114/18' 2024-05-27T09:30:33.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/114/19' -> 'copyofsnap1/test1/114/19' 2024-05-27T09:30:33.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/114/20' -> 'copyofsnap1/test1/114/20' 2024-05-27T09:30:33.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/115' -> 'copyofsnap1/test1/115' 2024-05-27T09:30:33.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/115/10' -> 'copyofsnap1/test1/115/10' 2024-05-27T09:30:33.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/115/11' -> 'copyofsnap1/test1/115/11' 2024-05-27T09:30:33.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/115/12' -> 'copyofsnap1/test1/115/12' 2024-05-27T09:30:33.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/115/13' -> 'copyofsnap1/test1/115/13' 2024-05-27T09:30:33.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/115/14' -> 'copyofsnap1/test1/115/14' 2024-05-27T09:30:33.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/115/15' -> 'copyofsnap1/test1/115/15' 2024-05-27T09:30:33.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/115/16' -> 'copyofsnap1/test1/115/16' 2024-05-27T09:30:33.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/115/17' -> 'copyofsnap1/test1/115/17' 2024-05-27T09:30:33.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/115/18' -> 'copyofsnap1/test1/115/18' 2024-05-27T09:30:33.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/115/19' -> 'copyofsnap1/test1/115/19' 2024-05-27T09:30:33.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/115/20' -> 'copyofsnap1/test1/115/20' 2024-05-27T09:30:33.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/116' -> 'copyofsnap1/test1/116' 2024-05-27T09:30:33.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/116/10' -> 'copyofsnap1/test1/116/10' 2024-05-27T09:30:33.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/116/11' -> 'copyofsnap1/test1/116/11' 2024-05-27T09:30:33.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/116/12' -> 'copyofsnap1/test1/116/12' 2024-05-27T09:30:33.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/116/13' -> 'copyofsnap1/test1/116/13' 2024-05-27T09:30:33.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/116/14' -> 'copyofsnap1/test1/116/14' 2024-05-27T09:30:33.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/116/15' -> 'copyofsnap1/test1/116/15' 2024-05-27T09:30:33.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/116/16' -> 'copyofsnap1/test1/116/16' 2024-05-27T09:30:33.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/116/17' -> 'copyofsnap1/test1/116/17' 2024-05-27T09:30:33.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/116/18' -> 'copyofsnap1/test1/116/18' 2024-05-27T09:30:33.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/116/19' -> 'copyofsnap1/test1/116/19' 2024-05-27T09:30:33.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/116/20' -> 'copyofsnap1/test1/116/20' 2024-05-27T09:30:33.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/117' -> 'copyofsnap1/test1/117' 2024-05-27T09:30:33.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/117/10' -> 'copyofsnap1/test1/117/10' 2024-05-27T09:30:33.264 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/117/11' -> 'copyofsnap1/test1/117/11' 2024-05-27T09:30:33.264 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/117/12' -> 'copyofsnap1/test1/117/12' 2024-05-27T09:30:33.264 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/117/13' -> 'copyofsnap1/test1/117/13' 2024-05-27T09:30:33.264 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/117/14' -> 'copyofsnap1/test1/117/14' 2024-05-27T09:30:33.264 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/117/15' -> 'copyofsnap1/test1/117/15' 2024-05-27T09:30:33.264 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/117/16' -> 'copyofsnap1/test1/117/16' 2024-05-27T09:30:33.264 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/117/17' -> 'copyofsnap1/test1/117/17' 2024-05-27T09:30:33.264 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/117/18' -> 'copyofsnap1/test1/117/18' 2024-05-27T09:30:33.264 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/117/19' -> 'copyofsnap1/test1/117/19' 2024-05-27T09:30:33.264 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/117/20' -> 'copyofsnap1/test1/117/20' 2024-05-27T09:30:33.264 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/118' -> 'copyofsnap1/test1/118' 2024-05-27T09:30:33.264 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/118/10' -> 'copyofsnap1/test1/118/10' 2024-05-27T09:30:33.264 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/118/11' -> 'copyofsnap1/test1/118/11' 2024-05-27T09:30:33.264 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/118/12' -> 'copyofsnap1/test1/118/12' 2024-05-27T09:30:33.265 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/118/13' -> 'copyofsnap1/test1/118/13' 2024-05-27T09:30:33.265 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/118/14' -> 'copyofsnap1/test1/118/14' 2024-05-27T09:30:33.265 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/118/15' -> 'copyofsnap1/test1/118/15' 2024-05-27T09:30:33.265 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/118/16' -> 'copyofsnap1/test1/118/16' 2024-05-27T09:30:33.265 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/118/17' -> 'copyofsnap1/test1/118/17' 2024-05-27T09:30:33.265 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/118/18' -> 'copyofsnap1/test1/118/18' 2024-05-27T09:30:33.265 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/118/19' -> 'copyofsnap1/test1/118/19' 2024-05-27T09:30:33.265 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/118/20' -> 'copyofsnap1/test1/118/20' 2024-05-27T09:30:33.265 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/119' -> 'copyofsnap1/test1/119' 2024-05-27T09:30:33.265 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/119/10' -> 'copyofsnap1/test1/119/10' 2024-05-27T09:30:33.265 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/119/11' -> 'copyofsnap1/test1/119/11' 2024-05-27T09:30:33.265 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/119/12' -> 'copyofsnap1/test1/119/12' 2024-05-27T09:30:33.265 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/119/13' -> 'copyofsnap1/test1/119/13' 2024-05-27T09:30:33.266 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/119/14' -> 'copyofsnap1/test1/119/14' 2024-05-27T09:30:33.266 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/119/15' -> 'copyofsnap1/test1/119/15' 2024-05-27T09:30:33.266 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/119/16' -> 'copyofsnap1/test1/119/16' 2024-05-27T09:30:33.266 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/119/17' -> 'copyofsnap1/test1/119/17' 2024-05-27T09:30:33.266 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/119/18' -> 'copyofsnap1/test1/119/18' 2024-05-27T09:30:33.266 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/119/19' -> 'copyofsnap1/test1/119/19' 2024-05-27T09:30:33.266 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/119/20' -> 'copyofsnap1/test1/119/20' 2024-05-27T09:30:33.266 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/120' -> 'copyofsnap1/test1/120' 2024-05-27T09:30:33.266 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/120/10' -> 'copyofsnap1/test1/120/10' 2024-05-27T09:30:33.640 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/120/11' -> 'copyofsnap1/test1/120/11' 2024-05-27T09:30:33.640 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/120/12' -> 'copyofsnap1/test1/120/12' 2024-05-27T09:30:33.640 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/120/13' -> 'copyofsnap1/test1/120/13' 2024-05-27T09:30:33.640 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/120/14' -> 'copyofsnap1/test1/120/14' 2024-05-27T09:30:33.640 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/120/15' -> 'copyofsnap1/test1/120/15' 2024-05-27T09:30:33.640 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/120/16' -> 'copyofsnap1/test1/120/16' 2024-05-27T09:30:33.640 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/120/17' -> 'copyofsnap1/test1/120/17' 2024-05-27T09:30:33.641 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/120/18' -> 'copyofsnap1/test1/120/18' 2024-05-27T09:30:33.641 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/120/19' -> 'copyofsnap1/test1/120/19' 2024-05-27T09:30:33.641 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/120/20' -> 'copyofsnap1/test1/120/20' 2024-05-27T09:30:33.641 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/121' -> 'copyofsnap1/test1/121' 2024-05-27T09:30:33.641 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/121/10' -> 'copyofsnap1/test1/121/10' 2024-05-27T09:30:33.641 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/121/11' -> 'copyofsnap1/test1/121/11' 2024-05-27T09:30:33.641 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/121/12' -> 'copyofsnap1/test1/121/12' 2024-05-27T09:30:33.641 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/121/13' -> 'copyofsnap1/test1/121/13' 2024-05-27T09:30:33.641 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/121/14' -> 'copyofsnap1/test1/121/14' 2024-05-27T09:30:33.641 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/121/15' -> 'copyofsnap1/test1/121/15' 2024-05-27T09:30:33.641 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/121/16' -> 'copyofsnap1/test1/121/16' 2024-05-27T09:30:33.641 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/121/17' -> 'copyofsnap1/test1/121/17' 2024-05-27T09:30:33.641 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/121/18' -> 'copyofsnap1/test1/121/18' 2024-05-27T09:30:33.641 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/121/19' -> 'copyofsnap1/test1/121/19' 2024-05-27T09:30:33.642 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/121/20' -> 'copyofsnap1/test1/121/20' 2024-05-27T09:30:33.642 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/122' -> 'copyofsnap1/test1/122' 2024-05-27T09:30:33.642 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/122/10' -> 'copyofsnap1/test1/122/10' 2024-05-27T09:30:33.642 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/122/11' -> 'copyofsnap1/test1/122/11' 2024-05-27T09:30:33.642 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/122/12' -> 'copyofsnap1/test1/122/12' 2024-05-27T09:30:33.642 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/122/13' -> 'copyofsnap1/test1/122/13' 2024-05-27T09:30:33.642 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/122/14' -> 'copyofsnap1/test1/122/14' 2024-05-27T09:30:33.642 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/122/15' -> 'copyofsnap1/test1/122/15' 2024-05-27T09:30:33.642 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/122/16' -> 'copyofsnap1/test1/122/16' 2024-05-27T09:30:33.642 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/122/17' -> 'copyofsnap1/test1/122/17' 2024-05-27T09:30:33.642 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/122/18' -> 'copyofsnap1/test1/122/18' 2024-05-27T09:30:33.642 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/122/19' -> 'copyofsnap1/test1/122/19' 2024-05-27T09:30:33.642 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/122/20' -> 'copyofsnap1/test1/122/20' 2024-05-27T09:30:33.643 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/123' -> 'copyofsnap1/test1/123' 2024-05-27T09:30:33.643 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/123/10' -> 'copyofsnap1/test1/123/10' 2024-05-27T09:30:33.643 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/123/11' -> 'copyofsnap1/test1/123/11' 2024-05-27T09:30:33.643 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/123/12' -> 'copyofsnap1/test1/123/12' 2024-05-27T09:30:33.643 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/123/13' -> 'copyofsnap1/test1/123/13' 2024-05-27T09:30:33.643 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/123/14' -> 'copyofsnap1/test1/123/14' 2024-05-27T09:30:33.643 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/123/15' -> 'copyofsnap1/test1/123/15' 2024-05-27T09:30:33.643 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/123/16' -> 'copyofsnap1/test1/123/16' 2024-05-27T09:30:33.643 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/123/17' -> 'copyofsnap1/test1/123/17' 2024-05-27T09:30:33.643 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/123/18' -> 'copyofsnap1/test1/123/18' 2024-05-27T09:30:33.643 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/123/19' -> 'copyofsnap1/test1/123/19' 2024-05-27T09:30:33.643 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/123/20' -> 'copyofsnap1/test1/123/20' 2024-05-27T09:30:33.643 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/124' -> 'copyofsnap1/test1/124' 2024-05-27T09:30:33.644 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/124/10' -> 'copyofsnap1/test1/124/10' 2024-05-27T09:30:33.644 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/124/11' -> 'copyofsnap1/test1/124/11' 2024-05-27T09:30:33.644 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/124/12' -> 'copyofsnap1/test1/124/12' 2024-05-27T09:30:33.644 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/124/13' -> 'copyofsnap1/test1/124/13' 2024-05-27T09:30:33.644 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/124/14' -> 'copyofsnap1/test1/124/14' 2024-05-27T09:30:33.644 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/124/15' -> 'copyofsnap1/test1/124/15' 2024-05-27T09:30:33.644 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/124/16' -> 'copyofsnap1/test1/124/16' 2024-05-27T09:30:33.644 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/124/17' -> 'copyofsnap1/test1/124/17' 2024-05-27T09:30:33.644 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/124/18' -> 'copyofsnap1/test1/124/18' 2024-05-27T09:30:33.644 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/124/19' -> 'copyofsnap1/test1/124/19' 2024-05-27T09:30:33.644 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/124/20' -> 'copyofsnap1/test1/124/20' 2024-05-27T09:30:33.644 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/125' -> 'copyofsnap1/test1/125' 2024-05-27T09:30:33.644 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/125/10' -> 'copyofsnap1/test1/125/10' 2024-05-27T09:30:33.644 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/125/11' -> 'copyofsnap1/test1/125/11' 2024-05-27T09:30:33.644 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/125/12' -> 'copyofsnap1/test1/125/12' 2024-05-27T09:30:33.645 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/125/13' -> 'copyofsnap1/test1/125/13' 2024-05-27T09:30:33.645 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/125/14' -> 'copyofsnap1/test1/125/14' 2024-05-27T09:30:33.645 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/125/15' -> 'copyofsnap1/test1/125/15' 2024-05-27T09:30:33.645 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/125/16' -> 'copyofsnap1/test1/125/16' 2024-05-27T09:30:33.645 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/125/17' -> 'copyofsnap1/test1/125/17' 2024-05-27T09:30:33.645 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/125/18' -> 'copyofsnap1/test1/125/18' 2024-05-27T09:30:33.645 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/125/19' -> 'copyofsnap1/test1/125/19' 2024-05-27T09:30:33.645 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/125/20' -> 'copyofsnap1/test1/125/20' 2024-05-27T09:30:33.645 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/126' -> 'copyofsnap1/test1/126' 2024-05-27T09:30:33.645 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/126/10' -> 'copyofsnap1/test1/126/10' 2024-05-27T09:30:33.645 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/126/11' -> 'copyofsnap1/test1/126/11' 2024-05-27T09:30:33.645 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/126/12' -> 'copyofsnap1/test1/126/12' 2024-05-27T09:30:33.645 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/126/13' -> 'copyofsnap1/test1/126/13' 2024-05-27T09:30:33.645 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/126/14' -> 'copyofsnap1/test1/126/14' 2024-05-27T09:30:33.646 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/126/15' -> 'copyofsnap1/test1/126/15' 2024-05-27T09:30:33.646 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/126/16' -> 'copyofsnap1/test1/126/16' 2024-05-27T09:30:33.646 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/126/17' -> 'copyofsnap1/test1/126/17' 2024-05-27T09:30:33.646 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/126/18' -> 'copyofsnap1/test1/126/18' 2024-05-27T09:30:33.646 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/126/19' -> 'copyofsnap1/test1/126/19' 2024-05-27T09:30:34.119 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/126/20' -> 'copyofsnap1/test1/126/20' 2024-05-27T09:30:34.120 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/127' -> 'copyofsnap1/test1/127' 2024-05-27T09:30:34.120 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/127/10' -> 'copyofsnap1/test1/127/10' 2024-05-27T09:30:34.120 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/127/11' -> 'copyofsnap1/test1/127/11' 2024-05-27T09:30:34.120 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/127/12' -> 'copyofsnap1/test1/127/12' 2024-05-27T09:30:34.120 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/127/13' -> 'copyofsnap1/test1/127/13' 2024-05-27T09:30:34.120 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/127/14' -> 'copyofsnap1/test1/127/14' 2024-05-27T09:30:34.120 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/127/15' -> 'copyofsnap1/test1/127/15' 2024-05-27T09:30:34.120 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/127/16' -> 'copyofsnap1/test1/127/16' 2024-05-27T09:30:34.120 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/127/17' -> 'copyofsnap1/test1/127/17' 2024-05-27T09:30:34.120 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/127/18' -> 'copyofsnap1/test1/127/18' 2024-05-27T09:30:34.120 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/127/19' -> 'copyofsnap1/test1/127/19' 2024-05-27T09:30:34.120 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/127/20' -> 'copyofsnap1/test1/127/20' 2024-05-27T09:30:34.121 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/128' -> 'copyofsnap1/test1/128' 2024-05-27T09:30:34.121 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/128/10' -> 'copyofsnap1/test1/128/10' 2024-05-27T09:30:34.121 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/128/11' -> 'copyofsnap1/test1/128/11' 2024-05-27T09:30:34.121 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/128/12' -> 'copyofsnap1/test1/128/12' 2024-05-27T09:30:34.121 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/128/13' -> 'copyofsnap1/test1/128/13' 2024-05-27T09:30:34.121 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/128/14' -> 'copyofsnap1/test1/128/14' 2024-05-27T09:30:34.121 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/128/15' -> 'copyofsnap1/test1/128/15' 2024-05-27T09:30:34.121 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/128/16' -> 'copyofsnap1/test1/128/16' 2024-05-27T09:30:34.121 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/128/17' -> 'copyofsnap1/test1/128/17' 2024-05-27T09:30:34.121 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/128/18' -> 'copyofsnap1/test1/128/18' 2024-05-27T09:30:34.121 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/128/19' -> 'copyofsnap1/test1/128/19' 2024-05-27T09:30:34.121 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/128/20' -> 'copyofsnap1/test1/128/20' 2024-05-27T09:30:34.121 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/129' -> 'copyofsnap1/test1/129' 2024-05-27T09:30:34.121 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/129/10' -> 'copyofsnap1/test1/129/10' 2024-05-27T09:30:34.122 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/129/11' -> 'copyofsnap1/test1/129/11' 2024-05-27T09:30:34.122 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/129/12' -> 'copyofsnap1/test1/129/12' 2024-05-27T09:30:34.122 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/129/13' -> 'copyofsnap1/test1/129/13' 2024-05-27T09:30:34.122 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/129/14' -> 'copyofsnap1/test1/129/14' 2024-05-27T09:30:34.122 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/129/15' -> 'copyofsnap1/test1/129/15' 2024-05-27T09:30:34.122 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/129/16' -> 'copyofsnap1/test1/129/16' 2024-05-27T09:30:34.122 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/129/17' -> 'copyofsnap1/test1/129/17' 2024-05-27T09:30:34.122 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/129/18' -> 'copyofsnap1/test1/129/18' 2024-05-27T09:30:34.122 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/129/19' -> 'copyofsnap1/test1/129/19' 2024-05-27T09:30:34.122 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/129/20' -> 'copyofsnap1/test1/129/20' 2024-05-27T09:30:34.122 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/130' -> 'copyofsnap1/test1/130' 2024-05-27T09:30:34.122 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/130/10' -> 'copyofsnap1/test1/130/10' 2024-05-27T09:30:34.123 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/130/11' -> 'copyofsnap1/test1/130/11' 2024-05-27T09:30:34.123 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/130/12' -> 'copyofsnap1/test1/130/12' 2024-05-27T09:30:34.123 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/130/13' -> 'copyofsnap1/test1/130/13' 2024-05-27T09:30:34.123 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/130/14' -> 'copyofsnap1/test1/130/14' 2024-05-27T09:30:34.123 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/130/15' -> 'copyofsnap1/test1/130/15' 2024-05-27T09:30:34.123 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/130/16' -> 'copyofsnap1/test1/130/16' 2024-05-27T09:30:34.123 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/130/17' -> 'copyofsnap1/test1/130/17' 2024-05-27T09:30:34.123 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/130/18' -> 'copyofsnap1/test1/130/18' 2024-05-27T09:30:34.123 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/130/19' -> 'copyofsnap1/test1/130/19' 2024-05-27T09:30:34.123 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/130/20' -> 'copyofsnap1/test1/130/20' 2024-05-27T09:30:34.123 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/131' -> 'copyofsnap1/test1/131' 2024-05-27T09:30:34.123 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/131/10' -> 'copyofsnap1/test1/131/10' 2024-05-27T09:30:34.123 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/131/11' -> 'copyofsnap1/test1/131/11' 2024-05-27T09:30:34.123 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/131/12' -> 'copyofsnap1/test1/131/12' 2024-05-27T09:30:34.124 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/131/13' -> 'copyofsnap1/test1/131/13' 2024-05-27T09:30:34.124 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/131/14' -> 'copyofsnap1/test1/131/14' 2024-05-27T09:30:34.124 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/131/15' -> 'copyofsnap1/test1/131/15' 2024-05-27T09:30:34.124 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/131/16' -> 'copyofsnap1/test1/131/16' 2024-05-27T09:30:34.124 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/131/17' -> 'copyofsnap1/test1/131/17' 2024-05-27T09:30:34.124 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/131/18' -> 'copyofsnap1/test1/131/18' 2024-05-27T09:30:34.124 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/131/19' -> 'copyofsnap1/test1/131/19' 2024-05-27T09:30:34.124 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/131/20' -> 'copyofsnap1/test1/131/20' 2024-05-27T09:30:34.124 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/132' -> 'copyofsnap1/test1/132' 2024-05-27T09:30:34.124 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/132/10' -> 'copyofsnap1/test1/132/10' 2024-05-27T09:30:34.124 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/132/11' -> 'copyofsnap1/test1/132/11' 2024-05-27T09:30:34.124 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/132/12' -> 'copyofsnap1/test1/132/12' 2024-05-27T09:30:34.124 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/132/13' -> 'copyofsnap1/test1/132/13' 2024-05-27T09:30:34.124 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/132/14' -> 'copyofsnap1/test1/132/14' 2024-05-27T09:30:34.125 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/132/15' -> 'copyofsnap1/test1/132/15' 2024-05-27T09:30:34.125 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/132/16' -> 'copyofsnap1/test1/132/16' 2024-05-27T09:30:34.125 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/132/17' -> 'copyofsnap1/test1/132/17' 2024-05-27T09:30:34.125 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/132/18' -> 'copyofsnap1/test1/132/18' 2024-05-27T09:30:34.125 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/132/19' -> 'copyofsnap1/test1/132/19' 2024-05-27T09:30:34.125 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/132/20' -> 'copyofsnap1/test1/132/20' 2024-05-27T09:30:34.125 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/133' -> 'copyofsnap1/test1/133' 2024-05-27T09:30:34.125 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/133/10' -> 'copyofsnap1/test1/133/10' 2024-05-27T09:30:34.125 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/133/11' -> 'copyofsnap1/test1/133/11' 2024-05-27T09:30:34.125 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/133/12' -> 'copyofsnap1/test1/133/12' 2024-05-27T09:30:34.125 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/133/13' -> 'copyofsnap1/test1/133/13' 2024-05-27T09:30:34.125 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/133/14' -> 'copyofsnap1/test1/133/14' 2024-05-27T09:30:34.125 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/133/15' -> 'copyofsnap1/test1/133/15' 2024-05-27T09:30:34.125 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/133/16' -> 'copyofsnap1/test1/133/16' 2024-05-27T09:30:34.525 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/133/17' -> 'copyofsnap1/test1/133/17' 2024-05-27T09:30:34.525 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/133/18' -> 'copyofsnap1/test1/133/18' 2024-05-27T09:30:34.525 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/133/19' -> 'copyofsnap1/test1/133/19' 2024-05-27T09:30:34.525 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/133/20' -> 'copyofsnap1/test1/133/20' 2024-05-27T09:30:34.526 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/134' -> 'copyofsnap1/test1/134' 2024-05-27T09:30:34.526 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/134/10' -> 'copyofsnap1/test1/134/10' 2024-05-27T09:30:34.526 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/134/11' -> 'copyofsnap1/test1/134/11' 2024-05-27T09:30:34.526 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/134/12' -> 'copyofsnap1/test1/134/12' 2024-05-27T09:30:34.526 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/134/13' -> 'copyofsnap1/test1/134/13' 2024-05-27T09:30:34.526 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/134/14' -> 'copyofsnap1/test1/134/14' 2024-05-27T09:30:34.526 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/134/15' -> 'copyofsnap1/test1/134/15' 2024-05-27T09:30:34.526 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/134/16' -> 'copyofsnap1/test1/134/16' 2024-05-27T09:30:34.526 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/134/17' -> 'copyofsnap1/test1/134/17' 2024-05-27T09:30:34.526 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/134/18' -> 'copyofsnap1/test1/134/18' 2024-05-27T09:30:34.526 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/134/19' -> 'copyofsnap1/test1/134/19' 2024-05-27T09:30:34.526 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/134/20' -> 'copyofsnap1/test1/134/20' 2024-05-27T09:30:34.527 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/135' -> 'copyofsnap1/test1/135' 2024-05-27T09:30:34.527 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/135/10' -> 'copyofsnap1/test1/135/10' 2024-05-27T09:30:34.527 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/135/11' -> 'copyofsnap1/test1/135/11' 2024-05-27T09:30:34.527 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/135/12' -> 'copyofsnap1/test1/135/12' 2024-05-27T09:30:34.527 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/135/13' -> 'copyofsnap1/test1/135/13' 2024-05-27T09:30:34.527 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/135/14' -> 'copyofsnap1/test1/135/14' 2024-05-27T09:30:34.527 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/135/15' -> 'copyofsnap1/test1/135/15' 2024-05-27T09:30:34.527 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/135/16' -> 'copyofsnap1/test1/135/16' 2024-05-27T09:30:34.527 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/135/17' -> 'copyofsnap1/test1/135/17' 2024-05-27T09:30:34.527 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/135/18' -> 'copyofsnap1/test1/135/18' 2024-05-27T09:30:34.527 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/135/19' -> 'copyofsnap1/test1/135/19' 2024-05-27T09:30:34.527 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/135/20' -> 'copyofsnap1/test1/135/20' 2024-05-27T09:30:34.527 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/136' -> 'copyofsnap1/test1/136' 2024-05-27T09:30:34.528 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/136/10' -> 'copyofsnap1/test1/136/10' 2024-05-27T09:30:34.528 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/136/11' -> 'copyofsnap1/test1/136/11' 2024-05-27T09:30:34.528 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/136/12' -> 'copyofsnap1/test1/136/12' 2024-05-27T09:30:34.528 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/136/13' -> 'copyofsnap1/test1/136/13' 2024-05-27T09:30:34.528 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/136/14' -> 'copyofsnap1/test1/136/14' 2024-05-27T09:30:34.528 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/136/15' -> 'copyofsnap1/test1/136/15' 2024-05-27T09:30:34.528 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/136/16' -> 'copyofsnap1/test1/136/16' 2024-05-27T09:30:34.528 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/136/17' -> 'copyofsnap1/test1/136/17' 2024-05-27T09:30:34.528 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/136/18' -> 'copyofsnap1/test1/136/18' 2024-05-27T09:30:34.528 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/136/19' -> 'copyofsnap1/test1/136/19' 2024-05-27T09:30:34.528 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/136/20' -> 'copyofsnap1/test1/136/20' 2024-05-27T09:30:34.529 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/137' -> 'copyofsnap1/test1/137' 2024-05-27T09:30:34.529 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/137/10' -> 'copyofsnap1/test1/137/10' 2024-05-27T09:30:34.529 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/137/11' -> 'copyofsnap1/test1/137/11' 2024-05-27T09:30:34.529 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/137/12' -> 'copyofsnap1/test1/137/12' 2024-05-27T09:30:34.529 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/137/13' -> 'copyofsnap1/test1/137/13' 2024-05-27T09:30:34.529 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/137/14' -> 'copyofsnap1/test1/137/14' 2024-05-27T09:30:34.529 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/137/15' -> 'copyofsnap1/test1/137/15' 2024-05-27T09:30:34.529 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/137/16' -> 'copyofsnap1/test1/137/16' 2024-05-27T09:30:34.529 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/137/17' -> 'copyofsnap1/test1/137/17' 2024-05-27T09:30:34.529 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/137/18' -> 'copyofsnap1/test1/137/18' 2024-05-27T09:30:34.529 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/137/19' -> 'copyofsnap1/test1/137/19' 2024-05-27T09:30:34.530 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/137/20' -> 'copyofsnap1/test1/137/20' 2024-05-27T09:30:34.530 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/138' -> 'copyofsnap1/test1/138' 2024-05-27T09:30:34.530 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/138/10' -> 'copyofsnap1/test1/138/10' 2024-05-27T09:30:34.530 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/138/11' -> 'copyofsnap1/test1/138/11' 2024-05-27T09:30:34.530 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/138/12' -> 'copyofsnap1/test1/138/12' 2024-05-27T09:30:34.530 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/138/13' -> 'copyofsnap1/test1/138/13' 2024-05-27T09:30:34.530 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/138/14' -> 'copyofsnap1/test1/138/14' 2024-05-27T09:30:34.530 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/138/15' -> 'copyofsnap1/test1/138/15' 2024-05-27T09:30:34.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/138/16' -> 'copyofsnap1/test1/138/16' 2024-05-27T09:30:34.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/138/17' -> 'copyofsnap1/test1/138/17' 2024-05-27T09:30:34.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/138/18' -> 'copyofsnap1/test1/138/18' 2024-05-27T09:30:34.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/138/19' -> 'copyofsnap1/test1/138/19' 2024-05-27T09:30:34.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/138/20' -> 'copyofsnap1/test1/138/20' 2024-05-27T09:30:34.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/139' -> 'copyofsnap1/test1/139' 2024-05-27T09:30:34.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/139/10' -> 'copyofsnap1/test1/139/10' 2024-05-27T09:30:34.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/139/11' -> 'copyofsnap1/test1/139/11' 2024-05-27T09:30:34.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/139/12' -> 'copyofsnap1/test1/139/12' 2024-05-27T09:30:34.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/139/13' -> 'copyofsnap1/test1/139/13' 2024-05-27T09:30:34.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/139/14' -> 'copyofsnap1/test1/139/14' 2024-05-27T09:30:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/139/15' -> 'copyofsnap1/test1/139/15' 2024-05-27T09:30:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/139/16' -> 'copyofsnap1/test1/139/16' 2024-05-27T09:30:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/139/17' -> 'copyofsnap1/test1/139/17' 2024-05-27T09:30:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/139/18' -> 'copyofsnap1/test1/139/18' 2024-05-27T09:30:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/139/19' -> 'copyofsnap1/test1/139/19' 2024-05-27T09:30:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/139/20' -> 'copyofsnap1/test1/139/20' 2024-05-27T09:30:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/140' -> 'copyofsnap1/test1/140' 2024-05-27T09:30:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/140/10' -> 'copyofsnap1/test1/140/10' 2024-05-27T09:30:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/140/11' -> 'copyofsnap1/test1/140/11' 2024-05-27T09:30:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/140/12' -> 'copyofsnap1/test1/140/12' 2024-05-27T09:30:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/140/13' -> 'copyofsnap1/test1/140/13' 2024-05-27T09:30:34.905 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/140/14' -> 'copyofsnap1/test1/140/14' 2024-05-27T09:30:34.905 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/140/15' -> 'copyofsnap1/test1/140/15' 2024-05-27T09:30:34.906 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/140/16' -> 'copyofsnap1/test1/140/16' 2024-05-27T09:30:34.906 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/140/17' -> 'copyofsnap1/test1/140/17' 2024-05-27T09:30:34.906 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/140/18' -> 'copyofsnap1/test1/140/18' 2024-05-27T09:30:34.906 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/140/19' -> 'copyofsnap1/test1/140/19' 2024-05-27T09:30:34.906 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/140/20' -> 'copyofsnap1/test1/140/20' 2024-05-27T09:30:34.906 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/141' -> 'copyofsnap1/test1/141' 2024-05-27T09:30:34.906 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/141/10' -> 'copyofsnap1/test1/141/10' 2024-05-27T09:30:34.906 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/141/11' -> 'copyofsnap1/test1/141/11' 2024-05-27T09:30:34.906 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/141/12' -> 'copyofsnap1/test1/141/12' 2024-05-27T09:30:34.906 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/141/13' -> 'copyofsnap1/test1/141/13' 2024-05-27T09:30:34.906 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/141/14' -> 'copyofsnap1/test1/141/14' 2024-05-27T09:30:34.906 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/141/15' -> 'copyofsnap1/test1/141/15' 2024-05-27T09:30:34.906 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/141/16' -> 'copyofsnap1/test1/141/16' 2024-05-27T09:30:34.907 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/141/17' -> 'copyofsnap1/test1/141/17' 2024-05-27T09:30:34.907 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/141/18' -> 'copyofsnap1/test1/141/18' 2024-05-27T09:30:34.907 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/141/19' -> 'copyofsnap1/test1/141/19' 2024-05-27T09:30:34.907 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/141/20' -> 'copyofsnap1/test1/141/20' 2024-05-27T09:30:34.907 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/142' -> 'copyofsnap1/test1/142' 2024-05-27T09:30:34.907 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/142/10' -> 'copyofsnap1/test1/142/10' 2024-05-27T09:30:34.907 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/142/11' -> 'copyofsnap1/test1/142/11' 2024-05-27T09:30:34.907 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/142/12' -> 'copyofsnap1/test1/142/12' 2024-05-27T09:30:34.907 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/142/13' -> 'copyofsnap1/test1/142/13' 2024-05-27T09:30:34.907 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/142/14' -> 'copyofsnap1/test1/142/14' 2024-05-27T09:30:34.907 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/142/15' -> 'copyofsnap1/test1/142/15' 2024-05-27T09:30:34.907 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/142/16' -> 'copyofsnap1/test1/142/16' 2024-05-27T09:30:34.907 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/142/17' -> 'copyofsnap1/test1/142/17' 2024-05-27T09:30:34.908 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/142/18' -> 'copyofsnap1/test1/142/18' 2024-05-27T09:30:34.908 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/142/19' -> 'copyofsnap1/test1/142/19' 2024-05-27T09:30:34.908 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/142/20' -> 'copyofsnap1/test1/142/20' 2024-05-27T09:30:34.908 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/143' -> 'copyofsnap1/test1/143' 2024-05-27T09:30:34.908 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/143/10' -> 'copyofsnap1/test1/143/10' 2024-05-27T09:30:34.908 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/143/11' -> 'copyofsnap1/test1/143/11' 2024-05-27T09:30:34.908 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/143/12' -> 'copyofsnap1/test1/143/12' 2024-05-27T09:30:34.908 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/143/13' -> 'copyofsnap1/test1/143/13' 2024-05-27T09:30:34.908 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/143/14' -> 'copyofsnap1/test1/143/14' 2024-05-27T09:30:34.908 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/143/15' -> 'copyofsnap1/test1/143/15' 2024-05-27T09:30:34.908 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/143/16' -> 'copyofsnap1/test1/143/16' 2024-05-27T09:30:34.908 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/143/17' -> 'copyofsnap1/test1/143/17' 2024-05-27T09:30:34.908 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/143/18' -> 'copyofsnap1/test1/143/18' 2024-05-27T09:30:34.908 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/143/19' -> 'copyofsnap1/test1/143/19' 2024-05-27T09:30:34.909 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/143/20' -> 'copyofsnap1/test1/143/20' 2024-05-27T09:30:34.909 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/144' -> 'copyofsnap1/test1/144' 2024-05-27T09:30:34.909 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/144/10' -> 'copyofsnap1/test1/144/10' 2024-05-27T09:30:34.909 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/144/11' -> 'copyofsnap1/test1/144/11' 2024-05-27T09:30:34.909 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/144/12' -> 'copyofsnap1/test1/144/12' 2024-05-27T09:30:34.909 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/144/13' -> 'copyofsnap1/test1/144/13' 2024-05-27T09:30:34.909 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/144/14' -> 'copyofsnap1/test1/144/14' 2024-05-27T09:30:34.909 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/144/15' -> 'copyofsnap1/test1/144/15' 2024-05-27T09:30:34.909 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/144/16' -> 'copyofsnap1/test1/144/16' 2024-05-27T09:30:34.909 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/144/17' -> 'copyofsnap1/test1/144/17' 2024-05-27T09:30:34.909 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/144/18' -> 'copyofsnap1/test1/144/18' 2024-05-27T09:30:34.909 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/144/19' -> 'copyofsnap1/test1/144/19' 2024-05-27T09:30:34.909 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/144/20' -> 'copyofsnap1/test1/144/20' 2024-05-27T09:30:34.909 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/145' -> 'copyofsnap1/test1/145' 2024-05-27T09:30:34.910 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/145/10' -> 'copyofsnap1/test1/145/10' 2024-05-27T09:30:34.910 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/145/11' -> 'copyofsnap1/test1/145/11' 2024-05-27T09:30:34.910 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/145/12' -> 'copyofsnap1/test1/145/12' 2024-05-27T09:30:34.910 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/145/13' -> 'copyofsnap1/test1/145/13' 2024-05-27T09:30:34.910 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/145/14' -> 'copyofsnap1/test1/145/14' 2024-05-27T09:30:34.910 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/145/15' -> 'copyofsnap1/test1/145/15' 2024-05-27T09:30:34.910 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/145/16' -> 'copyofsnap1/test1/145/16' 2024-05-27T09:30:34.910 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/145/17' -> 'copyofsnap1/test1/145/17' 2024-05-27T09:30:34.910 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/145/18' -> 'copyofsnap1/test1/145/18' 2024-05-27T09:30:34.910 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/145/19' -> 'copyofsnap1/test1/145/19' 2024-05-27T09:30:34.910 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/145/20' -> 'copyofsnap1/test1/145/20' 2024-05-27T09:30:34.910 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/146' -> 'copyofsnap1/test1/146' 2024-05-27T09:30:34.910 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/146/10' -> 'copyofsnap1/test1/146/10' 2024-05-27T09:30:34.911 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/146/11' -> 'copyofsnap1/test1/146/11' 2024-05-27T09:30:34.911 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/146/12' -> 'copyofsnap1/test1/146/12' 2024-05-27T09:30:34.911 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/146/13' -> 'copyofsnap1/test1/146/13' 2024-05-27T09:30:34.911 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/146/14' -> 'copyofsnap1/test1/146/14' 2024-05-27T09:30:34.911 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/146/15' -> 'copyofsnap1/test1/146/15' 2024-05-27T09:30:34.911 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/146/16' -> 'copyofsnap1/test1/146/16' 2024-05-27T09:30:34.911 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/146/17' -> 'copyofsnap1/test1/146/17' 2024-05-27T09:30:34.911 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/146/18' -> 'copyofsnap1/test1/146/18' 2024-05-27T09:30:34.911 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/146/19' -> 'copyofsnap1/test1/146/19' 2024-05-27T09:30:34.911 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/146/20' -> 'copyofsnap1/test1/146/20' 2024-05-27T09:30:34.911 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/147' -> 'copyofsnap1/test1/147' 2024-05-27T09:30:34.911 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/147/10' -> 'copyofsnap1/test1/147/10' 2024-05-27T09:30:34.911 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/147/11' -> 'copyofsnap1/test1/147/11' 2024-05-27T09:30:35.295 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/147/12' -> 'copyofsnap1/test1/147/12' 2024-05-27T09:30:35.295 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/147/13' -> 'copyofsnap1/test1/147/13' 2024-05-27T09:30:35.295 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/147/14' -> 'copyofsnap1/test1/147/14' 2024-05-27T09:30:35.295 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/147/15' -> 'copyofsnap1/test1/147/15' 2024-05-27T09:30:35.296 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/147/16' -> 'copyofsnap1/test1/147/16' 2024-05-27T09:30:35.296 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/147/17' -> 'copyofsnap1/test1/147/17' 2024-05-27T09:30:35.296 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/147/18' -> 'copyofsnap1/test1/147/18' 2024-05-27T09:30:35.296 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/147/19' -> 'copyofsnap1/test1/147/19' 2024-05-27T09:30:35.296 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/147/20' -> 'copyofsnap1/test1/147/20' 2024-05-27T09:30:35.296 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/148' -> 'copyofsnap1/test1/148' 2024-05-27T09:30:35.296 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/148/10' -> 'copyofsnap1/test1/148/10' 2024-05-27T09:30:35.296 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/148/11' -> 'copyofsnap1/test1/148/11' 2024-05-27T09:30:35.296 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/148/12' -> 'copyofsnap1/test1/148/12' 2024-05-27T09:30:35.296 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/148/13' -> 'copyofsnap1/test1/148/13' 2024-05-27T09:30:35.296 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/148/14' -> 'copyofsnap1/test1/148/14' 2024-05-27T09:30:35.296 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/148/15' -> 'copyofsnap1/test1/148/15' 2024-05-27T09:30:35.296 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/148/16' -> 'copyofsnap1/test1/148/16' 2024-05-27T09:30:35.296 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/148/17' -> 'copyofsnap1/test1/148/17' 2024-05-27T09:30:35.297 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/148/18' -> 'copyofsnap1/test1/148/18' 2024-05-27T09:30:35.297 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/148/19' -> 'copyofsnap1/test1/148/19' 2024-05-27T09:30:35.297 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/148/20' -> 'copyofsnap1/test1/148/20' 2024-05-27T09:30:35.297 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/149' -> 'copyofsnap1/test1/149' 2024-05-27T09:30:35.297 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/149/10' -> 'copyofsnap1/test1/149/10' 2024-05-27T09:30:35.297 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/149/11' -> 'copyofsnap1/test1/149/11' 2024-05-27T09:30:35.297 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/149/12' -> 'copyofsnap1/test1/149/12' 2024-05-27T09:30:35.297 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/149/13' -> 'copyofsnap1/test1/149/13' 2024-05-27T09:30:35.297 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/149/14' -> 'copyofsnap1/test1/149/14' 2024-05-27T09:30:35.297 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/149/15' -> 'copyofsnap1/test1/149/15' 2024-05-27T09:30:35.297 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/149/16' -> 'copyofsnap1/test1/149/16' 2024-05-27T09:30:35.297 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/149/17' -> 'copyofsnap1/test1/149/17' 2024-05-27T09:30:35.297 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/149/18' -> 'copyofsnap1/test1/149/18' 2024-05-27T09:30:35.297 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/149/19' -> 'copyofsnap1/test1/149/19' 2024-05-27T09:30:35.297 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/149/20' -> 'copyofsnap1/test1/149/20' 2024-05-27T09:30:35.298 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/150' -> 'copyofsnap1/test1/150' 2024-05-27T09:30:35.298 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/150/10' -> 'copyofsnap1/test1/150/10' 2024-05-27T09:30:35.298 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/150/11' -> 'copyofsnap1/test1/150/11' 2024-05-27T09:30:35.298 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/150/12' -> 'copyofsnap1/test1/150/12' 2024-05-27T09:30:35.298 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/150/13' -> 'copyofsnap1/test1/150/13' 2024-05-27T09:30:35.298 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/150/14' -> 'copyofsnap1/test1/150/14' 2024-05-27T09:30:35.298 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/150/15' -> 'copyofsnap1/test1/150/15' 2024-05-27T09:30:35.298 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/150/16' -> 'copyofsnap1/test1/150/16' 2024-05-27T09:30:35.298 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/150/17' -> 'copyofsnap1/test1/150/17' 2024-05-27T09:30:35.298 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/150/18' -> 'copyofsnap1/test1/150/18' 2024-05-27T09:30:35.298 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/150/19' -> 'copyofsnap1/test1/150/19' 2024-05-27T09:30:35.298 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/150/20' -> 'copyofsnap1/test1/150/20' 2024-05-27T09:30:35.298 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/151' -> 'copyofsnap1/test1/151' 2024-05-27T09:30:35.298 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/151/10' -> 'copyofsnap1/test1/151/10' 2024-05-27T09:30:35.299 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/151/11' -> 'copyofsnap1/test1/151/11' 2024-05-27T09:30:35.299 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/151/12' -> 'copyofsnap1/test1/151/12' 2024-05-27T09:30:35.299 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/151/13' -> 'copyofsnap1/test1/151/13' 2024-05-27T09:30:35.299 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/151/14' -> 'copyofsnap1/test1/151/14' 2024-05-27T09:30:35.299 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/151/15' -> 'copyofsnap1/test1/151/15' 2024-05-27T09:30:35.299 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/151/16' -> 'copyofsnap1/test1/151/16' 2024-05-27T09:30:35.299 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/151/17' -> 'copyofsnap1/test1/151/17' 2024-05-27T09:30:35.299 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/151/18' -> 'copyofsnap1/test1/151/18' 2024-05-27T09:30:35.299 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/151/19' -> 'copyofsnap1/test1/151/19' 2024-05-27T09:30:35.299 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/151/20' -> 'copyofsnap1/test1/151/20' 2024-05-27T09:30:35.299 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/152' -> 'copyofsnap1/test1/152' 2024-05-27T09:30:35.299 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/152/10' -> 'copyofsnap1/test1/152/10' 2024-05-27T09:30:35.299 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/152/11' -> 'copyofsnap1/test1/152/11' 2024-05-27T09:30:35.299 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/152/12' -> 'copyofsnap1/test1/152/12' 2024-05-27T09:30:35.299 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/152/13' -> 'copyofsnap1/test1/152/13' 2024-05-27T09:30:35.300 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/152/14' -> 'copyofsnap1/test1/152/14' 2024-05-27T09:30:35.300 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/152/15' -> 'copyofsnap1/test1/152/15' 2024-05-27T09:30:35.300 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/152/16' -> 'copyofsnap1/test1/152/16' 2024-05-27T09:30:35.300 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/152/17' -> 'copyofsnap1/test1/152/17' 2024-05-27T09:30:35.300 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/152/18' -> 'copyofsnap1/test1/152/18' 2024-05-27T09:30:35.300 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/152/19' -> 'copyofsnap1/test1/152/19' 2024-05-27T09:30:35.300 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/152/20' -> 'copyofsnap1/test1/152/20' 2024-05-27T09:30:35.300 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/153' -> 'copyofsnap1/test1/153' 2024-05-27T09:30:35.300 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/153/10' -> 'copyofsnap1/test1/153/10' 2024-05-27T09:30:35.300 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/153/11' -> 'copyofsnap1/test1/153/11' 2024-05-27T09:30:35.300 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/153/12' -> 'copyofsnap1/test1/153/12' 2024-05-27T09:30:35.300 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/153/13' -> 'copyofsnap1/test1/153/13' 2024-05-27T09:30:35.300 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/153/14' -> 'copyofsnap1/test1/153/14' 2024-05-27T09:30:35.300 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/153/15' -> 'copyofsnap1/test1/153/15' 2024-05-27T09:30:35.300 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/153/16' -> 'copyofsnap1/test1/153/16' 2024-05-27T09:30:35.301 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/153/17' -> 'copyofsnap1/test1/153/17' 2024-05-27T09:30:35.301 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/153/18' -> 'copyofsnap1/test1/153/18' 2024-05-27T09:30:35.301 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/153/19' -> 'copyofsnap1/test1/153/19' 2024-05-27T09:30:35.301 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/153/20' -> 'copyofsnap1/test1/153/20' 2024-05-27T09:30:35.751 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/154' -> 'copyofsnap1/test1/154' 2024-05-27T09:30:35.751 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/154/10' -> 'copyofsnap1/test1/154/10' 2024-05-27T09:30:35.751 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/154/11' -> 'copyofsnap1/test1/154/11' 2024-05-27T09:30:35.751 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/154/12' -> 'copyofsnap1/test1/154/12' 2024-05-27T09:30:35.751 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/154/13' -> 'copyofsnap1/test1/154/13' 2024-05-27T09:30:35.751 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/154/14' -> 'copyofsnap1/test1/154/14' 2024-05-27T09:30:35.751 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/154/15' -> 'copyofsnap1/test1/154/15' 2024-05-27T09:30:35.752 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/154/16' -> 'copyofsnap1/test1/154/16' 2024-05-27T09:30:35.752 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/154/17' -> 'copyofsnap1/test1/154/17' 2024-05-27T09:30:35.752 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/154/18' -> 'copyofsnap1/test1/154/18' 2024-05-27T09:30:35.752 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/154/19' -> 'copyofsnap1/test1/154/19' 2024-05-27T09:30:35.752 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/154/20' -> 'copyofsnap1/test1/154/20' 2024-05-27T09:30:35.752 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/155' -> 'copyofsnap1/test1/155' 2024-05-27T09:30:35.752 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/155/10' -> 'copyofsnap1/test1/155/10' 2024-05-27T09:30:35.752 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/155/11' -> 'copyofsnap1/test1/155/11' 2024-05-27T09:30:35.752 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/155/12' -> 'copyofsnap1/test1/155/12' 2024-05-27T09:30:35.752 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/155/13' -> 'copyofsnap1/test1/155/13' 2024-05-27T09:30:35.752 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/155/14' -> 'copyofsnap1/test1/155/14' 2024-05-27T09:30:35.752 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/155/15' -> 'copyofsnap1/test1/155/15' 2024-05-27T09:30:35.752 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/155/16' -> 'copyofsnap1/test1/155/16' 2024-05-27T09:30:35.752 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/155/17' -> 'copyofsnap1/test1/155/17' 2024-05-27T09:30:35.753 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/155/18' -> 'copyofsnap1/test1/155/18' 2024-05-27T09:30:35.753 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/155/19' -> 'copyofsnap1/test1/155/19' 2024-05-27T09:30:35.753 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/155/20' -> 'copyofsnap1/test1/155/20' 2024-05-27T09:30:35.753 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/156' -> 'copyofsnap1/test1/156' 2024-05-27T09:30:35.753 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/156/10' -> 'copyofsnap1/test1/156/10' 2024-05-27T09:30:35.753 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/156/11' -> 'copyofsnap1/test1/156/11' 2024-05-27T09:30:35.753 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/156/12' -> 'copyofsnap1/test1/156/12' 2024-05-27T09:30:35.753 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/156/13' -> 'copyofsnap1/test1/156/13' 2024-05-27T09:30:35.753 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/156/14' -> 'copyofsnap1/test1/156/14' 2024-05-27T09:30:35.753 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/156/15' -> 'copyofsnap1/test1/156/15' 2024-05-27T09:30:35.753 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/156/16' -> 'copyofsnap1/test1/156/16' 2024-05-27T09:30:35.753 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/156/17' -> 'copyofsnap1/test1/156/17' 2024-05-27T09:30:35.753 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/156/18' -> 'copyofsnap1/test1/156/18' 2024-05-27T09:30:35.753 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/156/19' -> 'copyofsnap1/test1/156/19' 2024-05-27T09:30:35.754 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/156/20' -> 'copyofsnap1/test1/156/20' 2024-05-27T09:30:35.754 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/157' -> 'copyofsnap1/test1/157' 2024-05-27T09:30:35.754 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/157/10' -> 'copyofsnap1/test1/157/10' 2024-05-27T09:30:35.754 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/157/11' -> 'copyofsnap1/test1/157/11' 2024-05-27T09:30:35.754 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/157/12' -> 'copyofsnap1/test1/157/12' 2024-05-27T09:30:35.754 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/157/13' -> 'copyofsnap1/test1/157/13' 2024-05-27T09:30:35.754 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/157/14' -> 'copyofsnap1/test1/157/14' 2024-05-27T09:30:35.754 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/157/15' -> 'copyofsnap1/test1/157/15' 2024-05-27T09:30:35.754 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/157/16' -> 'copyofsnap1/test1/157/16' 2024-05-27T09:30:35.754 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/157/17' -> 'copyofsnap1/test1/157/17' 2024-05-27T09:30:35.754 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/157/18' -> 'copyofsnap1/test1/157/18' 2024-05-27T09:30:35.754 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/157/19' -> 'copyofsnap1/test1/157/19' 2024-05-27T09:30:35.754 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/157/20' -> 'copyofsnap1/test1/157/20' 2024-05-27T09:30:35.754 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/158' -> 'copyofsnap1/test1/158' 2024-05-27T09:30:35.755 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/158/10' -> 'copyofsnap1/test1/158/10' 2024-05-27T09:30:35.755 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/158/11' -> 'copyofsnap1/test1/158/11' 2024-05-27T09:30:35.755 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/158/12' -> 'copyofsnap1/test1/158/12' 2024-05-27T09:30:35.755 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/158/13' -> 'copyofsnap1/test1/158/13' 2024-05-27T09:30:35.755 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/158/14' -> 'copyofsnap1/test1/158/14' 2024-05-27T09:30:35.755 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/158/15' -> 'copyofsnap1/test1/158/15' 2024-05-27T09:30:35.755 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/158/16' -> 'copyofsnap1/test1/158/16' 2024-05-27T09:30:35.755 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/158/17' -> 'copyofsnap1/test1/158/17' 2024-05-27T09:30:35.755 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/158/18' -> 'copyofsnap1/test1/158/18' 2024-05-27T09:30:35.755 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/158/19' -> 'copyofsnap1/test1/158/19' 2024-05-27T09:30:35.755 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/158/20' -> 'copyofsnap1/test1/158/20' 2024-05-27T09:30:35.755 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/159' -> 'copyofsnap1/test1/159' 2024-05-27T09:30:35.755 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/159/10' -> 'copyofsnap1/test1/159/10' 2024-05-27T09:30:35.755 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/159/11' -> 'copyofsnap1/test1/159/11' 2024-05-27T09:30:35.755 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/159/12' -> 'copyofsnap1/test1/159/12' 2024-05-27T09:30:35.756 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/159/13' -> 'copyofsnap1/test1/159/13' 2024-05-27T09:30:35.756 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/159/14' -> 'copyofsnap1/test1/159/14' 2024-05-27T09:30:35.756 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/159/15' -> 'copyofsnap1/test1/159/15' 2024-05-27T09:30:35.756 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/159/16' -> 'copyofsnap1/test1/159/16' 2024-05-27T09:30:35.756 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/159/17' -> 'copyofsnap1/test1/159/17' 2024-05-27T09:30:35.756 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/159/18' -> 'copyofsnap1/test1/159/18' 2024-05-27T09:30:35.756 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/159/19' -> 'copyofsnap1/test1/159/19' 2024-05-27T09:30:35.756 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/159/20' -> 'copyofsnap1/test1/159/20' 2024-05-27T09:30:35.756 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/160' -> 'copyofsnap1/test1/160' 2024-05-27T09:30:35.756 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/160/10' -> 'copyofsnap1/test1/160/10' 2024-05-27T09:30:35.756 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/160/11' -> 'copyofsnap1/test1/160/11' 2024-05-27T09:30:35.756 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/160/12' -> 'copyofsnap1/test1/160/12' 2024-05-27T09:30:35.756 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/160/13' -> 'copyofsnap1/test1/160/13' 2024-05-27T09:30:35.757 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/160/14' -> 'copyofsnap1/test1/160/14' 2024-05-27T09:30:35.757 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/160/15' -> 'copyofsnap1/test1/160/15' 2024-05-27T09:30:35.757 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/160/16' -> 'copyofsnap1/test1/160/16' 2024-05-27T09:30:35.757 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/160/17' -> 'copyofsnap1/test1/160/17' 2024-05-27T09:30:36.141 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/160/18' -> 'copyofsnap1/test1/160/18' 2024-05-27T09:30:36.141 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/160/19' -> 'copyofsnap1/test1/160/19' 2024-05-27T09:30:36.141 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/160/20' -> 'copyofsnap1/test1/160/20' 2024-05-27T09:30:36.141 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/161' -> 'copyofsnap1/test1/161' 2024-05-27T09:30:36.142 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/161/10' -> 'copyofsnap1/test1/161/10' 2024-05-27T09:30:36.142 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/161/11' -> 'copyofsnap1/test1/161/11' 2024-05-27T09:30:36.142 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/161/12' -> 'copyofsnap1/test1/161/12' 2024-05-27T09:30:36.142 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/161/13' -> 'copyofsnap1/test1/161/13' 2024-05-27T09:30:36.142 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/161/14' -> 'copyofsnap1/test1/161/14' 2024-05-27T09:30:36.142 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/161/15' -> 'copyofsnap1/test1/161/15' 2024-05-27T09:30:36.142 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/161/16' -> 'copyofsnap1/test1/161/16' 2024-05-27T09:30:36.142 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/161/17' -> 'copyofsnap1/test1/161/17' 2024-05-27T09:30:36.142 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/161/18' -> 'copyofsnap1/test1/161/18' 2024-05-27T09:30:36.142 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/161/19' -> 'copyofsnap1/test1/161/19' 2024-05-27T09:30:36.142 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/161/20' -> 'copyofsnap1/test1/161/20' 2024-05-27T09:30:36.142 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/162' -> 'copyofsnap1/test1/162' 2024-05-27T09:30:36.143 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/162/10' -> 'copyofsnap1/test1/162/10' 2024-05-27T09:30:36.143 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/162/11' -> 'copyofsnap1/test1/162/11' 2024-05-27T09:30:36.143 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/162/12' -> 'copyofsnap1/test1/162/12' 2024-05-27T09:30:36.143 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/162/13' -> 'copyofsnap1/test1/162/13' 2024-05-27T09:30:36.143 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/162/14' -> 'copyofsnap1/test1/162/14' 2024-05-27T09:30:36.143 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/162/15' -> 'copyofsnap1/test1/162/15' 2024-05-27T09:30:36.143 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/162/16' -> 'copyofsnap1/test1/162/16' 2024-05-27T09:30:36.143 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/162/17' -> 'copyofsnap1/test1/162/17' 2024-05-27T09:30:36.143 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/162/18' -> 'copyofsnap1/test1/162/18' 2024-05-27T09:30:36.143 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/162/19' -> 'copyofsnap1/test1/162/19' 2024-05-27T09:30:36.143 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/162/20' -> 'copyofsnap1/test1/162/20' 2024-05-27T09:30:36.143 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/163' -> 'copyofsnap1/test1/163' 2024-05-27T09:30:36.143 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/163/10' -> 'copyofsnap1/test1/163/10' 2024-05-27T09:30:36.143 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/163/11' -> 'copyofsnap1/test1/163/11' 2024-05-27T09:30:36.144 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/163/12' -> 'copyofsnap1/test1/163/12' 2024-05-27T09:30:36.144 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/163/13' -> 'copyofsnap1/test1/163/13' 2024-05-27T09:30:36.144 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/163/14' -> 'copyofsnap1/test1/163/14' 2024-05-27T09:30:36.144 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/163/15' -> 'copyofsnap1/test1/163/15' 2024-05-27T09:30:36.144 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/163/16' -> 'copyofsnap1/test1/163/16' 2024-05-27T09:30:36.144 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/163/17' -> 'copyofsnap1/test1/163/17' 2024-05-27T09:30:36.144 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/163/18' -> 'copyofsnap1/test1/163/18' 2024-05-27T09:30:36.144 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/163/19' -> 'copyofsnap1/test1/163/19' 2024-05-27T09:30:36.144 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/163/20' -> 'copyofsnap1/test1/163/20' 2024-05-27T09:30:36.144 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/164' -> 'copyofsnap1/test1/164' 2024-05-27T09:30:36.144 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/164/10' -> 'copyofsnap1/test1/164/10' 2024-05-27T09:30:36.144 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/164/11' -> 'copyofsnap1/test1/164/11' 2024-05-27T09:30:36.144 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/164/12' -> 'copyofsnap1/test1/164/12' 2024-05-27T09:30:36.145 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/164/13' -> 'copyofsnap1/test1/164/13' 2024-05-27T09:30:36.145 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/164/14' -> 'copyofsnap1/test1/164/14' 2024-05-27T09:30:36.145 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/164/15' -> 'copyofsnap1/test1/164/15' 2024-05-27T09:30:36.145 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/164/16' -> 'copyofsnap1/test1/164/16' 2024-05-27T09:30:36.145 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/164/17' -> 'copyofsnap1/test1/164/17' 2024-05-27T09:30:36.145 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/164/18' -> 'copyofsnap1/test1/164/18' 2024-05-27T09:30:36.145 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/164/19' -> 'copyofsnap1/test1/164/19' 2024-05-27T09:30:36.145 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/164/20' -> 'copyofsnap1/test1/164/20' 2024-05-27T09:30:36.145 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/165' -> 'copyofsnap1/test1/165' 2024-05-27T09:30:36.145 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/165/10' -> 'copyofsnap1/test1/165/10' 2024-05-27T09:30:36.145 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/165/11' -> 'copyofsnap1/test1/165/11' 2024-05-27T09:30:36.145 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/165/12' -> 'copyofsnap1/test1/165/12' 2024-05-27T09:30:36.145 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/165/13' -> 'copyofsnap1/test1/165/13' 2024-05-27T09:30:36.145 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/165/14' -> 'copyofsnap1/test1/165/14' 2024-05-27T09:30:36.145 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/165/15' -> 'copyofsnap1/test1/165/15' 2024-05-27T09:30:36.146 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/165/16' -> 'copyofsnap1/test1/165/16' 2024-05-27T09:30:36.146 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/165/17' -> 'copyofsnap1/test1/165/17' 2024-05-27T09:30:36.146 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/165/18' -> 'copyofsnap1/test1/165/18' 2024-05-27T09:30:36.146 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/165/19' -> 'copyofsnap1/test1/165/19' 2024-05-27T09:30:36.146 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/165/20' -> 'copyofsnap1/test1/165/20' 2024-05-27T09:30:36.146 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/166' -> 'copyofsnap1/test1/166' 2024-05-27T09:30:36.146 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/166/10' -> 'copyofsnap1/test1/166/10' 2024-05-27T09:30:36.146 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/166/11' -> 'copyofsnap1/test1/166/11' 2024-05-27T09:30:36.146 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/166/12' -> 'copyofsnap1/test1/166/12' 2024-05-27T09:30:36.146 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/166/13' -> 'copyofsnap1/test1/166/13' 2024-05-27T09:30:36.146 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/166/14' -> 'copyofsnap1/test1/166/14' 2024-05-27T09:30:36.146 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/166/15' -> 'copyofsnap1/test1/166/15' 2024-05-27T09:30:36.146 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/166/16' -> 'copyofsnap1/test1/166/16' 2024-05-27T09:30:36.147 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/166/17' -> 'copyofsnap1/test1/166/17' 2024-05-27T09:30:36.147 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/166/18' -> 'copyofsnap1/test1/166/18' 2024-05-27T09:30:36.147 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/166/19' -> 'copyofsnap1/test1/166/19' 2024-05-27T09:30:36.147 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/166/20' -> 'copyofsnap1/test1/166/20' 2024-05-27T09:30:36.147 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/167' -> 'copyofsnap1/test1/167' 2024-05-27T09:30:36.147 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/167/10' -> 'copyofsnap1/test1/167/10' 2024-05-27T09:30:36.147 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/167/11' -> 'copyofsnap1/test1/167/11' 2024-05-27T09:30:36.147 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/167/12' -> 'copyofsnap1/test1/167/12' 2024-05-27T09:30:36.147 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/167/13' -> 'copyofsnap1/test1/167/13' 2024-05-27T09:30:36.147 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/167/14' -> 'copyofsnap1/test1/167/14' 2024-05-27T09:30:36.530 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/167/15' -> 'copyofsnap1/test1/167/15' 2024-05-27T09:30:36.530 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/167/16' -> 'copyofsnap1/test1/167/16' 2024-05-27T09:30:36.530 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/167/17' -> 'copyofsnap1/test1/167/17' 2024-05-27T09:30:36.530 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/167/18' -> 'copyofsnap1/test1/167/18' 2024-05-27T09:30:36.530 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/167/19' -> 'copyofsnap1/test1/167/19' 2024-05-27T09:30:36.530 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/167/20' -> 'copyofsnap1/test1/167/20' 2024-05-27T09:30:36.530 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/168' -> 'copyofsnap1/test1/168' 2024-05-27T09:30:36.530 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/168/10' -> 'copyofsnap1/test1/168/10' 2024-05-27T09:30:36.530 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/168/11' -> 'copyofsnap1/test1/168/11' 2024-05-27T09:30:36.530 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/168/12' -> 'copyofsnap1/test1/168/12' 2024-05-27T09:30:36.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/168/13' -> 'copyofsnap1/test1/168/13' 2024-05-27T09:30:36.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/168/14' -> 'copyofsnap1/test1/168/14' 2024-05-27T09:30:36.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/168/15' -> 'copyofsnap1/test1/168/15' 2024-05-27T09:30:36.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/168/16' -> 'copyofsnap1/test1/168/16' 2024-05-27T09:30:36.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/168/17' -> 'copyofsnap1/test1/168/17' 2024-05-27T09:30:36.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/168/18' -> 'copyofsnap1/test1/168/18' 2024-05-27T09:30:36.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/168/19' -> 'copyofsnap1/test1/168/19' 2024-05-27T09:30:36.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/168/20' -> 'copyofsnap1/test1/168/20' 2024-05-27T09:30:36.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/169' -> 'copyofsnap1/test1/169' 2024-05-27T09:30:36.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/169/10' -> 'copyofsnap1/test1/169/10' 2024-05-27T09:30:36.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/169/11' -> 'copyofsnap1/test1/169/11' 2024-05-27T09:30:36.531 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/169/12' -> 'copyofsnap1/test1/169/12' 2024-05-27T09:30:36.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/169/13' -> 'copyofsnap1/test1/169/13' 2024-05-27T09:30:36.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/169/14' -> 'copyofsnap1/test1/169/14' 2024-05-27T09:30:36.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/169/15' -> 'copyofsnap1/test1/169/15' 2024-05-27T09:30:36.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/169/16' -> 'copyofsnap1/test1/169/16' 2024-05-27T09:30:36.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/169/17' -> 'copyofsnap1/test1/169/17' 2024-05-27T09:30:36.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/169/18' -> 'copyofsnap1/test1/169/18' 2024-05-27T09:30:36.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/169/19' -> 'copyofsnap1/test1/169/19' 2024-05-27T09:30:36.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/169/20' -> 'copyofsnap1/test1/169/20' 2024-05-27T09:30:36.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/170' -> 'copyofsnap1/test1/170' 2024-05-27T09:30:36.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/170/10' -> 'copyofsnap1/test1/170/10' 2024-05-27T09:30:36.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/170/11' -> 'copyofsnap1/test1/170/11' 2024-05-27T09:30:36.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/170/12' -> 'copyofsnap1/test1/170/12' 2024-05-27T09:30:36.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/170/13' -> 'copyofsnap1/test1/170/13' 2024-05-27T09:30:36.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/170/14' -> 'copyofsnap1/test1/170/14' 2024-05-27T09:30:36.532 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/170/15' -> 'copyofsnap1/test1/170/15' 2024-05-27T09:30:36.533 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/170/16' -> 'copyofsnap1/test1/170/16' 2024-05-27T09:30:36.533 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/170/17' -> 'copyofsnap1/test1/170/17' 2024-05-27T09:30:36.533 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/170/18' -> 'copyofsnap1/test1/170/18' 2024-05-27T09:30:36.533 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/170/19' -> 'copyofsnap1/test1/170/19' 2024-05-27T09:30:36.533 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/170/20' -> 'copyofsnap1/test1/170/20' 2024-05-27T09:30:36.533 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/171' -> 'copyofsnap1/test1/171' 2024-05-27T09:30:36.533 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/171/10' -> 'copyofsnap1/test1/171/10' 2024-05-27T09:30:36.533 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/171/11' -> 'copyofsnap1/test1/171/11' 2024-05-27T09:30:36.533 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/171/12' -> 'copyofsnap1/test1/171/12' 2024-05-27T09:30:36.533 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/171/13' -> 'copyofsnap1/test1/171/13' 2024-05-27T09:30:36.533 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/171/14' -> 'copyofsnap1/test1/171/14' 2024-05-27T09:30:36.533 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/171/15' -> 'copyofsnap1/test1/171/15' 2024-05-27T09:30:36.533 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/171/16' -> 'copyofsnap1/test1/171/16' 2024-05-27T09:30:36.534 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/171/17' -> 'copyofsnap1/test1/171/17' 2024-05-27T09:30:36.534 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/171/18' -> 'copyofsnap1/test1/171/18' 2024-05-27T09:30:36.534 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/171/19' -> 'copyofsnap1/test1/171/19' 2024-05-27T09:30:36.534 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/171/20' -> 'copyofsnap1/test1/171/20' 2024-05-27T09:30:36.534 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/172' -> 'copyofsnap1/test1/172' 2024-05-27T09:30:36.534 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/172/10' -> 'copyofsnap1/test1/172/10' 2024-05-27T09:30:36.534 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/172/11' -> 'copyofsnap1/test1/172/11' 2024-05-27T09:30:36.534 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/172/12' -> 'copyofsnap1/test1/172/12' 2024-05-27T09:30:36.534 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/172/13' -> 'copyofsnap1/test1/172/13' 2024-05-27T09:30:36.534 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/172/14' -> 'copyofsnap1/test1/172/14' 2024-05-27T09:30:36.535 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/172/15' -> 'copyofsnap1/test1/172/15' 2024-05-27T09:30:36.535 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/172/16' -> 'copyofsnap1/test1/172/16' 2024-05-27T09:30:36.535 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/172/17' -> 'copyofsnap1/test1/172/17' 2024-05-27T09:30:36.535 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/172/18' -> 'copyofsnap1/test1/172/18' 2024-05-27T09:30:36.535 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/172/19' -> 'copyofsnap1/test1/172/19' 2024-05-27T09:30:36.535 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/172/20' -> 'copyofsnap1/test1/172/20' 2024-05-27T09:30:36.535 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/173' -> 'copyofsnap1/test1/173' 2024-05-27T09:30:36.535 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/173/10' -> 'copyofsnap1/test1/173/10' 2024-05-27T09:30:36.535 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/173/11' -> 'copyofsnap1/test1/173/11' 2024-05-27T09:30:36.535 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/173/12' -> 'copyofsnap1/test1/173/12' 2024-05-27T09:30:36.536 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/173/13' -> 'copyofsnap1/test1/173/13' 2024-05-27T09:30:36.536 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/173/14' -> 'copyofsnap1/test1/173/14' 2024-05-27T09:30:36.536 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/173/15' -> 'copyofsnap1/test1/173/15' 2024-05-27T09:30:36.536 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/173/16' -> 'copyofsnap1/test1/173/16' 2024-05-27T09:30:36.536 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/173/17' -> 'copyofsnap1/test1/173/17' 2024-05-27T09:30:36.536 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/173/18' -> 'copyofsnap1/test1/173/18' 2024-05-27T09:30:36.536 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/173/19' -> 'copyofsnap1/test1/173/19' 2024-05-27T09:30:36.536 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/173/20' -> 'copyofsnap1/test1/173/20' 2024-05-27T09:30:36.536 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/174' -> 'copyofsnap1/test1/174' 2024-05-27T09:30:36.536 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/174/10' -> 'copyofsnap1/test1/174/10' 2024-05-27T09:30:36.536 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/174/11' -> 'copyofsnap1/test1/174/11' 2024-05-27T09:30:36.927 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/174/12' -> 'copyofsnap1/test1/174/12' 2024-05-27T09:30:36.927 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/174/13' -> 'copyofsnap1/test1/174/13' 2024-05-27T09:30:36.927 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/174/14' -> 'copyofsnap1/test1/174/14' 2024-05-27T09:30:36.927 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/174/15' -> 'copyofsnap1/test1/174/15' 2024-05-27T09:30:36.928 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/174/16' -> 'copyofsnap1/test1/174/16' 2024-05-27T09:30:36.928 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/174/17' -> 'copyofsnap1/test1/174/17' 2024-05-27T09:30:36.928 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/174/18' -> 'copyofsnap1/test1/174/18' 2024-05-27T09:30:36.928 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/174/19' -> 'copyofsnap1/test1/174/19' 2024-05-27T09:30:36.928 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/174/20' -> 'copyofsnap1/test1/174/20' 2024-05-27T09:30:36.928 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/175' -> 'copyofsnap1/test1/175' 2024-05-27T09:30:36.928 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/175/10' -> 'copyofsnap1/test1/175/10' 2024-05-27T09:30:36.928 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/175/11' -> 'copyofsnap1/test1/175/11' 2024-05-27T09:30:36.928 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/175/12' -> 'copyofsnap1/test1/175/12' 2024-05-27T09:30:36.928 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/175/13' -> 'copyofsnap1/test1/175/13' 2024-05-27T09:30:36.928 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/175/14' -> 'copyofsnap1/test1/175/14' 2024-05-27T09:30:36.928 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/175/15' -> 'copyofsnap1/test1/175/15' 2024-05-27T09:30:36.928 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/175/16' -> 'copyofsnap1/test1/175/16' 2024-05-27T09:30:36.928 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/175/17' -> 'copyofsnap1/test1/175/17' 2024-05-27T09:30:36.929 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/175/18' -> 'copyofsnap1/test1/175/18' 2024-05-27T09:30:36.929 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/175/19' -> 'copyofsnap1/test1/175/19' 2024-05-27T09:30:36.929 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/175/20' -> 'copyofsnap1/test1/175/20' 2024-05-27T09:30:36.929 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/176' -> 'copyofsnap1/test1/176' 2024-05-27T09:30:36.929 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/176/10' -> 'copyofsnap1/test1/176/10' 2024-05-27T09:30:36.929 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/176/11' -> 'copyofsnap1/test1/176/11' 2024-05-27T09:30:36.929 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/176/12' -> 'copyofsnap1/test1/176/12' 2024-05-27T09:30:36.929 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/176/13' -> 'copyofsnap1/test1/176/13' 2024-05-27T09:30:36.929 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/176/14' -> 'copyofsnap1/test1/176/14' 2024-05-27T09:30:36.929 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/176/15' -> 'copyofsnap1/test1/176/15' 2024-05-27T09:30:36.929 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/176/16' -> 'copyofsnap1/test1/176/16' 2024-05-27T09:30:36.929 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/176/17' -> 'copyofsnap1/test1/176/17' 2024-05-27T09:30:36.929 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/176/18' -> 'copyofsnap1/test1/176/18' 2024-05-27T09:30:36.930 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/176/19' -> 'copyofsnap1/test1/176/19' 2024-05-27T09:30:36.930 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/176/20' -> 'copyofsnap1/test1/176/20' 2024-05-27T09:30:36.930 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/177' -> 'copyofsnap1/test1/177' 2024-05-27T09:30:36.930 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/177/10' -> 'copyofsnap1/test1/177/10' 2024-05-27T09:30:36.930 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/177/11' -> 'copyofsnap1/test1/177/11' 2024-05-27T09:30:36.930 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/177/12' -> 'copyofsnap1/test1/177/12' 2024-05-27T09:30:36.930 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/177/13' -> 'copyofsnap1/test1/177/13' 2024-05-27T09:30:36.930 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/177/14' -> 'copyofsnap1/test1/177/14' 2024-05-27T09:30:36.930 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/177/15' -> 'copyofsnap1/test1/177/15' 2024-05-27T09:30:36.930 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/177/16' -> 'copyofsnap1/test1/177/16' 2024-05-27T09:30:36.930 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/177/17' -> 'copyofsnap1/test1/177/17' 2024-05-27T09:30:36.930 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/177/18' -> 'copyofsnap1/test1/177/18' 2024-05-27T09:30:36.930 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/177/19' -> 'copyofsnap1/test1/177/19' 2024-05-27T09:30:36.931 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/177/20' -> 'copyofsnap1/test1/177/20' 2024-05-27T09:30:36.931 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/178' -> 'copyofsnap1/test1/178' 2024-05-27T09:30:36.931 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/178/10' -> 'copyofsnap1/test1/178/10' 2024-05-27T09:30:36.931 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/178/11' -> 'copyofsnap1/test1/178/11' 2024-05-27T09:30:36.931 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/178/12' -> 'copyofsnap1/test1/178/12' 2024-05-27T09:30:36.931 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/178/13' -> 'copyofsnap1/test1/178/13' 2024-05-27T09:30:36.931 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/178/14' -> 'copyofsnap1/test1/178/14' 2024-05-27T09:30:36.931 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/178/15' -> 'copyofsnap1/test1/178/15' 2024-05-27T09:30:36.931 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/178/16' -> 'copyofsnap1/test1/178/16' 2024-05-27T09:30:36.931 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/178/17' -> 'copyofsnap1/test1/178/17' 2024-05-27T09:30:36.931 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/178/18' -> 'copyofsnap1/test1/178/18' 2024-05-27T09:30:36.931 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/178/19' -> 'copyofsnap1/test1/178/19' 2024-05-27T09:30:36.931 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/178/20' -> 'copyofsnap1/test1/178/20' 2024-05-27T09:30:36.932 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/179' -> 'copyofsnap1/test1/179' 2024-05-27T09:30:36.932 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/179/10' -> 'copyofsnap1/test1/179/10' 2024-05-27T09:30:36.932 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/179/11' -> 'copyofsnap1/test1/179/11' 2024-05-27T09:30:36.932 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/179/12' -> 'copyofsnap1/test1/179/12' 2024-05-27T09:30:36.932 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/179/13' -> 'copyofsnap1/test1/179/13' 2024-05-27T09:30:36.932 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/179/14' -> 'copyofsnap1/test1/179/14' 2024-05-27T09:30:36.932 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/179/15' -> 'copyofsnap1/test1/179/15' 2024-05-27T09:30:36.932 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/179/16' -> 'copyofsnap1/test1/179/16' 2024-05-27T09:30:36.932 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/179/17' -> 'copyofsnap1/test1/179/17' 2024-05-27T09:30:36.932 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/179/18' -> 'copyofsnap1/test1/179/18' 2024-05-27T09:30:36.932 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/179/19' -> 'copyofsnap1/test1/179/19' 2024-05-27T09:30:36.932 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/179/20' -> 'copyofsnap1/test1/179/20' 2024-05-27T09:30:36.932 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/180' -> 'copyofsnap1/test1/180' 2024-05-27T09:30:36.932 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/180/10' -> 'copyofsnap1/test1/180/10' 2024-05-27T09:30:36.932 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/180/11' -> 'copyofsnap1/test1/180/11' 2024-05-27T09:30:36.933 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/180/12' -> 'copyofsnap1/test1/180/12' 2024-05-27T09:30:36.933 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/180/13' -> 'copyofsnap1/test1/180/13' 2024-05-27T09:30:36.933 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/180/14' -> 'copyofsnap1/test1/180/14' 2024-05-27T09:30:36.933 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/180/15' -> 'copyofsnap1/test1/180/15' 2024-05-27T09:30:36.933 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/180/16' -> 'copyofsnap1/test1/180/16' 2024-05-27T09:30:36.933 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/180/17' -> 'copyofsnap1/test1/180/17' 2024-05-27T09:30:36.933 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/180/18' -> 'copyofsnap1/test1/180/18' 2024-05-27T09:30:36.933 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/180/19' -> 'copyofsnap1/test1/180/19' 2024-05-27T09:30:36.933 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/180/20' -> 'copyofsnap1/test1/180/20' 2024-05-27T09:30:41.257 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/181' -> 'copyofsnap1/test1/181' 2024-05-27T09:30:41.257 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/181/10' -> 'copyofsnap1/test1/181/10' 2024-05-27T09:30:41.257 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/181/11' -> 'copyofsnap1/test1/181/11' 2024-05-27T09:30:41.257 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/181/12' -> 'copyofsnap1/test1/181/12' 2024-05-27T09:30:41.257 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/181/13' -> 'copyofsnap1/test1/181/13' 2024-05-27T09:30:41.257 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/181/14' -> 'copyofsnap1/test1/181/14' 2024-05-27T09:30:41.258 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/181/15' -> 'copyofsnap1/test1/181/15' 2024-05-27T09:30:41.258 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/181/16' -> 'copyofsnap1/test1/181/16' 2024-05-27T09:30:41.258 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/181/17' -> 'copyofsnap1/test1/181/17' 2024-05-27T09:30:41.258 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/181/18' -> 'copyofsnap1/test1/181/18' 2024-05-27T09:30:41.258 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/181/19' -> 'copyofsnap1/test1/181/19' 2024-05-27T09:30:41.258 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/181/20' -> 'copyofsnap1/test1/181/20' 2024-05-27T09:30:41.258 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/182' -> 'copyofsnap1/test1/182' 2024-05-27T09:30:41.258 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/182/10' -> 'copyofsnap1/test1/182/10' 2024-05-27T09:30:41.258 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/182/11' -> 'copyofsnap1/test1/182/11' 2024-05-27T09:30:41.258 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/182/12' -> 'copyofsnap1/test1/182/12' 2024-05-27T09:30:41.258 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/182/13' -> 'copyofsnap1/test1/182/13' 2024-05-27T09:30:41.259 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/182/14' -> 'copyofsnap1/test1/182/14' 2024-05-27T09:30:41.259 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/182/15' -> 'copyofsnap1/test1/182/15' 2024-05-27T09:30:41.259 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/182/16' -> 'copyofsnap1/test1/182/16' 2024-05-27T09:30:41.259 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/182/17' -> 'copyofsnap1/test1/182/17' 2024-05-27T09:30:41.259 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/182/19' -> 'copyofsnap1/test1/182/19' 2024-05-27T09:30:41.259 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/182/20' -> 'copyofsnap1/test1/182/20' 2024-05-27T09:30:41.259 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/182/18' -> 'copyofsnap1/test1/182/18' 2024-05-27T09:30:41.259 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/183' -> 'copyofsnap1/test1/183' 2024-05-27T09:30:41.259 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/183/10' -> 'copyofsnap1/test1/183/10' 2024-05-27T09:30:41.259 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/183/11' -> 'copyofsnap1/test1/183/11' 2024-05-27T09:30:41.259 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/183/12' -> 'copyofsnap1/test1/183/12' 2024-05-27T09:30:41.259 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/183/13' -> 'copyofsnap1/test1/183/13' 2024-05-27T09:30:41.259 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/183/14' -> 'copyofsnap1/test1/183/14' 2024-05-27T09:30:41.260 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/183/15' -> 'copyofsnap1/test1/183/15' 2024-05-27T09:30:41.260 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/183/16' -> 'copyofsnap1/test1/183/16' 2024-05-27T09:30:41.260 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/183/17' -> 'copyofsnap1/test1/183/17' 2024-05-27T09:30:41.260 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/183/18' -> 'copyofsnap1/test1/183/18' 2024-05-27T09:30:41.260 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/183/19' -> 'copyofsnap1/test1/183/19' 2024-05-27T09:30:41.260 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/183/20' -> 'copyofsnap1/test1/183/20' 2024-05-27T09:30:41.260 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/184' -> 'copyofsnap1/test1/184' 2024-05-27T09:30:41.260 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/184/10' -> 'copyofsnap1/test1/184/10' 2024-05-27T09:30:41.260 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/184/11' -> 'copyofsnap1/test1/184/11' 2024-05-27T09:30:41.260 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/184/12' -> 'copyofsnap1/test1/184/12' 2024-05-27T09:30:41.260 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/184/13' -> 'copyofsnap1/test1/184/13' 2024-05-27T09:30:41.260 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/184/14' -> 'copyofsnap1/test1/184/14' 2024-05-27T09:30:41.260 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/184/15' -> 'copyofsnap1/test1/184/15' 2024-05-27T09:30:41.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/184/16' -> 'copyofsnap1/test1/184/16' 2024-05-27T09:30:41.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/184/17' -> 'copyofsnap1/test1/184/17' 2024-05-27T09:30:41.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/184/18' -> 'copyofsnap1/test1/184/18' 2024-05-27T09:30:41.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/184/19' -> 'copyofsnap1/test1/184/19' 2024-05-27T09:30:41.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/184/20' -> 'copyofsnap1/test1/184/20' 2024-05-27T09:30:41.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/185' -> 'copyofsnap1/test1/185' 2024-05-27T09:30:41.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/185/10' -> 'copyofsnap1/test1/185/10' 2024-05-27T09:30:41.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/185/11' -> 'copyofsnap1/test1/185/11' 2024-05-27T09:30:41.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/185/12' -> 'copyofsnap1/test1/185/12' 2024-05-27T09:30:41.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/185/13' -> 'copyofsnap1/test1/185/13' 2024-05-27T09:30:41.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/185/14' -> 'copyofsnap1/test1/185/14' 2024-05-27T09:30:41.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/185/15' -> 'copyofsnap1/test1/185/15' 2024-05-27T09:30:41.261 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/185/16' -> 'copyofsnap1/test1/185/16' 2024-05-27T09:30:41.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/185/17' -> 'copyofsnap1/test1/185/17' 2024-05-27T09:30:41.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/185/18' -> 'copyofsnap1/test1/185/18' 2024-05-27T09:30:41.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/185/19' -> 'copyofsnap1/test1/185/19' 2024-05-27T09:30:41.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/185/20' -> 'copyofsnap1/test1/185/20' 2024-05-27T09:30:41.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/186' -> 'copyofsnap1/test1/186' 2024-05-27T09:30:41.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/186/10' -> 'copyofsnap1/test1/186/10' 2024-05-27T09:30:41.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/186/11' -> 'copyofsnap1/test1/186/11' 2024-05-27T09:30:41.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/186/12' -> 'copyofsnap1/test1/186/12' 2024-05-27T09:30:41.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/186/13' -> 'copyofsnap1/test1/186/13' 2024-05-27T09:30:41.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/186/14' -> 'copyofsnap1/test1/186/14' 2024-05-27T09:30:41.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/186/15' -> 'copyofsnap1/test1/186/15' 2024-05-27T09:30:41.262 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/186/16' -> 'copyofsnap1/test1/186/16' 2024-05-27T09:30:41.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/186/17' -> 'copyofsnap1/test1/186/17' 2024-05-27T09:30:41.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/186/18' -> 'copyofsnap1/test1/186/18' 2024-05-27T09:30:41.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/186/19' -> 'copyofsnap1/test1/186/19' 2024-05-27T09:30:41.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/186/20' -> 'copyofsnap1/test1/186/20' 2024-05-27T09:30:41.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/187' -> 'copyofsnap1/test1/187' 2024-05-27T09:30:41.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/187/10' -> 'copyofsnap1/test1/187/10' 2024-05-27T09:30:41.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/187/11' -> 'copyofsnap1/test1/187/11' 2024-05-27T09:30:41.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/187/12' -> 'copyofsnap1/test1/187/12' 2024-05-27T09:30:41.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/187/13' -> 'copyofsnap1/test1/187/13' 2024-05-27T09:30:41.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/187/14' -> 'copyofsnap1/test1/187/14' 2024-05-27T09:30:41.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/187/15' -> 'copyofsnap1/test1/187/15' 2024-05-27T09:30:41.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/187/16' -> 'copyofsnap1/test1/187/16' 2024-05-27T09:30:41.263 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/187/17' -> 'copyofsnap1/test1/187/17' 2024-05-27T09:30:41.635 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/187/18' -> 'copyofsnap1/test1/187/18' 2024-05-27T09:30:41.635 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/187/19' -> 'copyofsnap1/test1/187/19' 2024-05-27T09:30:41.635 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/187/20' -> 'copyofsnap1/test1/187/20' 2024-05-27T09:30:41.635 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/188' -> 'copyofsnap1/test1/188' 2024-05-27T09:30:41.635 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/188/10' -> 'copyofsnap1/test1/188/10' 2024-05-27T09:30:41.635 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/188/11' -> 'copyofsnap1/test1/188/11' 2024-05-27T09:30:41.635 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/188/12' -> 'copyofsnap1/test1/188/12' 2024-05-27T09:30:41.635 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/188/13' -> 'copyofsnap1/test1/188/13' 2024-05-27T09:30:41.635 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/188/14' -> 'copyofsnap1/test1/188/14' 2024-05-27T09:30:41.635 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/188/15' -> 'copyofsnap1/test1/188/15' 2024-05-27T09:30:41.636 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/188/16' -> 'copyofsnap1/test1/188/16' 2024-05-27T09:30:41.636 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/188/17' -> 'copyofsnap1/test1/188/17' 2024-05-27T09:30:41.636 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/188/18' -> 'copyofsnap1/test1/188/18' 2024-05-27T09:30:41.636 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/188/19' -> 'copyofsnap1/test1/188/19' 2024-05-27T09:30:41.636 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/188/20' -> 'copyofsnap1/test1/188/20' 2024-05-27T09:30:41.636 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/189' -> 'copyofsnap1/test1/189' 2024-05-27T09:30:41.636 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/189/10' -> 'copyofsnap1/test1/189/10' 2024-05-27T09:30:41.636 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/189/11' -> 'copyofsnap1/test1/189/11' 2024-05-27T09:30:41.636 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/189/12' -> 'copyofsnap1/test1/189/12' 2024-05-27T09:30:41.636 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/189/13' -> 'copyofsnap1/test1/189/13' 2024-05-27T09:30:41.636 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/189/14' -> 'copyofsnap1/test1/189/14' 2024-05-27T09:30:41.636 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/189/15' -> 'copyofsnap1/test1/189/15' 2024-05-27T09:30:41.636 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/189/16' -> 'copyofsnap1/test1/189/16' 2024-05-27T09:30:41.636 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/189/17' -> 'copyofsnap1/test1/189/17' 2024-05-27T09:30:41.637 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/189/18' -> 'copyofsnap1/test1/189/18' 2024-05-27T09:30:41.637 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/189/19' -> 'copyofsnap1/test1/189/19' 2024-05-27T09:30:41.637 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/189/20' -> 'copyofsnap1/test1/189/20' 2024-05-27T09:30:41.637 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/190' -> 'copyofsnap1/test1/190' 2024-05-27T09:30:41.637 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/190/10' -> 'copyofsnap1/test1/190/10' 2024-05-27T09:30:41.637 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/190/11' -> 'copyofsnap1/test1/190/11' 2024-05-27T09:30:41.637 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/190/12' -> 'copyofsnap1/test1/190/12' 2024-05-27T09:30:41.637 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/190/13' -> 'copyofsnap1/test1/190/13' 2024-05-27T09:30:41.637 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/190/14' -> 'copyofsnap1/test1/190/14' 2024-05-27T09:30:41.637 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/190/15' -> 'copyofsnap1/test1/190/15' 2024-05-27T09:30:41.637 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/190/16' -> 'copyofsnap1/test1/190/16' 2024-05-27T09:30:41.637 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/190/17' -> 'copyofsnap1/test1/190/17' 2024-05-27T09:30:41.637 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/190/18' -> 'copyofsnap1/test1/190/18' 2024-05-27T09:30:41.637 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/190/19' -> 'copyofsnap1/test1/190/19' 2024-05-27T09:30:41.638 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/190/20' -> 'copyofsnap1/test1/190/20' 2024-05-27T09:30:41.638 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/191' -> 'copyofsnap1/test1/191' 2024-05-27T09:30:41.638 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/191/10' -> 'copyofsnap1/test1/191/10' 2024-05-27T09:30:41.638 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/191/11' -> 'copyofsnap1/test1/191/11' 2024-05-27T09:30:41.638 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/191/12' -> 'copyofsnap1/test1/191/12' 2024-05-27T09:30:41.638 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/191/13' -> 'copyofsnap1/test1/191/13' 2024-05-27T09:30:41.638 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/191/14' -> 'copyofsnap1/test1/191/14' 2024-05-27T09:30:41.638 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/191/15' -> 'copyofsnap1/test1/191/15' 2024-05-27T09:30:41.638 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/191/16' -> 'copyofsnap1/test1/191/16' 2024-05-27T09:30:41.638 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/191/17' -> 'copyofsnap1/test1/191/17' 2024-05-27T09:30:41.638 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/191/18' -> 'copyofsnap1/test1/191/18' 2024-05-27T09:30:41.638 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/191/19' -> 'copyofsnap1/test1/191/19' 2024-05-27T09:30:41.638 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/191/20' -> 'copyofsnap1/test1/191/20' 2024-05-27T09:30:41.639 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/192' -> 'copyofsnap1/test1/192' 2024-05-27T09:30:41.639 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/192/10' -> 'copyofsnap1/test1/192/10' 2024-05-27T09:30:41.639 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/192/11' -> 'copyofsnap1/test1/192/11' 2024-05-27T09:30:41.639 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/192/12' -> 'copyofsnap1/test1/192/12' 2024-05-27T09:30:41.639 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/192/13' -> 'copyofsnap1/test1/192/13' 2024-05-27T09:30:41.639 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/192/14' -> 'copyofsnap1/test1/192/14' 2024-05-27T09:30:41.639 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/192/15' -> 'copyofsnap1/test1/192/15' 2024-05-27T09:30:41.639 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/192/16' -> 'copyofsnap1/test1/192/16' 2024-05-27T09:30:41.639 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/192/17' -> 'copyofsnap1/test1/192/17' 2024-05-27T09:30:41.639 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/192/18' -> 'copyofsnap1/test1/192/18' 2024-05-27T09:30:41.639 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/192/19' -> 'copyofsnap1/test1/192/19' 2024-05-27T09:30:41.639 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/192/20' -> 'copyofsnap1/test1/192/20' 2024-05-27T09:30:41.639 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/193' -> 'copyofsnap1/test1/193' 2024-05-27T09:30:41.639 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/193/10' -> 'copyofsnap1/test1/193/10' 2024-05-27T09:30:41.640 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/193/11' -> 'copyofsnap1/test1/193/11' 2024-05-27T09:30:41.640 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/193/12' -> 'copyofsnap1/test1/193/12' 2024-05-27T09:30:41.640 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/193/13' -> 'copyofsnap1/test1/193/13' 2024-05-27T09:30:41.640 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/193/14' -> 'copyofsnap1/test1/193/14' 2024-05-27T09:30:41.640 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/193/15' -> 'copyofsnap1/test1/193/15' 2024-05-27T09:30:41.640 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/193/16' -> 'copyofsnap1/test1/193/16' 2024-05-27T09:30:41.640 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/193/17' -> 'copyofsnap1/test1/193/17' 2024-05-27T09:30:41.640 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/193/18' -> 'copyofsnap1/test1/193/18' 2024-05-27T09:30:41.640 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/193/19' -> 'copyofsnap1/test1/193/19' 2024-05-27T09:30:41.640 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/193/20' -> 'copyofsnap1/test1/193/20' 2024-05-27T09:30:41.640 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/194' -> 'copyofsnap1/test1/194' 2024-05-27T09:30:41.640 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/194/10' -> 'copyofsnap1/test1/194/10' 2024-05-27T09:30:41.640 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/194/11' -> 'copyofsnap1/test1/194/11' 2024-05-27T09:30:41.640 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/194/12' -> 'copyofsnap1/test1/194/12' 2024-05-27T09:30:41.641 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/194/13' -> 'copyofsnap1/test1/194/13' 2024-05-27T09:30:41.641 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/194/14' -> 'copyofsnap1/test1/194/14' 2024-05-27T09:30:41.941 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/194/15' -> 'copyofsnap1/test1/194/15' 2024-05-27T09:30:41.941 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/194/16' -> 'copyofsnap1/test1/194/16' 2024-05-27T09:30:41.941 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/194/17' -> 'copyofsnap1/test1/194/17' 2024-05-27T09:30:41.942 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/194/18' -> 'copyofsnap1/test1/194/18' 2024-05-27T09:30:41.942 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/194/19' -> 'copyofsnap1/test1/194/19' 2024-05-27T09:30:41.942 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/194/20' -> 'copyofsnap1/test1/194/20' 2024-05-27T09:30:41.942 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/195' -> 'copyofsnap1/test1/195' 2024-05-27T09:30:41.942 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/195/10' -> 'copyofsnap1/test1/195/10' 2024-05-27T09:30:41.942 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/195/11' -> 'copyofsnap1/test1/195/11' 2024-05-27T09:30:41.942 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/195/12' -> 'copyofsnap1/test1/195/12' 2024-05-27T09:30:41.942 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/195/13' -> 'copyofsnap1/test1/195/13' 2024-05-27T09:30:41.942 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/195/14' -> 'copyofsnap1/test1/195/14' 2024-05-27T09:30:41.942 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/195/15' -> 'copyofsnap1/test1/195/15' 2024-05-27T09:30:41.942 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/195/16' -> 'copyofsnap1/test1/195/16' 2024-05-27T09:30:41.942 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/195/17' -> 'copyofsnap1/test1/195/17' 2024-05-27T09:30:41.942 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/195/18' -> 'copyofsnap1/test1/195/18' 2024-05-27T09:30:41.943 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/195/19' -> 'copyofsnap1/test1/195/19' 2024-05-27T09:30:41.943 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/195/20' -> 'copyofsnap1/test1/195/20' 2024-05-27T09:30:41.943 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/196' -> 'copyofsnap1/test1/196' 2024-05-27T09:30:41.943 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/196/10' -> 'copyofsnap1/test1/196/10' 2024-05-27T09:30:41.943 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/196/11' -> 'copyofsnap1/test1/196/11' 2024-05-27T09:30:41.943 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/196/12' -> 'copyofsnap1/test1/196/12' 2024-05-27T09:30:41.943 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/196/13' -> 'copyofsnap1/test1/196/13' 2024-05-27T09:30:41.943 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/196/14' -> 'copyofsnap1/test1/196/14' 2024-05-27T09:30:41.943 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/196/15' -> 'copyofsnap1/test1/196/15' 2024-05-27T09:30:41.943 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/196/16' -> 'copyofsnap1/test1/196/16' 2024-05-27T09:30:41.943 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/196/17' -> 'copyofsnap1/test1/196/17' 2024-05-27T09:30:41.943 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/196/18' -> 'copyofsnap1/test1/196/18' 2024-05-27T09:30:41.943 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/196/19' -> 'copyofsnap1/test1/196/19' 2024-05-27T09:30:41.944 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/196/20' -> 'copyofsnap1/test1/196/20' 2024-05-27T09:30:41.944 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/197' -> 'copyofsnap1/test1/197' 2024-05-27T09:30:41.944 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/197/10' -> 'copyofsnap1/test1/197/10' 2024-05-27T09:30:41.944 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/197/11' -> 'copyofsnap1/test1/197/11' 2024-05-27T09:30:41.944 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/197/12' -> 'copyofsnap1/test1/197/12' 2024-05-27T09:30:41.944 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/197/13' -> 'copyofsnap1/test1/197/13' 2024-05-27T09:30:41.944 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/197/14' -> 'copyofsnap1/test1/197/14' 2024-05-27T09:30:41.944 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/197/15' -> 'copyofsnap1/test1/197/15' 2024-05-27T09:30:41.944 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/197/16' -> 'copyofsnap1/test1/197/16' 2024-05-27T09:30:41.944 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/197/17' -> 'copyofsnap1/test1/197/17' 2024-05-27T09:30:41.944 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/197/18' -> 'copyofsnap1/test1/197/18' 2024-05-27T09:30:41.944 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/197/19' -> 'copyofsnap1/test1/197/19' 2024-05-27T09:30:41.944 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/197/20' -> 'copyofsnap1/test1/197/20' 2024-05-27T09:30:41.945 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/198' -> 'copyofsnap1/test1/198' 2024-05-27T09:30:41.945 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/198/10' -> 'copyofsnap1/test1/198/10' 2024-05-27T09:30:41.945 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/198/11' -> 'copyofsnap1/test1/198/11' 2024-05-27T09:30:41.945 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/198/12' -> 'copyofsnap1/test1/198/12' 2024-05-27T09:30:41.945 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/198/13' -> 'copyofsnap1/test1/198/13' 2024-05-27T09:30:41.945 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/198/14' -> 'copyofsnap1/test1/198/14' 2024-05-27T09:30:41.945 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/198/15' -> 'copyofsnap1/test1/198/15' 2024-05-27T09:30:41.945 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/198/16' -> 'copyofsnap1/test1/198/16' 2024-05-27T09:30:41.945 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/198/17' -> 'copyofsnap1/test1/198/17' 2024-05-27T09:30:41.945 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/198/18' -> 'copyofsnap1/test1/198/18' 2024-05-27T09:30:41.945 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/198/19' -> 'copyofsnap1/test1/198/19' 2024-05-27T09:30:41.945 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/198/20' -> 'copyofsnap1/test1/198/20' 2024-05-27T09:30:41.945 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/199' -> 'copyofsnap1/test1/199' 2024-05-27T09:30:41.945 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/199/10' -> 'copyofsnap1/test1/199/10' 2024-05-27T09:30:41.946 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/199/11' -> 'copyofsnap1/test1/199/11' 2024-05-27T09:30:41.946 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/199/12' -> 'copyofsnap1/test1/199/12' 2024-05-27T09:30:41.946 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/199/13' -> 'copyofsnap1/test1/199/13' 2024-05-27T09:30:41.946 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/199/14' -> 'copyofsnap1/test1/199/14' 2024-05-27T09:30:41.946 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/199/15' -> 'copyofsnap1/test1/199/15' 2024-05-27T09:30:41.946 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/199/16' -> 'copyofsnap1/test1/199/16' 2024-05-27T09:30:41.946 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/199/17' -> 'copyofsnap1/test1/199/17' 2024-05-27T09:30:41.946 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/199/18' -> 'copyofsnap1/test1/199/18' 2024-05-27T09:30:41.946 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/199/19' -> 'copyofsnap1/test1/199/19' 2024-05-27T09:30:41.946 INFO:tasks.workunit.client.0.smithi114.stdout:'.snap/test1/199/20' -> 'copyofsnap1/test1/199/20' 2024-05-27T09:30:41.947 INFO:tasks.workunit.client.0.smithi114.stdout:Take third snapshot .snap/test3 2024-05-27T09:30:42.216 INFO:tasks.workunit.client.0.smithi114.stdout:Delete the snapshots... 2024-05-27T09:30:43.302 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:30:43.314 INFO:tasks.workunit.client.0.smithi114.stdout:.//.snap/test1 2024-05-27T09:30:43.314 INFO:tasks.workunit.client.0.smithi114.stdout:.//.snap/test2 2024-05-27T09:30:43.314 INFO:tasks.workunit.client.0.smithi114.stdout:.//.snap/test3 2024-05-27T09:30:43.327 INFO:tasks.workunit.client.0.smithi114.stdout:./274/.snap/snap-subdir-test 2024-05-27T09:30:43.338 INFO:tasks.workunit.client.0.smithi114.stdout:./166/.snap/snap-subdir-test 2024-05-27T09:30:43.349 INFO:tasks.workunit.client.0.smithi114.stdout:./256/.snap/snap-subdir-test 2024-05-27T09:30:43.360 INFO:tasks.workunit.client.0.smithi114.stdout:./247/.snap/snap-subdir-test 2024-05-27T09:30:43.374 INFO:tasks.workunit.client.0.smithi114.stdout:./248/.snap/snap-subdir-test 2024-05-27T09:30:43.381 INFO:tasks.workunit.client.0.smithi114.stdout:./147/.snap/snap-subdir-test 2024-05-27T09:30:43.387 INFO:tasks.workunit.client.0.smithi114.stdout:./151/.snap/snap-subdir-test 2024-05-27T09:30:43.394 INFO:tasks.workunit.client.0.smithi114.stdout:./170/.snap/snap-subdir-test 2024-05-27T09:30:43.404 INFO:tasks.workunit.client.0.smithi114.stdout:./167/.snap/snap-subdir-test 2024-05-27T09:30:43.412 INFO:tasks.workunit.client.0.smithi114.stdout:./140/.snap/snap-subdir-test 2024-05-27T09:30:43.420 INFO:tasks.workunit.client.0.smithi114.stdout:./142/.snap/snap-subdir-test 2024-05-27T09:30:43.428 INFO:tasks.workunit.client.0.smithi114.stdout:./150/.snap/snap-subdir-test 2024-05-27T09:30:43.439 INFO:tasks.workunit.client.0.smithi114.stdout:./273/.snap/snap-subdir-test 2024-05-27T09:30:43.452 INFO:tasks.workunit.client.0.smithi114.stdout:./269/.snap/snap-subdir-test 2024-05-27T09:30:43.463 INFO:tasks.workunit.client.0.smithi114.stdout:./105/.snap/snap-subdir-test 2024-05-27T09:30:43.477 INFO:tasks.workunit.client.0.smithi114.stdout:./210/.snap/snap-subdir-test 2024-05-27T09:30:43.489 INFO:tasks.workunit.client.0.smithi114.stdout:./265/.snap/snap-subdir-test 2024-05-27T09:30:43.499 INFO:tasks.workunit.client.0.smithi114.stdout:./218/.snap/snap-subdir-test 2024-05-27T09:30:43.515 INFO:tasks.workunit.client.0.smithi114.stdout:./290/.snap/snap-subdir-test 2024-05-27T09:30:43.526 INFO:tasks.workunit.client.0.smithi114.stdout:./149/.snap/snap-subdir-test 2024-05-27T09:30:43.537 INFO:tasks.workunit.client.0.smithi114.stdout:./109/.snap/snap-subdir-test 2024-05-27T09:30:43.549 INFO:tasks.workunit.client.0.smithi114.stdout:./292/.snap/snap-subdir-test 2024-05-27T09:30:43.561 INFO:tasks.workunit.client.0.smithi114.stdout:./242/.snap/snap-subdir-test 2024-05-27T09:30:43.573 INFO:tasks.workunit.client.0.smithi114.stdout:./158/.snap/snap-subdir-test 2024-05-27T09:30:43.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.582+0000 7f6a6be00640 1 Processor -- start 2024-05-27T09:30:43.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.583+0000 7f6a6be00640 1 -- start start 2024-05-27T09:30:43.584 INFO:tasks.workunit.client.0.smithi114.stdout:./243/.snap/snap-subdir-test 2024-05-27T09:30:43.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.585+0000 7f6a6be00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6a6c1547d0 0x7f6a6c0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:43.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.585+0000 7f6a6be00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a6c06fe50 con 0x7f6a6c074840 2024-05-27T09:30:43.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.585+0000 7f6a6be00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a6c0700f0 con 0x7f6a6c0741f0 2024-05-27T09:30:43.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.585+0000 7f6a6be00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f6a6c0703d0 con 0x7f6a6c1547d0 2024-05-27T09:30:43.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.585+0000 7f6a6b400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f6a6c074840 0x7f6a6c164b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55798/0 (socket says 172.21.15.114:55798) 2024-05-27T09:30:43.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.585+0000 7f6a6b400640 1 -- 172.21.15.114:0/1028801416 learned_addr learned my addr 172.21.15.114:0/1028801416 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:30:43.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.585+0000 7f6a6a000640 1 --2- 172.21.15.114:0/1028801416 >> v2:172.21.15.163:3300/0 conn(0x7f6a6c1547d0 0x7f6a6c0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:43.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.586+0000 7f6a69600640 1 -- 172.21.15.114:0/1028801416 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3670530629 0 0) 0x7f6a6c06fe50 con 0x7f6a6c074840 2024-05-27T09:30:43.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.586+0000 7f6a69600640 1 -- 172.21.15.114:0/1028801416 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a50003520 con 0x7f6a6c074840 2024-05-27T09:30:43.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.586+0000 7f6a6a000640 1 -- 172.21.15.114:0/1028801416 >> v1:172.21.15.163:6789/0 conn(0x7f6a6c0741f0 legacy=0x7f6a6c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:43.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.586+0000 7f6a69600640 1 -- 172.21.15.114:0/1028801416 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1316489942 0 0) 0x7f6a6c0700f0 con 0x7f6a6c0741f0 2024-05-27T09:30:43.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.586+0000 7f6a6a000640 1 -- 172.21.15.114:0/1028801416 >> v1:172.21.15.114:6789/0 conn(0x7f6a6c074840 legacy=0x7f6a6c164b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:43.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.586+0000 7f6a6a000640 1 -- 172.21.15.114:0/1028801416 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a6c0731d0 con 0x7f6a6c1547d0 2024-05-27T09:30:43.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.587+0000 7f6a6a000640 1 --2- 172.21.15.114:0/1028801416 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a6c1547d0 0x7f6a6c0c63a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6a580097a0 tx=0x7f6a5802a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82e129d08d7e1990 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:43.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.587+0000 7f6a69600640 1 -- 172.21.15.114:0/1028801416 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a58037070 con 0x7f6a6c1547d0 2024-05-27T09:30:43.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.587+0000 7f6a69600640 1 -- 172.21.15.114:0/1028801416 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a58032640 con 0x7f6a6c1547d0 2024-05-27T09:30:43.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.587+0000 7f6a69600640 1 -- 172.21.15.114:0/1028801416 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a58032a10 con 0x7f6a6c1547d0 2024-05-27T09:30:43.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.588+0000 7f6a6be00640 1 -- 172.21.15.114:0/1028801416 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a6c1547d0 msgr2=0x7f6a6c0c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:43.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.588+0000 7f6a6be00640 1 --2- 172.21.15.114:0/1028801416 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a6c1547d0 0x7f6a6c0c63a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6a580097a0 tx=0x7f6a5802a2c0 comp rx=0 tx=0).stop 2024-05-27T09:30:43.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.588+0000 7f6a6be00640 1 -- 172.21.15.114:0/1028801416 shutdown_connections 2024-05-27T09:30:43.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.588+0000 7f6a6be00640 1 --2- 172.21.15.114:0/1028801416 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a6c1547d0 0x7f6a6c0c63a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:43.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.588+0000 7f6a6be00640 1 -- 172.21.15.114:0/1028801416 >> 172.21.15.114:0/1028801416 conn(0x7f6a6c0ca320 msgr2=0x7f6a6c0ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:43.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.588+0000 7f6a6be00640 1 -- 172.21.15.114:0/1028801416 shutdown_connections 2024-05-27T09:30:43.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.588+0000 7f6a6be00640 1 -- 172.21.15.114:0/1028801416 wait complete. 2024-05-27T09:30:43.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.591+0000 7f6a6be00640 1 Processor -- start 2024-05-27T09:30:43.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.591+0000 7f6a6be00640 1 -- start start 2024-05-27T09:30:43.593 INFO:tasks.workunit.client.0.smithi114.stdout:./106/.snap/snap-subdir-test 2024-05-27T09:30:43.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.592+0000 7f6a6be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a6c0741f0 0x7f6a6c1f1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:43.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.592+0000 7f6a6be00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a6c074840 0x7f6a6c0c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:43.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.592+0000 7f6a6be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a6c1547d0 0x7f6a6c0ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:43.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.592+0000 7f6a6be00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c06f500 con 0x7f6a6c0741f0 2024-05-27T09:30:43.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.592+0000 7f6a6be00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c073790 con 0x7f6a6c1547d0 2024-05-27T09:30:43.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.592+0000 7f6a6be00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f6a6c06f680 con 0x7f6a6c074840 2024-05-27T09:30:43.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.593+0000 7f6a6aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a6c0741f0 0x7f6a6c1f1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:43.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.593+0000 7f6a6aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a6c0741f0 0x7f6a6c1f1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34870/0 (socket says 172.21.15.114:34870) 2024-05-27T09:30:43.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.593+0000 7f6a6aa00640 1 -- 172.21.15.114:0/1788664699 learned_addr learned my addr 172.21.15.114:0/1788664699 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:30:43.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.593+0000 7f6a6b400640 1 --2- 172.21.15.114:0/1788664699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a6c1547d0 0x7f6a6c0ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:43.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.593+0000 7f6a6a000640 1 --2- 172.21.15.114:0/1788664699 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a6c074840 0x7f6a6c0c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:43.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.593+0000 7f6a6b400640 1 -- 172.21.15.114:0/1788664699 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a6c074840 msgr2=0x7f6a6c0c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:43.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.593+0000 7f6a6b400640 1 --2- 172.21.15.114:0/1788664699 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a6c074840 0x7f6a6c0c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:43.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.593+0000 7f6a6b400640 1 -- 172.21.15.114:0/1788664699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a6c0741f0 msgr2=0x7f6a6c1f1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:43.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.593+0000 7f6a6b400640 1 --2- 172.21.15.114:0/1788664699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a6c0741f0 0x7f6a6c1f1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:43.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.593+0000 7f6a6b400640 1 -- 172.21.15.114:0/1788664699 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a6c0c8fd0 con 0x7f6a6c1547d0 2024-05-27T09:30:43.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.594+0000 7f6a6b400640 1 --2- 172.21.15.114:0/1788664699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a6c1547d0 0x7f6a6c0ccf20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400ca70 tx=0x7f6a6400cf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:43.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.594+0000 7f6a63400640 1 -- 172.21.15.114:0/1788664699 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6400ab70 con 0x7f6a6c1547d0 2024-05-27T09:30:43.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.594+0000 7f6a63400640 1 -- 172.21.15.114:0/1788664699 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a6400ad10 con 0x7f6a6c1547d0 2024-05-27T09:30:43.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.594+0000 7f6a63400640 1 -- 172.21.15.114:0/1788664699 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a64012600 con 0x7f6a6c1547d0 2024-05-27T09:30:43.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.594+0000 7f6a6be00640 1 -- 172.21.15.114:0/1788664699 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a6c1551b0 con 0x7f6a6c1547d0 2024-05-27T09:30:43.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.594+0000 7f6a6be00640 1 -- 172.21.15.114:0/1788664699 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a6c0c8290 con 0x7f6a6c1547d0 2024-05-27T09:30:43.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.595+0000 7f6a6be00640 1 -- 172.21.15.114:0/1788664699 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a6c158820 con 0x7f6a6c1547d0 2024-05-27T09:30:43.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.597+0000 7f6a63400640 1 -- 172.21.15.114:0/1788664699 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6a640127a0 con 0x7f6a6c1547d0 2024-05-27T09:30:43.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.598+0000 7f6a63400640 1 --2- 172.21.15.114:0/1788664699 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6a4c076300 0x7f6a4c078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:43.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.598+0000 7f6a63400640 1 -- 172.21.15.114:0/1788664699 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5824+0+0 (secure 0 0 0) 0x7f6a64097b80 con 0x7f6a6c1547d0 2024-05-27T09:30:43.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.598+0000 7f6a6aa00640 1 --2- 172.21.15.114:0/1788664699 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6a4c076300 0x7f6a4c078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:43.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.599+0000 7f6a6aa00640 1 --2- 172.21.15.114:0/1788664699 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6a4c076300 0x7f6a4c078810 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c007fd0 tx=0x7f6a5c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:43.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:43.603+0000 7f6a63400640 1 -- 172.21.15.114:0/1788664699 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6a6c158820 con 0x7f6a6c1547d0 2024-05-27T09:30:43.604 INFO:tasks.workunit.client.0.smithi114.stdout:./250/.snap/snap-subdir-test 2024-05-27T09:30:43.616 INFO:tasks.workunit.client.0.smithi114.stdout:./221/.snap/snap-subdir-test 2024-05-27T09:30:43.626 INFO:tasks.workunit.client.0.smithi114.stdout:./244/.snap/snap-subdir-test 2024-05-27T09:30:43.642 INFO:tasks.workunit.client.0.smithi114.stdout:./258/.snap/snap-subdir-test 2024-05-27T09:30:43.655 INFO:tasks.workunit.client.0.smithi114.stdout:./268/.snap/snap-subdir-test 2024-05-27T09:30:43.666 INFO:tasks.workunit.client.0.smithi114.stdout:./189/.snap/snap-subdir-test 2024-05-27T09:30:43.681 INFO:tasks.workunit.client.0.smithi114.stdout:./285/.snap/snap-subdir-test 2024-05-27T09:30:43.692 INFO:tasks.workunit.client.0.smithi114.stdout:./208/.snap/snap-subdir-test 2024-05-27T09:30:43.706 INFO:tasks.workunit.client.0.smithi114.stdout:./288/.snap/snap-subdir-test 2024-05-27T09:30:43.717 INFO:tasks.workunit.client.0.smithi114.stdout:./209/.snap/snap-subdir-test 2024-05-27T09:30:43.730 INFO:tasks.workunit.client.0.smithi114.stdout:./253/.snap/snap-subdir-test 2024-05-27T09:30:43.741 INFO:tasks.workunit.client.0.smithi114.stdout:./249/.snap/snap-subdir-test 2024-05-27T09:30:43.750 INFO:tasks.workunit.client.0.smithi114.stdout:./133/.snap/snap-subdir-test 2024-05-27T09:30:43.763 INFO:tasks.workunit.client.0.smithi114.stdout:./291/.snap/snap-subdir-test 2024-05-27T09:30:43.773 INFO:tasks.workunit.client.0.smithi114.stdout:./226/.snap/snap-subdir-test 2024-05-27T09:30:43.786 INFO:tasks.workunit.client.0.smithi114.stdout:./173/.snap/snap-subdir-test 2024-05-27T09:30:43.797 INFO:tasks.workunit.client.0.smithi114.stdout:./174/.snap/snap-subdir-test 2024-05-27T09:30:43.810 INFO:tasks.workunit.client.0.smithi114.stdout:./299/.snap/snap-subdir-test 2024-05-27T09:30:43.823 INFO:tasks.workunit.client.0.smithi114.stdout:./262/.snap/snap-subdir-test 2024-05-27T09:30:43.836 INFO:tasks.workunit.client.0.smithi114.stdout:./231/.snap/snap-subdir-test 2024-05-27T09:30:43.849 INFO:tasks.workunit.client.0.smithi114.stdout:./214/.snap/snap-subdir-test 2024-05-27T09:30:43.866 INFO:tasks.workunit.client.0.smithi114.stdout:./281/.snap/snap-subdir-test 2024-05-27T09:30:43.883 INFO:tasks.workunit.client.0.smithi114.stdout:./289/.snap/snap-subdir-test 2024-05-27T09:30:43.900 INFO:tasks.workunit.client.0.smithi114.stdout:./283/.snap/snap-subdir-test 2024-05-27T09:30:43.912 INFO:tasks.workunit.client.0.smithi114.stdout:./232/.snap/snap-subdir-test 2024-05-27T09:30:43.924 INFO:tasks.workunit.client.0.smithi114.stdout:./298/.snap/snap-subdir-test 2024-05-27T09:30:43.938 INFO:tasks.workunit.client.0.smithi114.stdout:./233/.snap/snap-subdir-test 2024-05-27T09:30:43.948 INFO:tasks.workunit.client.0.smithi114.stdout:./155/.snap/snap-subdir-test 2024-05-27T09:30:43.958 INFO:tasks.workunit.client.0.smithi114.stdout:./114/.snap/snap-subdir-test 2024-05-27T09:30:43.970 INFO:tasks.workunit.client.0.smithi114.stdout:./159/.snap/snap-subdir-test 2024-05-27T09:30:43.981 INFO:tasks.workunit.client.0.smithi114.stdout:./187/.snap/snap-subdir-test 2024-05-27T09:30:43.992 INFO:tasks.workunit.client.0.smithi114.stdout:./185/.snap/snap-subdir-test 2024-05-27T09:30:44.004 INFO:tasks.workunit.client.0.smithi114.stdout:./293/.snap/snap-subdir-test 2024-05-27T09:30:44.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.005+0000 7f6a6be00640 1 -- 172.21.15.114:0/1788664699 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a6c15ada0 con 0x7f6a6c1547d0 2024-05-27T09:30:44.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.006+0000 7f6a63400640 1 -- 172.21.15.114:0/1788664699 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6883 (secure 0 0 0) 0x7f6a6c15ada0 con 0x7f6a6c1547d0 2024-05-27T09:30:44.008 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:30:44.008 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4828,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:30:05.919439+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"a","rank":2,"incarnation":13,"state":"up:active","state_seq":47,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4297,4256]},"id":1}]} 2024-05-27T09:30:44.008 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 18 2024-05-27T09:30:44.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.009+0000 7f6a6be00640 1 -- 172.21.15.114:0/1788664699 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6a4c076300 msgr2=0x7f6a4c078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:44.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.009+0000 7f6a6be00640 1 --2- 172.21.15.114:0/1788664699 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6a4c076300 0x7f6a4c078810 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c007fd0 tx=0x7f6a5c009040 comp rx=0 tx=0).stop 2024-05-27T09:30:44.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.009+0000 7f6a6be00640 1 -- 172.21.15.114:0/1788664699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a6c1547d0 msgr2=0x7f6a6c0ccf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:44.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.009+0000 7f6a6be00640 1 --2- 172.21.15.114:0/1788664699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a6c1547d0 0x7f6a6c0ccf20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400ca70 tx=0x7f6a6400cf30 comp rx=0 tx=0).stop 2024-05-27T09:30:44.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.009+0000 7f6a6be00640 1 -- 172.21.15.114:0/1788664699 shutdown_connections 2024-05-27T09:30:44.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.009+0000 7f6a6be00640 1 --2- 172.21.15.114:0/1788664699 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6a4c076300 0x7f6a4c078810 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:44.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.009+0000 7f6a6be00640 1 --2- 172.21.15.114:0/1788664699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a6c1547d0 0x7f6a6c0ccf20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:44.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.009+0000 7f6a6be00640 1 --2- 172.21.15.114:0/1788664699 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a6c074840 0x7f6a6c0c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:44.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.009+0000 7f6a6be00640 1 --2- 172.21.15.114:0/1788664699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a6c0741f0 0x7f6a6c1f1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:44.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.009+0000 7f6a6be00640 1 -- 172.21.15.114:0/1788664699 >> 172.21.15.114:0/1788664699 conn(0x7f6a6c0ca320 msgr2=0x7f6a6c158110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:44.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.009+0000 7f6a6be00640 1 -- 172.21.15.114:0/1788664699 shutdown_connections 2024-05-27T09:30:44.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.010+0000 7f6a6be00640 1 -- 172.21.15.114:0/1788664699 wait complete. 2024-05-27T09:30:44.016 INFO:tasks.workunit.client.0.smithi114.stdout:./228/.snap/snap-subdir-test 2024-05-27T09:30:44.026 INFO:tasks.workunit.client.0.smithi114.stdout:./120/.snap/snap-subdir-test 2024-05-27T09:30:44.034 INFO:tasks.workunit.client.0.smithi114.stdout:./110/.snap/snap-subdir-test 2024-05-27T09:30:44.040 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=2) 2024-05-27T09:30:44.041 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-27T09:30:44.041 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:30:44.044 INFO:tasks.workunit.client.0.smithi114.stdout:./267/.snap/snap-subdir-test 2024-05-27T09:30:44.058 INFO:tasks.workunit.client.0.smithi114.stdout:./202/.snap/snap-subdir-test 2024-05-27T09:30:44.064 INFO:tasks.workunit.client.0.smithi114.stdout:./137/.snap/snap-subdir-test 2024-05-27T09:30:44.071 INFO:tasks.ceph.mds.a:Stopped 2024-05-27T09:30:44.071 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:30:44.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.371+0000 7fa526200640 1 Processor -- start 2024-05-27T09:30:44.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.371+0000 7fa526200640 1 -- start start 2024-05-27T09:30:44.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.374+0000 7fa526200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa520074840 0x7fa5200c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:44.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.374+0000 7fa526200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa52006fe50 con 0x7fa5200741f0 2024-05-27T09:30:44.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.374+0000 7fa526200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5200700f0 con 0x7fa5201547d0 2024-05-27T09:30:44.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.374+0000 7fa526200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa5200703d0 con 0x7fa520074840 2024-05-27T09:30:44.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.374+0000 7fa524e00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fa5200741f0 0x7fa52006edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55806/0 (socket says 172.21.15.114:55806) 2024-05-27T09:30:44.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.374+0000 7fa524e00640 1 -- 172.21.15.114:0/1881052769 learned_addr learned my addr 172.21.15.114:0/1881052769 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:30:44.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.374+0000 7fa525800640 1 --2- 172.21.15.114:0/1881052769 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa520074840 0x7fa5200c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:44.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.374+0000 7fa51f400640 1 -- 172.21.15.114:0/1881052769 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2475561513 0 0) 0x7fa52006fe50 con 0x7fa5200741f0 2024-05-27T09:30:44.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.374+0000 7fa525800640 1 -- 172.21.15.114:0/1881052769 >> v1:172.21.15.163:6789/0 conn(0x7fa5201547d0 legacy=0x7fa520162b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:44.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.374+0000 7fa525800640 1 -- 172.21.15.114:0/1881052769 >> v1:172.21.15.114:6789/0 conn(0x7fa5200741f0 legacy=0x7fa52006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:44.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.374+0000 7fa525800640 1 -- 172.21.15.114:0/1881052769 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5200731d0 con 0x7fa520074840 2024-05-27T09:30:44.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.375+0000 7fa525800640 1 --2- 172.21.15.114:0/1881052769 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa520074840 0x7fa5200c63a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa518009750 tx=0x7fa51802a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ce3578fcf86b0f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:44.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.375+0000 7fa51f400640 1 -- 172.21.15.114:0/1881052769 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518037070 con 0x7fa520074840 2024-05-27T09:30:44.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.375+0000 7fa51f400640 1 -- 172.21.15.114:0/1881052769 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa51802ae40 con 0x7fa520074840 2024-05-27T09:30:44.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.375+0000 7fa51f400640 1 -- 172.21.15.114:0/1881052769 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518032660 con 0x7fa520074840 2024-05-27T09:30:44.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.376+0000 7fa526200640 1 -- 172.21.15.114:0/1881052769 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa520074840 msgr2=0x7fa5200c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:44.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.376+0000 7fa526200640 1 --2- 172.21.15.114:0/1881052769 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa520074840 0x7fa5200c63a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa518009750 tx=0x7fa51802a130 comp rx=0 tx=0).stop 2024-05-27T09:30:44.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.376+0000 7fa526200640 1 -- 172.21.15.114:0/1881052769 shutdown_connections 2024-05-27T09:30:44.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.376+0000 7fa526200640 1 --2- 172.21.15.114:0/1881052769 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa520074840 0x7fa5200c63a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:44.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.376+0000 7fa526200640 1 -- 172.21.15.114:0/1881052769 >> 172.21.15.114:0/1881052769 conn(0x7fa5200ca320 msgr2=0x7fa5200ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:44.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.376+0000 7fa526200640 1 -- 172.21.15.114:0/1881052769 shutdown_connections 2024-05-27T09:30:44.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.376+0000 7fa526200640 1 -- 172.21.15.114:0/1881052769 wait complete. 2024-05-27T09:30:44.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.378+0000 7fa526200640 1 Processor -- start 2024-05-27T09:30:44.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.379+0000 7fa526200640 1 -- start start 2024-05-27T09:30:44.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.380+0000 7fa526200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5200741f0 0x7fa520156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:44.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.380+0000 7fa526200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa520074840 0x7fa52014f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:44.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.380+0000 7fa526200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5201547d0 0x7fa5200c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:44.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.380+0000 7fa526200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa52006f500 con 0x7fa520074840 2024-05-27T09:30:44.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.380+0000 7fa526200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa520073790 con 0x7fa5200741f0 2024-05-27T09:30:44.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.380+0000 7fa526200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa52006f680 con 0x7fa5201547d0 2024-05-27T09:30:44.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.380+0000 7fa51fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa520074840 0x7fa52014f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:44.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.380+0000 7fa524e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5200741f0 0x7fa520156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:44.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.380+0000 7fa51fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa520074840 0x7fa52014f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34882/0 (socket says 172.21.15.114:34882) 2024-05-27T09:30:44.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.380+0000 7fa51fe00640 1 -- 172.21.15.114:0/2308316787 learned_addr learned my addr 172.21.15.114:0/2308316787 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:30:44.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.380+0000 7fa525800640 1 --2- 172.21.15.114:0/2308316787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5201547d0 0x7fa5200c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:44.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.381+0000 7fa51fe00640 1 -- 172.21.15.114:0/2308316787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5201547d0 msgr2=0x7fa5200c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:44.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.381+0000 7fa51fe00640 1 --2- 172.21.15.114:0/2308316787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5201547d0 0x7fa5200c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:44.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.381+0000 7fa51fe00640 1 -- 172.21.15.114:0/2308316787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5200741f0 msgr2=0x7fa520156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:44.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.381+0000 7fa51fe00640 1 --2- 172.21.15.114:0/2308316787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5200741f0 0x7fa520156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:44.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.381+0000 7fa51fe00640 1 -- 172.21.15.114:0/2308316787 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5201551c0 con 0x7fa520074840 2024-05-27T09:30:44.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.381+0000 7fa51fe00640 1 --2- 172.21.15.114:0/2308316787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa520074840 0x7fa52014f1e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa50800c730 tx=0x7fa50800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:44.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.382+0000 7fa51cc00640 1 -- 172.21.15.114:0/2308316787 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508017070 con 0x7fa520074840 2024-05-27T09:30:44.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.382+0000 7fa51cc00640 1 -- 172.21.15.114:0/2308316787 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa508005270 con 0x7fa520074840 2024-05-27T09:30:44.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.382+0000 7fa51cc00640 1 -- 172.21.15.114:0/2308316787 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50800a880 con 0x7fa520074840 2024-05-27T09:30:44.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.382+0000 7fa526200640 1 -- 172.21.15.114:0/2308316787 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52014fb50 con 0x7fa520074840 2024-05-27T09:30:44.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.382+0000 7fa526200640 1 -- 172.21.15.114:0/2308316787 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5201f0690 con 0x7fa520074840 2024-05-27T09:30:44.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.383+0000 7fa526200640 1 -- 172.21.15.114:0/2308316787 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5201585e0 con 0x7fa520074840 2024-05-27T09:30:44.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.384+0000 7fa51cc00640 1 -- 172.21.15.114:0/2308316787 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa50801a070 con 0x7fa520074840 2024-05-27T09:30:44.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.385+0000 7fa51cc00640 1 --2- 172.21.15.114:0/2308316787 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa4fc0762d0 0x7fa4fc0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:44.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.386+0000 7fa51cc00640 1 -- 172.21.15.114:0/2308316787 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5824+0+0 (secure 0 0 0) 0x7fa508096b70 con 0x7fa520074840 2024-05-27T09:30:44.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.386+0000 7fa524e00640 1 --2- 172.21.15.114:0/2308316787 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa4fc0762d0 0x7fa4fc0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:44.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.388+0000 7fa524e00640 1 --2- 172.21.15.114:0/2308316787 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa4fc0762d0 0x7fa4fc0787e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa5100012e0 tx=0x7fa510004d40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:44.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.391+0000 7fa51cc00640 1 -- 172.21.15.114:0/2308316787 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa5201551c0 con 0x7fa520074840 2024-05-27T09:30:44.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.643+0000 7fa526200640 1 -- 172.21.15.114:0/2308316787 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5200b34a0 con 0x7fa520074840 2024-05-27T09:30:44.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.645+0000 7fa51cc00640 1 -- 172.21.15.114:0/2308316787 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6883 (secure 0 0 0) 0x7fa50805fdc0 con 0x7fa520074840 2024-05-27T09:30:44.646 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:30:44.646 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4828,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:30:05.919439+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"a","rank":2,"incarnation":13,"state":"up:active","state_seq":47,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4297,4256]},"id":1}]} 2024-05-27T09:30:44.646 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 18 2024-05-27T09:30:44.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.647+0000 7fa526200640 1 -- 172.21.15.114:0/2308316787 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa4fc0762d0 msgr2=0x7fa4fc0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:44.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.647+0000 7fa526200640 1 --2- 172.21.15.114:0/2308316787 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa4fc0762d0 0x7fa4fc0787e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa5100012e0 tx=0x7fa510004d40 comp rx=0 tx=0).stop 2024-05-27T09:30:44.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.647+0000 7fa526200640 1 -- 172.21.15.114:0/2308316787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa520074840 msgr2=0x7fa52014f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:44.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.647+0000 7fa526200640 1 --2- 172.21.15.114:0/2308316787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa520074840 0x7fa52014f1e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa50800c730 tx=0x7fa50800cbf0 comp rx=0 tx=0).stop 2024-05-27T09:30:44.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.648+0000 7fa526200640 1 -- 172.21.15.114:0/2308316787 shutdown_connections 2024-05-27T09:30:44.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.648+0000 7fa526200640 1 --2- 172.21.15.114:0/2308316787 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa4fc0762d0 0x7fa4fc0787e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:44.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.648+0000 7fa526200640 1 --2- 172.21.15.114:0/2308316787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5201547d0 0x7fa5200c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:44.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.648+0000 7fa526200640 1 --2- 172.21.15.114:0/2308316787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa520074840 0x7fa52014f1e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:44.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.648+0000 7fa526200640 1 --2- 172.21.15.114:0/2308316787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5200741f0 0x7fa520156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:44.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.648+0000 7fa526200640 1 -- 172.21.15.114:0/2308316787 >> 172.21.15.114:0/2308316787 conn(0x7fa5200ca320 msgr2=0x7fa520159960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:44.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.648+0000 7fa526200640 1 -- 172.21.15.114:0/2308316787 shutdown_connections 2024-05-27T09:30:44.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:44.648+0000 7fa526200640 1 -- 172.21.15.114:0/2308316787 wait complete. 2024-05-27T09:30:44.675 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-05-27T09:30:46.676 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:30:46.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.955+0000 7f4612400640 1 Processor -- start 2024-05-27T09:30:46.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.955+0000 7f4612400640 1 -- start start 2024-05-27T09:30:46.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.957+0000 7f4612400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f460c0ca9e0 0x7f460c15dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:46.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.958+0000 7f4612400640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f460c15a810 con 0x7f460c154820 2024-05-27T09:30:46.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.958+0000 7f4612400640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f460c15aaf0 con 0x7f460c074a60 2024-05-27T09:30:46.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.958+0000 7f4612400640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f460c154c30 con 0x7f460c0ca9e0 2024-05-27T09:30:46.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.958+0000 7f4603e00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f460c154820 0x7f460c0ca2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:60724/0 (socket says 172.21.15.114:60724) 2024-05-27T09:30:46.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.958+0000 7f4603e00640 1 -- 172.21.15.114:0/1472894371 learned_addr learned my addr 172.21.15.114:0/1472894371 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:30:46.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.958+0000 7f460be00640 1 --2- 172.21.15.114:0/1472894371 >> v2:172.21.15.114:3300/0 conn(0x7f460c0ca9e0 0x7f460c15dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:46.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.958+0000 7f460aa00640 1 -- 172.21.15.114:0/1472894371 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4087070213 0 0) 0x7f460c15aaf0 con 0x7f460c074a60 2024-05-27T09:30:46.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.958+0000 7f460aa00640 1 -- 172.21.15.114:0/1472894371 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45f0003520 con 0x7f460c074a60 2024-05-27T09:30:46.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.959+0000 7f460aa00640 1 -- 172.21.15.114:0/1472894371 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3564169486 0 0) 0x7f460c15a810 con 0x7f460c154820 2024-05-27T09:30:46.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.959+0000 7f460be00640 1 -- 172.21.15.114:0/1472894371 >> v1:172.21.15.163:6789/0 conn(0x7f460c074a60 legacy=0x7f460c073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:46.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.959+0000 7f460be00640 1 -- 172.21.15.114:0/1472894371 >> v1:172.21.15.114:6789/0 conn(0x7f460c154820 legacy=0x7f460c0ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:46.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.959+0000 7f460be00640 1 -- 172.21.15.114:0/1472894371 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460c15add0 con 0x7f460c0ca9e0 2024-05-27T09:30:46.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.960+0000 7f460be00640 1 --2- 172.21.15.114:0/1472894371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f460c0ca9e0 0x7f460c15dff0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4604009d90 tx=0x7f460402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e97d8b0ebcb5f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:46.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.960+0000 7f460aa00640 1 -- 172.21.15.114:0/1472894371 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604037070 con 0x7f460c0ca9e0 2024-05-27T09:30:46.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.960+0000 7f460aa00640 1 -- 172.21.15.114:0/1472894371 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4604007740 con 0x7f460c0ca9e0 2024-05-27T09:30:46.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.960+0000 7f460aa00640 1 -- 172.21.15.114:0/1472894371 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604007b10 con 0x7f460c0ca9e0 2024-05-27T09:30:46.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.961+0000 7f4612400640 1 -- 172.21.15.114:0/1472894371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f460c0ca9e0 msgr2=0x7f460c15dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:46.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.961+0000 7f4612400640 1 --2- 172.21.15.114:0/1472894371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f460c0ca9e0 0x7f460c15dff0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4604009d90 tx=0x7f460402a780 comp rx=0 tx=0).stop 2024-05-27T09:30:46.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.961+0000 7f4612400640 1 -- 172.21.15.114:0/1472894371 shutdown_connections 2024-05-27T09:30:46.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.961+0000 7f4612400640 1 --2- 172.21.15.114:0/1472894371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f460c0ca9e0 0x7f460c15dff0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:46.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.961+0000 7f4612400640 1 -- 172.21.15.114:0/1472894371 >> 172.21.15.114:0/1472894371 conn(0x7f460c06f6e0 msgr2=0x7f460c070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:46.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.961+0000 7f4612400640 1 -- 172.21.15.114:0/1472894371 shutdown_connections 2024-05-27T09:30:46.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.961+0000 7f4612400640 1 -- 172.21.15.114:0/1472894371 wait complete. 2024-05-27T09:30:46.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.963+0000 7f4612400640 1 Processor -- start 2024-05-27T09:30:46.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.963+0000 7f4612400640 1 -- start start 2024-05-27T09:30:46.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.965+0000 7f4612400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f460c074a60 0x7f460c158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:46.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.965+0000 7f4612400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f460c0ca9e0 0x7f460c0c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:46.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.965+0000 7f4612400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f460c154820 0x7f460c1470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:46.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.965+0000 7f4612400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f460c15b390 con 0x7f460c074a60 2024-05-27T09:30:46.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.965+0000 7f4612400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f460c15db60 con 0x7f460c154820 2024-05-27T09:30:46.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.965+0000 7f4612400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f460c073970 con 0x7f460c0ca9e0 2024-05-27T09:30:46.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.965+0000 7f460b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f460c074a60 0x7f460c158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:46.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.965+0000 7f460b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f460c074a60 0x7f460c158df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50172/0 (socket says 172.21.15.114:50172) 2024-05-27T09:30:46.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.965+0000 7f460b400640 1 -- 172.21.15.114:0/2582676757 learned_addr learned my addr 172.21.15.114:0/2582676757 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:30:46.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.965+0000 7f4603e00640 1 --2- 172.21.15.114:0/2582676757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f460c0ca9e0 0x7f460c0c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:46.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.965+0000 7f460be00640 1 --2- 172.21.15.114:0/2582676757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f460c154820 0x7f460c1470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:46.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.966+0000 7f460b400640 1 -- 172.21.15.114:0/2582676757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f460c0ca9e0 msgr2=0x7f460c0c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:46.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.966+0000 7f460b400640 1 --2- 172.21.15.114:0/2582676757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f460c0ca9e0 0x7f460c0c6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:46.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.966+0000 7f460b400640 1 -- 172.21.15.114:0/2582676757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f460c154820 msgr2=0x7f460c1470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:46.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.966+0000 7f460b400640 1 --2- 172.21.15.114:0/2582676757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f460c154820 0x7f460c1470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:46.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.966+0000 7f460b400640 1 -- 172.21.15.114:0/2582676757 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460c15d160 con 0x7f460c074a60 2024-05-27T09:30:46.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.966+0000 7f460be00640 1 --2- 172.21.15.114:0/2582676757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f460c154820 0x7f460c1470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:30:46.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.966+0000 7f460b400640 1 --2- 172.21.15.114:0/2582676757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f460c074a60 0x7f460c158df0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00c430 tx=0x7f45fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:46.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.966+0000 7f4603400640 1 -- 172.21.15.114:0/2582676757 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45fc017070 con 0x7f460c074a60 2024-05-27T09:30:46.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.966+0000 7f4603400640 1 -- 172.21.15.114:0/2582676757 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45fc0052f0 con 0x7f460c074a60 2024-05-27T09:30:46.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.966+0000 7f4612400640 1 -- 172.21.15.114:0/2582676757 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f460c159b40 con 0x7f460c074a60 2024-05-27T09:30:46.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.966+0000 7f4603400640 1 -- 172.21.15.114:0/2582676757 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45fc003830 con 0x7f460c074a60 2024-05-27T09:30:46.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.966+0000 7f4612400640 1 -- 172.21.15.114:0/2582676757 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f460c1580e0 con 0x7f460c074a60 2024-05-27T09:30:46.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.967+0000 7f4612400640 1 -- 172.21.15.114:0/2582676757 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f460c06a380 con 0x7f460c074a60 2024-05-27T09:30:46.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.969+0000 7f4603400640 1 -- 172.21.15.114:0/2582676757 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f45fc01a070 con 0x7f460c074a60 2024-05-27T09:30:46.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.970+0000 7f4603400640 1 --2- 172.21.15.114:0/2582676757 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f45e80762d0 0x7f45e80787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:46.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.970+0000 7f4603400640 1 -- 172.21.15.114:0/2582676757 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5824+0+0 (secure 0 0 0) 0x7f45fc0968e0 con 0x7f460c074a60 2024-05-27T09:30:46.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.970+0000 7f4603e00640 1 --2- 172.21.15.114:0/2582676757 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f45e80762d0 0x7f45e80787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:46.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.971+0000 7f4603e00640 1 --2- 172.21.15.114:0/2582676757 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f45e80762d0 0x7f45e80787e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f45f4000f60 tx=0x7f45f4007e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:46.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:46.975+0000 7f4603400640 1 -- 172.21.15.114:0/2582676757 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f460c15d160 con 0x7f460c074a60 2024-05-27T09:30:47.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:47.222+0000 7f4612400640 1 -- 172.21.15.114:0/2582676757 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f460c073ca0 con 0x7f460c074a60 2024-05-27T09:30:47.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:47.223+0000 7f4603400640 1 -- 172.21.15.114:0/2582676757 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6883 (secure 0 0 0) 0x7f45fc068250 con 0x7f460c074a60 2024-05-27T09:30:47.224 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:30:47.224 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4828,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:30:05.919439+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"a","rank":2,"incarnation":13,"state":"up:active","state_seq":47,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4297,4256]},"id":1}]} 2024-05-27T09:30:47.224 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 18 2024-05-27T09:30:47.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:47.226+0000 7f4612400640 1 -- 172.21.15.114:0/2582676757 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f45e80762d0 msgr2=0x7f45e80787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:47.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:47.226+0000 7f4612400640 1 --2- 172.21.15.114:0/2582676757 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f45e80762d0 0x7f45e80787e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f45f4000f60 tx=0x7f45f4007e40 comp rx=0 tx=0).stop 2024-05-27T09:30:47.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:47.226+0000 7f4612400640 1 -- 172.21.15.114:0/2582676757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f460c074a60 msgr2=0x7f460c158df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:47.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:47.226+0000 7f4612400640 1 --2- 172.21.15.114:0/2582676757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f460c074a60 0x7f460c158df0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00c430 tx=0x7f45fc00c8f0 comp rx=0 tx=0).stop 2024-05-27T09:30:47.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:47.226+0000 7f4612400640 1 -- 172.21.15.114:0/2582676757 shutdown_connections 2024-05-27T09:30:47.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:47.226+0000 7f4612400640 1 --2- 172.21.15.114:0/2582676757 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f45e80762d0 0x7f45e80787e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:47.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:47.226+0000 7f4612400640 1 --2- 172.21.15.114:0/2582676757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f460c154820 0x7f460c1470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:47.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:47.226+0000 7f4612400640 1 --2- 172.21.15.114:0/2582676757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f460c0ca9e0 0x7f460c0c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:47.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:47.226+0000 7f4612400640 1 --2- 172.21.15.114:0/2582676757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f460c074a60 0x7f460c158df0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:47.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:47.226+0000 7f4612400640 1 -- 172.21.15.114:0/2582676757 >> 172.21.15.114:0/2582676757 conn(0x7f460c06f6e0 msgr2=0x7f460c0ccef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:47.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:47.227+0000 7f4612400640 1 -- 172.21.15.114:0/2582676757 shutdown_connections 2024-05-27T09:30:47.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:47.227+0000 7f4612400640 1 -- 172.21.15.114:0/2582676757 wait complete. 2024-05-27T09:30:47.252 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-05-27T09:30:48.444 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:30:48.452 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:30:49.253 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:30:49.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.537+0000 7f9e08a00640 1 Processor -- start 2024-05-27T09:30:49.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.537+0000 7f9e08a00640 1 -- start start 2024-05-27T09:30:49.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.539+0000 7f9e08a00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9e04156880 0x7f9e0406edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:49.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.539+0000 7f9e08a00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e040c8470 con 0x7f9e04070060 2024-05-27T09:30:49.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.540+0000 7f9e08a00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e040c8dd0 con 0x7f9e04074a80 2024-05-27T09:30:49.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.540+0000 7f9e08a00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f9e0415d480 con 0x7f9e04156880 2024-05-27T09:30:49.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.540+0000 7f9dfbe00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9e04156880 0x7f9e0406edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:49.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.540+0000 7f9e03e00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f9e04070060 0x7f9e04166de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:60740/0 (socket says 172.21.15.114:60740) 2024-05-27T09:30:49.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.540+0000 7f9e03e00640 1 -- 172.21.15.114:0/4255019905 learned_addr learned my addr 172.21.15.114:0/4255019905 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:30:49.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.540+0000 7f9dfbe00640 1 -- 172.21.15.114:0/4255019905 >> v1:172.21.15.163:6789/0 conn(0x7f9e04074a80 legacy=0x7f9e0415cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:49.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.540+0000 7f9dfbe00640 1 -- 172.21.15.114:0/4255019905 >> v1:172.21.15.114:6789/0 conn(0x7f9e04070060 legacy=0x7f9e04166de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:49.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.540+0000 7f9dfbe00640 1 -- 172.21.15.114:0/4255019905 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0415c570 con 0x7f9e04156880 2024-05-27T09:30:49.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.541+0000 7f9dfbe00640 1 --2- 172.21.15.114:0/4255019905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e04156880 0x7f9e0406edf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0097a0 tx=0x7f9dec02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb034a103dddade2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:49.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.542+0000 7f9e02a00640 1 -- 172.21.15.114:0/4255019905 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec037070 con 0x7f9e04156880 2024-05-27T09:30:49.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.542+0000 7f9e02a00640 1 -- 172.21.15.114:0/4255019905 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dec032440 con 0x7f9e04156880 2024-05-27T09:30:49.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.542+0000 7f9e02a00640 1 -- 172.21.15.114:0/4255019905 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec032810 con 0x7f9e04156880 2024-05-27T09:30:49.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.543+0000 7f9e08a00640 1 -- 172.21.15.114:0/4255019905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e04156880 msgr2=0x7f9e0406edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:49.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.543+0000 7f9e08a00640 1 --2- 172.21.15.114:0/4255019905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e04156880 0x7f9e0406edf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0097a0 tx=0x7f9dec02a230 comp rx=0 tx=0).stop 2024-05-27T09:30:49.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.543+0000 7f9e08a00640 1 -- 172.21.15.114:0/4255019905 shutdown_connections 2024-05-27T09:30:49.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.543+0000 7f9e08a00640 1 --2- 172.21.15.114:0/4255019905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e04156880 0x7f9e0406edf0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:49.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.543+0000 7f9e08a00640 1 -- 172.21.15.114:0/4255019905 >> 172.21.15.114:0/4255019905 conn(0x7f9e040ca320 msgr2=0x7f9e040ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:49.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.543+0000 7f9e08a00640 1 -- 172.21.15.114:0/4255019905 shutdown_connections 2024-05-27T09:30:49.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.543+0000 7f9e08a00640 1 -- 172.21.15.114:0/4255019905 wait complete. 2024-05-27T09:30:49.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.546+0000 7f9e08a00640 1 Processor -- start 2024-05-27T09:30:49.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.546+0000 7f9e08a00640 1 -- start start 2024-05-27T09:30:49.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.547+0000 7f9e08a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e04070060 0x7f9e041f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:49.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.547+0000 7f9e08a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e04074a80 0x7f9e0414c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:49.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.547+0000 7f9e08a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e04156880 0x7f9e0414c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:49.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.547+0000 7f9e08a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9e040c8750 con 0x7f9e04070060 2024-05-27T09:30:49.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.547+0000 7f9e08a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9e040c8220 con 0x7f9e04074a80 2024-05-27T09:30:49.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.547+0000 7f9e08a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f9e0415c8e0 con 0x7f9e04156880 2024-05-27T09:30:49.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.548+0000 7f9e03400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e04070060 0x7f9e041f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:49.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.548+0000 7f9e03400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e04070060 0x7f9e041f34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50186/0 (socket says 172.21.15.114:50186) 2024-05-27T09:30:49.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.548+0000 7f9e03400640 1 -- 172.21.15.114:0/4288500600 learned_addr learned my addr 172.21.15.114:0/4288500600 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:30:49.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.548+0000 7f9e03e00640 1 --2- 172.21.15.114:0/4288500600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e04156880 0x7f9e0414c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:49.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.548+0000 7f9dfbe00640 1 --2- 172.21.15.114:0/4288500600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e04074a80 0x7f9e0414c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:49.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.548+0000 7f9dfbe00640 1 -- 172.21.15.114:0/4288500600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e04156880 msgr2=0x7f9e0414c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:49.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.548+0000 7f9dfbe00640 1 --2- 172.21.15.114:0/4288500600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e04156880 0x7f9e0414c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:49.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.548+0000 7f9dfbe00640 1 -- 172.21.15.114:0/4288500600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e04070060 msgr2=0x7f9e041f34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:49.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.548+0000 7f9dfbe00640 1 --2- 172.21.15.114:0/4288500600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e04070060 0x7f9e041f34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:49.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.548+0000 7f9dfbe00640 1 -- 172.21.15.114:0/4288500600 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e041f3a30 con 0x7f9e04074a80 2024-05-27T09:30:49.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.548+0000 7f9e03400640 1 --2- 172.21.15.114:0/4288500600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e04070060 0x7f9e041f34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:30:49.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.548+0000 7f9e03e00640 1 --2- 172.21.15.114:0/4288500600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e04156880 0x7f9e0414c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:30:49.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.549+0000 7f9dfbe00640 1 --2- 172.21.15.114:0/4288500600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e04074a80 0x7f9e0414c270 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9e04074ec0 tx=0x7f9dec0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:49.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.549+0000 7f9dfb400640 1 -- 172.21.15.114:0/4288500600 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec032e60 con 0x7f9e04074a80 2024-05-27T09:30:49.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.549+0000 7f9dfb400640 1 -- 172.21.15.114:0/4288500600 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dec0307b0 con 0x7f9e04074a80 2024-05-27T09:30:49.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.549+0000 7f9e08a00640 1 -- 172.21.15.114:0/4288500600 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e0415f0e0 con 0x7f9e04074a80 2024-05-27T09:30:49.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.549+0000 7f9dfb400640 1 -- 172.21.15.114:0/4288500600 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec037040 con 0x7f9e04074a80 2024-05-27T09:30:49.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.549+0000 7f9e08a00640 1 -- 172.21.15.114:0/4288500600 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e041f3c40 con 0x7f9e04074a80 2024-05-27T09:30:49.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.550+0000 7f9e08a00640 1 -- 172.21.15.114:0/4288500600 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e04050860 con 0x7f9e04074a80 2024-05-27T09:30:49.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.552+0000 7f9dfb400640 1 -- 172.21.15.114:0/4288500600 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9dec044020 con 0x7f9e04074a80 2024-05-27T09:30:49.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.553+0000 7f9dfb400640 1 --2- 172.21.15.114:0/4288500600 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9de00762d0 0x7f9de00787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:49.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.553+0000 7f9dfb400640 1 -- 172.21.15.114:0/4288500600 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5824+0+0 (secure 0 0 0) 0x7f9dec0b7730 con 0x7f9e04074a80 2024-05-27T09:30:49.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.553+0000 7f9e03400640 1 --2- 172.21.15.114:0/4288500600 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9de00762d0 0x7f9de00787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:49.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.554+0000 7f9e03400640 1 --2- 172.21.15.114:0/4288500600 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9de00762d0 0x7f9de00787e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9df4001bd0 tx=0x7f9df4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:49.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.558+0000 7f9dfb400640 1 -- 172.21.15.114:0/4288500600 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9dec080950 con 0x7f9e04074a80 2024-05-27T09:30:49.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.808+0000 7f9e08a00640 1 -- 172.21.15.114:0/4288500600 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e0415eb00 con 0x7f9e04074a80 2024-05-27T09:30:49.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.809+0000 7f9dfb400640 1 -- 172.21.15.114:0/4288500600 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6883 (secure 0 0 0) 0x7f9dec084c50 con 0x7f9e04074a80 2024-05-27T09:30:49.810 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:30:49.810 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4828,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:30:05.919439+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"a","rank":2,"incarnation":13,"state":"up:active","state_seq":47,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4297,4256]},"id":1}]} 2024-05-27T09:30:49.811 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 18 2024-05-27T09:30:49.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.812+0000 7f9e08a00640 1 -- 172.21.15.114:0/4288500600 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9de00762d0 msgr2=0x7f9de00787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:49.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.812+0000 7f9e08a00640 1 --2- 172.21.15.114:0/4288500600 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9de00762d0 0x7f9de00787e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9df4001bd0 tx=0x7f9df4009040 comp rx=0 tx=0).stop 2024-05-27T09:30:49.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.812+0000 7f9e08a00640 1 -- 172.21.15.114:0/4288500600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e04074a80 msgr2=0x7f9e0414c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:49.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.812+0000 7f9e08a00640 1 --2- 172.21.15.114:0/4288500600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e04074a80 0x7f9e0414c270 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9e04074ec0 tx=0x7f9dec0046c0 comp rx=0 tx=0).stop 2024-05-27T09:30:49.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.813+0000 7f9e08a00640 1 -- 172.21.15.114:0/4288500600 shutdown_connections 2024-05-27T09:30:49.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.813+0000 7f9e08a00640 1 --2- 172.21.15.114:0/4288500600 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9de00762d0 0x7f9de00787e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:49.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.813+0000 7f9e08a00640 1 --2- 172.21.15.114:0/4288500600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e04156880 0x7f9e0414c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:49.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.813+0000 7f9e08a00640 1 --2- 172.21.15.114:0/4288500600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e04074a80 0x7f9e0414c270 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:49.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.813+0000 7f9e08a00640 1 --2- 172.21.15.114:0/4288500600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e04070060 0x7f9e041f34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:49.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.813+0000 7f9e08a00640 1 -- 172.21.15.114:0/4288500600 >> 172.21.15.114:0/4288500600 conn(0x7f9e040ca320 msgr2=0x7f9e04073260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:49.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.813+0000 7f9e08a00640 1 -- 172.21.15.114:0/4288500600 shutdown_connections 2024-05-27T09:30:49.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:49.813+0000 7f9e08a00640 1 -- 172.21.15.114:0/4288500600 wait complete. 2024-05-27T09:30:49.839 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-05-27T09:30:51.840 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:30:52.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.125+0000 7f19c3400640 1 Processor -- start 2024-05-27T09:30:52.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.125+0000 7f19c3400640 1 -- start start 2024-05-27T09:30:52.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.128+0000 7f19c3400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f19bc0c8370 0x7f19bc152f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:52.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.128+0000 7f19c3400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19bc0ca400 0x7f19bc0ca810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:52.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.128+0000 7f19c3400640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19bc0c42f0 con 0x7f19bc0c9170 2024-05-27T09:30:52.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.128+0000 7f19c3400640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f19bc0c9b40 con 0x7f19bc0c8370 2024-05-27T09:30:52.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.128+0000 7f19c3400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f19bc0c2b20 con 0x7f19bc0ca400 2024-05-27T09:30:52.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.128+0000 7f19c0a00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f19bc0c9170 0x7f19bc15c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:60742/0 (socket says 172.21.15.114:60742) 2024-05-27T09:30:52.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.128+0000 7f19c0a00640 1 -- 172.21.15.114:0/3132398377 learned_addr learned my addr 172.21.15.114:0/3132398377 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:30:52.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.128+0000 7f19b3e00640 1 --2- 172.21.15.114:0/3132398377 >> v2:172.21.15.163:3300/0 conn(0x7f19bc0c8370 0x7f19bc152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:52.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.128+0000 7f19c1400640 1 --2- 172.21.15.114:0/3132398377 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19bc0ca400 0x7f19bc0ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:52.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.129+0000 7f19b3e00640 1 -- 172.21.15.114:0/3132398377 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19bc0ca400 msgr2=0x7f19bc0ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:30:52.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.129+0000 7f19b3e00640 1 --2- 172.21.15.114:0/3132398377 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19bc0ca400 0x7f19bc0ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:52.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.129+0000 7f19b3e00640 1 -- 172.21.15.114:0/3132398377 >> v1:172.21.15.114:6789/0 conn(0x7f19bc0c9170 legacy=0x7f19bc15c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:52.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.129+0000 7f19bbe00640 1 -- 172.21.15.114:0/3132398377 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 302385527 0 0) 0x7f19bc0c42f0 con 0x7f19bc0c9170 2024-05-27T09:30:52.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.129+0000 7f19b3e00640 1 -- 172.21.15.114:0/3132398377 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19bc0c45d0 con 0x7f19bc0c8370 2024-05-27T09:30:52.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.129+0000 7f19c1400640 1 --2- 172.21.15.114:0/3132398377 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19bc0ca400 0x7f19bc0ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:30:52.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.130+0000 7f19b3e00640 1 --2- 172.21.15.114:0/3132398377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19bc0c8370 0x7f19bc152f00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f19a40097a0 tx=0x7f19a402c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0ff03b9a8a1c2dd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:52.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.130+0000 7f19bbe00640 1 -- 172.21.15.114:0/3132398377 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a4038070 con 0x7f19bc0c8370 2024-05-27T09:30:52.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.130+0000 7f19bbe00640 1 -- 172.21.15.114:0/3132398377 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19a4031710 con 0x7f19bc0c8370 2024-05-27T09:30:52.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.130+0000 7f19bbe00640 1 -- 172.21.15.114:0/3132398377 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a4031ae0 con 0x7f19bc0c8370 2024-05-27T09:30:52.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.131+0000 7f19c3400640 1 -- 172.21.15.114:0/3132398377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19bc0c8370 msgr2=0x7f19bc152f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:52.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.131+0000 7f19c3400640 1 --2- 172.21.15.114:0/3132398377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19bc0c8370 0x7f19bc152f00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f19a40097a0 tx=0x7f19a402c420 comp rx=0 tx=0).stop 2024-05-27T09:30:52.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.131+0000 7f19c3400640 1 -- 172.21.15.114:0/3132398377 shutdown_connections 2024-05-27T09:30:52.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.131+0000 7f19c3400640 1 --2- 172.21.15.114:0/3132398377 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19bc0ca400 0x7f19bc0ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:52.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.131+0000 7f19c3400640 1 --2- 172.21.15.114:0/3132398377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19bc0c8370 0x7f19bc152f00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:52.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.131+0000 7f19c3400640 1 -- 172.21.15.114:0/3132398377 >> 172.21.15.114:0/3132398377 conn(0x7f19bc06f6e0 msgr2=0x7f19bc070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:52.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.131+0000 7f19c3400640 1 -- 172.21.15.114:0/3132398377 shutdown_connections 2024-05-27T09:30:52.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.131+0000 7f19c3400640 1 -- 172.21.15.114:0/3132398377 wait complete. 2024-05-27T09:30:52.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.134+0000 7f19c3400640 1 Processor -- start 2024-05-27T09:30:52.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.134+0000 7f19c3400640 1 -- start start 2024-05-27T09:30:52.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.136+0000 7f19c3400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19bc0c8370 0x7f19bc0ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:52.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.136+0000 7f19c3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19bc0c9170 0x7f19bc14df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:52.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.136+0000 7f19c3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19bc0ca400 0x7f19bc1f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:52.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.136+0000 7f19c3400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f19bc0c9fd0 con 0x7f19bc0c9170 2024-05-27T09:30:52.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.136+0000 7f19c3400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f19bc0c97d0 con 0x7f19bc0ca400 2024-05-27T09:30:52.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.136+0000 7f19c3400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f19bc0ca150 con 0x7f19bc0c8370 2024-05-27T09:30:52.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.136+0000 7f19b3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19bc0c9170 0x7f19bc14df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:52.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.136+0000 7f19c0a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19bc0c8370 0x7f19bc0ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:52.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.136+0000 7f19b3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19bc0c9170 0x7f19bc14df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50200/0 (socket says 172.21.15.114:50200) 2024-05-27T09:30:52.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.136+0000 7f19b3e00640 1 -- 172.21.15.114:0/3451106951 learned_addr learned my addr 172.21.15.114:0/3451106951 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:30:52.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.136+0000 7f19c1400640 1 --2- 172.21.15.114:0/3451106951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19bc0ca400 0x7f19bc1f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:52.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.137+0000 7f19c0a00640 1 -- 172.21.15.114:0/3451106951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19bc0ca400 msgr2=0x7f19bc1f1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:52.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.137+0000 7f19c0a00640 1 --2- 172.21.15.114:0/3451106951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19bc0ca400 0x7f19bc1f1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:52.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.137+0000 7f19c0a00640 1 -- 172.21.15.114:0/3451106951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19bc0c9170 msgr2=0x7f19bc14df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:52.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.137+0000 7f19c0a00640 1 --2- 172.21.15.114:0/3451106951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19bc0c9170 0x7f19bc14df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:52.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.137+0000 7f19c0a00640 1 -- 172.21.15.114:0/3451106951 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19bc0cd1b0 con 0x7f19bc0c8370 2024-05-27T09:30:52.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.137+0000 7f19b3e00640 1 --2- 172.21.15.114:0/3451106951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19bc0c9170 0x7f19bc14df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:30:52.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.137+0000 7f19c0a00640 1 --2- 172.21.15.114:0/3451106951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19bc0c8370 0x7f19bc0ccc70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f19ac001750 tx=0x7f19ac001b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:52.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.137+0000 7f19b9600640 1 -- 172.21.15.114:0/3451106951 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac00e110 con 0x7f19bc0c8370 2024-05-27T09:30:52.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.137+0000 7f19b9600640 1 -- 172.21.15.114:0/3451106951 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19ac0070d0 con 0x7f19bc0c8370 2024-05-27T09:30:52.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.137+0000 7f19c3400640 1 -- 172.21.15.114:0/3451106951 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19bc0c6da0 con 0x7f19bc0c8370 2024-05-27T09:30:52.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.138+0000 7f19b9600640 1 -- 172.21.15.114:0/3451106951 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac00e2d0 con 0x7f19bc0c8370 2024-05-27T09:30:52.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.138+0000 7f19c3400640 1 -- 172.21.15.114:0/3451106951 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f19bc0c7040 con 0x7f19bc0c8370 2024-05-27T09:30:52.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.139+0000 7f19c3400640 1 -- 172.21.15.114:0/3451106951 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19bc06a380 con 0x7f19bc0c8370 2024-05-27T09:30:52.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.141+0000 7f19b9600640 1 -- 172.21.15.114:0/3451106951 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f19ac0074c0 con 0x7f19bc0c8370 2024-05-27T09:30:52.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.142+0000 7f19b9600640 1 --2- 172.21.15.114:0/3451106951 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1998076300 0x7f1998078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:52.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.142+0000 7f19b9600640 1 -- 172.21.15.114:0/3451106951 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 5824+0+0 (secure 0 0 0) 0x7f19ac09afc0 con 0x7f19bc0c8370 2024-05-27T09:30:52.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.142+0000 7f19b3e00640 1 --2- 172.21.15.114:0/3451106951 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1998076300 0x7f1998078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:52.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.143+0000 7f19b3e00640 1 --2- 172.21.15.114:0/3451106951 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1998076300 0x7f1998078810 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f19a40097a0 tx=0x7f19a4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:52.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.148+0000 7f19b9600640 1 -- 172.21.15.114:0/3451106951 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f19ac0648c0 con 0x7f19bc0c8370 2024-05-27T09:30:52.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.393+0000 7f19c3400640 1 -- 172.21.15.114:0/3451106951 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19bc158940 con 0x7f19bc0c8370 2024-05-27T09:30:52.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.394+0000 7f19b9600640 1 -- 172.21.15.114:0/3451106951 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6883 (secure 0 0 0) 0x7f19ac0641c0 con 0x7f19bc0c8370 2024-05-27T09:30:52.395 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:30:52.395 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4828,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:30:05.919439+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"a","rank":2,"incarnation":13,"state":"up:active","state_seq":47,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4297,4256]},"id":1}]} 2024-05-27T09:30:52.395 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 18 2024-05-27T09:30:52.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.397+0000 7f19c3400640 1 -- 172.21.15.114:0/3451106951 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1998076300 msgr2=0x7f1998078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:52.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.397+0000 7f19c3400640 1 --2- 172.21.15.114:0/3451106951 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1998076300 0x7f1998078810 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f19a40097a0 tx=0x7f19a4008030 comp rx=0 tx=0).stop 2024-05-27T09:30:52.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.397+0000 7f19c3400640 1 -- 172.21.15.114:0/3451106951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19bc0c8370 msgr2=0x7f19bc0ccc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:52.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.397+0000 7f19c3400640 1 --2- 172.21.15.114:0/3451106951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19bc0c8370 0x7f19bc0ccc70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f19ac001750 tx=0x7f19ac001b50 comp rx=0 tx=0).stop 2024-05-27T09:30:52.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.397+0000 7f19c3400640 1 -- 172.21.15.114:0/3451106951 shutdown_connections 2024-05-27T09:30:52.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.397+0000 7f19c3400640 1 --2- 172.21.15.114:0/3451106951 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1998076300 0x7f1998078810 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:52.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.397+0000 7f19c3400640 1 --2- 172.21.15.114:0/3451106951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19bc0ca400 0x7f19bc1f1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:52.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.397+0000 7f19c3400640 1 --2- 172.21.15.114:0/3451106951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19bc0c9170 0x7f19bc14df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:52.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.397+0000 7f19c3400640 1 --2- 172.21.15.114:0/3451106951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19bc0c8370 0x7f19bc0ccc70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:52.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.397+0000 7f19c3400640 1 -- 172.21.15.114:0/3451106951 >> 172.21.15.114:0/3451106951 conn(0x7f19bc06f6e0 msgr2=0x7f19bc06c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:52.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.397+0000 7f19c3400640 1 -- 172.21.15.114:0/3451106951 shutdown_connections 2024-05-27T09:30:52.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:52.398+0000 7f19c3400640 1 -- 172.21.15.114:0/3451106951 wait complete. 2024-05-27T09:30:52.426 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-05-27T09:30:54.426 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:30:54.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.714+0000 7f4ea5200640 1 Processor -- start 2024-05-27T09:30:54.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.715+0000 7f4ea5200640 1 -- start start 2024-05-27T09:30:54.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.717+0000 7f4ea5200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea00c8c40 0x7f4ea0154630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:54.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.717+0000 7f4ea5200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4ea006dbd0 0x7f4ea015e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:54.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.717+0000 7f4ea5200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ea00c96f0 con 0x7f4ea006d7c0 2024-05-27T09:30:54.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.717+0000 7f4ea5200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f4ea00c99d0 con 0x7f4ea006dbd0 2024-05-27T09:30:54.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.717+0000 7f4ea5200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea00c9cb0 con 0x7f4ea00c8c40 2024-05-27T09:30:54.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.717+0000 7f4e9e800640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4ea006dbd0 0x7f4ea015e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:54.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.717+0000 7f4e9e800640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4ea006dbd0 0x7f4ea015e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:33528/0 (socket says 172.21.15.114:33528) 2024-05-27T09:30:54.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.717+0000 7f4e9d400640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f4ea006d7c0 0x7f4ea00c7950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:53228/0 (socket says 172.21.15.114:53228) 2024-05-27T09:30:54.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.717+0000 7f4e9e800640 1 -- 172.21.15.114:0/2440251793 learned_addr learned my addr 172.21.15.114:0/2440251793 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:30:54.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.717+0000 7f4e9de00640 1 --2- 172.21.15.114:0/2440251793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea00c8c40 0x7f4ea0154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:54.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.718+0000 7f4e9e800640 1 -- 172.21.15.114:0/2440251793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea00c8c40 msgr2=0x7f4ea0154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:30:54.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.718+0000 7f4e9e800640 1 --2- 172.21.15.114:0/2440251793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea00c8c40 0x7f4ea0154630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:54.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.718+0000 7f4e9e800640 1 -- 172.21.15.114:0/2440251793 >> v1:172.21.15.163:6789/0 conn(0x7f4ea006d7c0 legacy=0x7f4ea00c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:54.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.718+0000 7f4e9e800640 1 -- 172.21.15.114:0/2440251793 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea00c9f90 con 0x7f4ea006dbd0 2024-05-27T09:30:54.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.718+0000 7f4e9ca00640 1 -- 172.21.15.114:0/2440251793 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 683606595 0 0) 0x7f4ea00c96f0 con 0x7f4ea006d7c0 2024-05-27T09:30:54.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.719+0000 7f4e9e800640 1 --2- 172.21.15.114:0/2440251793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ea006dbd0 0x7f4ea015e0c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009870 tx=0x7f4e9802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8c6812cdd7f1eea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:54.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.719+0000 7f4e9ca00640 1 -- 172.21.15.114:0/2440251793 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98037070 con 0x7f4ea006dbd0 2024-05-27T09:30:54.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.719+0000 7f4e9ca00640 1 -- 172.21.15.114:0/2440251793 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e98032440 con 0x7f4ea006dbd0 2024-05-27T09:30:54.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.719+0000 7f4e9ca00640 1 -- 172.21.15.114:0/2440251793 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98032810 con 0x7f4ea006dbd0 2024-05-27T09:30:54.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.720+0000 7f4ea5200640 1 -- 172.21.15.114:0/2440251793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ea006dbd0 msgr2=0x7f4ea015e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:54.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.720+0000 7f4ea5200640 1 --2- 172.21.15.114:0/2440251793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ea006dbd0 0x7f4ea015e0c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009870 tx=0x7f4e9802a250 comp rx=0 tx=0).stop 2024-05-27T09:30:54.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.720+0000 7f4ea5200640 1 -- 172.21.15.114:0/2440251793 shutdown_connections 2024-05-27T09:30:54.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.720+0000 7f4ea5200640 1 --2- 172.21.15.114:0/2440251793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ea006dbd0 0x7f4ea015e0c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:54.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.720+0000 7f4ea5200640 1 --2- 172.21.15.114:0/2440251793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea00c8c40 0x7f4ea0154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:54.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.720+0000 7f4ea5200640 1 -- 172.21.15.114:0/2440251793 >> 172.21.15.114:0/2440251793 conn(0x7f4ea00ce240 msgr2=0x7f4ea00ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:54.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.720+0000 7f4ea5200640 1 -- 172.21.15.114:0/2440251793 shutdown_connections 2024-05-27T09:30:54.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.720+0000 7f4ea5200640 1 -- 172.21.15.114:0/2440251793 wait complete. 2024-05-27T09:30:54.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.723+0000 7f4ea5200640 1 Processor -- start 2024-05-27T09:30:54.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.723+0000 7f4ea5200640 1 -- start start 2024-05-27T09:30:54.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.724+0000 7f4ea5200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea006d7c0 0x7f4ea015c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:54.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.724+0000 7f4ea5200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ea006dbd0 0x7f4ea00cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:54.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.724+0000 7f4ea5200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ea00c8c40 0x7f4ea01f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:54.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.724+0000 7f4ea5200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea0154cf0 con 0x7f4ea006dbd0 2024-05-27T09:30:54.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.724+0000 7f4ea5200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea0154b70 con 0x7f4ea00c8c40 2024-05-27T09:30:54.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.724+0000 7f4ea5200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea00c8060 con 0x7f4ea006d7c0 2024-05-27T09:30:54.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.725+0000 7f4e9de00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea006d7c0 0x7f4ea015c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:54.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.725+0000 7f4e9d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ea006dbd0 0x7f4ea00cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:54.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.725+0000 7f4e9e800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ea00c8c40 0x7f4ea01f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:54.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.725+0000 7f4e9de00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea006d7c0 0x7f4ea015c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:41040/0 (socket says 172.21.15.114:41040) 2024-05-27T09:30:54.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.725+0000 7f4e9de00640 1 -- 172.21.15.114:0/3668542071 learned_addr learned my addr 172.21.15.114:0/3668542071 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:30:54.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.725+0000 7f4e9e800640 1 -- 172.21.15.114:0/3668542071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea006d7c0 msgr2=0x7f4ea015c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:54.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.725+0000 7f4e9e800640 1 --2- 172.21.15.114:0/3668542071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea006d7c0 0x7f4ea015c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:54.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.725+0000 7f4e9e800640 1 -- 172.21.15.114:0/3668542071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ea006dbd0 msgr2=0x7f4ea00cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:54.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.725+0000 7f4e9e800640 1 --2- 172.21.15.114:0/3668542071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ea006dbd0 0x7f4ea00cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:54.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.725+0000 7f4e9e800640 1 -- 172.21.15.114:0/3668542071 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea0144340 con 0x7f4ea00c8c40 2024-05-27T09:30:54.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.725+0000 7f4e9de00640 1 --2- 172.21.15.114:0/3668542071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea006d7c0 0x7f4ea015c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:30:54.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.725+0000 7f4e9e800640 1 --2- 172.21.15.114:0/3668542071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ea00c8c40 0x7f4ea01f5c60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00cf090 tx=0x7f4e980046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:54.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.726+0000 7f4e96000640 1 -- 172.21.15.114:0/3668542071 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98041020 con 0x7f4ea00c8c40 2024-05-27T09:30:54.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.726+0000 7f4e96000640 1 -- 172.21.15.114:0/3668542071 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e9803d6e0 con 0x7f4ea00c8c40 2024-05-27T09:30:54.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.726+0000 7f4e96000640 1 -- 172.21.15.114:0/3668542071 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98037040 con 0x7f4ea00c8c40 2024-05-27T09:30:54.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.726+0000 7f4ea5200640 1 -- 172.21.15.114:0/3668542071 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea015c640 con 0x7f4ea00c8c40 2024-05-27T09:30:54.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.726+0000 7f4ea5200640 1 -- 172.21.15.114:0/3668542071 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea01e5920 con 0x7f4ea00c8c40 2024-05-27T09:30:54.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.727+0000 7f4ea5200640 1 -- 172.21.15.114:0/3668542071 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ea015c810 con 0x7f4ea00c8c40 2024-05-27T09:30:54.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.728+0000 7f4e96000640 1 -- 172.21.15.114:0/3668542071 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4e98044020 con 0x7f4ea00c8c40 2024-05-27T09:30:54.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.729+0000 7f4e96000640 1 --2- 172.21.15.114:0/3668542071 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4e7c0762d0 0x7f4e7c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:54.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.729+0000 7f4e96000640 1 -- 172.21.15.114:0/3668542071 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5824+0+0 (secure 0 0 0) 0x7f4e980b7c60 con 0x7f4ea00c8c40 2024-05-27T09:30:54.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.730+0000 7f4e9de00640 1 --2- 172.21.15.114:0/3668542071 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4e7c0762d0 0x7f4e7c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:54.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.730+0000 7f4e9de00640 1 --2- 172.21.15.114:0/3668542071 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4e7c0762d0 0x7f4e7c0787e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009f50 tx=0x7f4e88009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:54.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.734+0000 7f4e96000640 1 -- 172.21.15.114:0/3668542071 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4e9804b030 con 0x7f4ea00c8c40 2024-05-27T09:30:54.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.981+0000 7f4ea5200640 1 -- 172.21.15.114:0/3668542071 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ea0157fd0 con 0x7f4ea00c8c40 2024-05-27T09:30:54.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.982+0000 7f4e96000640 1 -- 172.21.15.114:0/3668542071 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6883 (secure 0 0 0) 0x7f4e98080df0 con 0x7f4ea00c8c40 2024-05-27T09:30:54.982 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:30:54.982 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4828,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:30:05.919439+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"a","rank":2,"incarnation":13,"state":"up:active","state_seq":47,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4297,4256]},"id":1}]} 2024-05-27T09:30:54.983 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 18 2024-05-27T09:30:54.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.985+0000 7f4ea5200640 1 -- 172.21.15.114:0/3668542071 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4e7c0762d0 msgr2=0x7f4e7c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:54.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.985+0000 7f4ea5200640 1 --2- 172.21.15.114:0/3668542071 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4e7c0762d0 0x7f4e7c0787e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009f50 tx=0x7f4e88009190 comp rx=0 tx=0).stop 2024-05-27T09:30:54.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.985+0000 7f4ea5200640 1 -- 172.21.15.114:0/3668542071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ea00c8c40 msgr2=0x7f4ea01f5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:54.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.985+0000 7f4ea5200640 1 --2- 172.21.15.114:0/3668542071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ea00c8c40 0x7f4ea01f5c60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00cf090 tx=0x7f4e980046c0 comp rx=0 tx=0).stop 2024-05-27T09:30:54.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.985+0000 7f4ea5200640 1 -- 172.21.15.114:0/3668542071 shutdown_connections 2024-05-27T09:30:54.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.985+0000 7f4ea5200640 1 --2- 172.21.15.114:0/3668542071 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4e7c0762d0 0x7f4e7c0787e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:54.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.985+0000 7f4ea5200640 1 --2- 172.21.15.114:0/3668542071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ea00c8c40 0x7f4ea01f5c60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:54.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.985+0000 7f4ea5200640 1 --2- 172.21.15.114:0/3668542071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ea006dbd0 0x7f4ea00cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:54.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.985+0000 7f4ea5200640 1 --2- 172.21.15.114:0/3668542071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea006d7c0 0x7f4ea015c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:54.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.985+0000 7f4ea5200640 1 -- 172.21.15.114:0/3668542071 >> 172.21.15.114:0/3668542071 conn(0x7f4ea00ce240 msgr2=0x7f4ea0147160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:54.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.986+0000 7f4ea5200640 1 -- 172.21.15.114:0/3668542071 shutdown_connections 2024-05-27T09:30:54.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:54.986+0000 7f4ea5200640 1 -- 172.21.15.114:0/3668542071 wait complete. 2024-05-27T09:30:55.012 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-05-27T09:30:57.014 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:30:57.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.296+0000 7f1aeda00640 1 Processor -- start 2024-05-27T09:30:57.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.297+0000 7f1aeda00640 1 -- start start 2024-05-27T09:30:57.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.299+0000 7f1aeda00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1ae80b2c60 0x7f1ae815d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:57.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.299+0000 7f1aeda00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1ae80ca550 0x7f1ae80c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:57.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.299+0000 7f1aeda00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ae8156f80 con 0x7f1ae80ca140 2024-05-27T09:30:57.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.299+0000 7f1aeda00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f1ae815d7c0 con 0x7f1ae80b2c60 2024-05-27T09:30:57.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.299+0000 7f1aeda00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f1ae815fd50 con 0x7f1ae80ca550 2024-05-27T09:30:57.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.299+0000 7f1ae6a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1ae80b2c60 0x7f1ae815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.299+0000 7f1ae6a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1ae80b2c60 0x7f1ae815d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46696/0 (socket says 172.21.15.114:46696) 2024-05-27T09:30:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.299+0000 7f1ae6a00640 1 -- 172.21.15.114:0/1368013608 learned_addr learned my addr 172.21.15.114:0/1368013608 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:30:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.299+0000 7f1ae7400640 1 --2- 172.21.15.114:0/1368013608 >> v2:172.21.15.163:3300/0 conn(0x7f1ae80ca550 0x7f1ae80c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.300+0000 7f1ae6a00640 1 -- 172.21.15.114:0/1368013608 >> v2:172.21.15.163:3300/0 conn(0x7f1ae80ca550 msgr2=0x7f1ae80c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:30:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.300+0000 7f1ae6a00640 1 --2- 172.21.15.114:0/1368013608 >> v2:172.21.15.163:3300/0 conn(0x7f1ae80ca550 0x7f1ae80c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.300+0000 7f1ae6a00640 1 -- 172.21.15.114:0/1368013608 >> v1:172.21.15.163:6789/0 conn(0x7f1ae80ca140 legacy=0x7f1ae8073310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.300+0000 7f1ae6a00640 1 -- 172.21.15.114:0/1368013608 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae80cac50 con 0x7f1ae80b2c60 2024-05-27T09:30:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.300+0000 7f1ae6000640 1 -- 172.21.15.114:0/1368013608 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2061963044 0 0) 0x7f1ae8156f80 con 0x7f1ae80ca140 2024-05-27T09:30:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.301+0000 7f1ae6a00640 1 --2- 172.21.15.114:0/1368013608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ae80b2c60 0x7f1ae815d280 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1ae80b3070 tx=0x7f1ad802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c1af3870a9587c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.301+0000 7f1ae6000640 1 -- 172.21.15.114:0/1368013608 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8037070 con 0x7f1ae80b2c60 2024-05-27T09:30:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.301+0000 7f1ae6000640 1 -- 172.21.15.114:0/1368013608 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ad802ae50 con 0x7f1ae80b2c60 2024-05-27T09:30:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.301+0000 7f1ae6000640 1 -- 172.21.15.114:0/1368013608 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8032860 con 0x7f1ae80b2c60 2024-05-27T09:30:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.302+0000 7f1aeda00640 1 -- 172.21.15.114:0/1368013608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ae80b2c60 msgr2=0x7f1ae815d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.302+0000 7f1aeda00640 1 --2- 172.21.15.114:0/1368013608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ae80b2c60 0x7f1ae815d280 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1ae80b3070 tx=0x7f1ad802a300 comp rx=0 tx=0).stop 2024-05-27T09:30:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.302+0000 7f1aeda00640 1 -- 172.21.15.114:0/1368013608 shutdown_connections 2024-05-27T09:30:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.302+0000 7f1aeda00640 1 --2- 172.21.15.114:0/1368013608 >> v2:172.21.15.163:3300/0 conn(0x7f1ae80ca550 0x7f1ae80c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.302+0000 7f1aeda00640 1 --2- 172.21.15.114:0/1368013608 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ae80b2c60 0x7f1ae815d280 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.302+0000 7f1aeda00640 1 -- 172.21.15.114:0/1368013608 >> 172.21.15.114:0/1368013608 conn(0x7f1ae806f6e0 msgr2=0x7f1ae8070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.302+0000 7f1aeda00640 1 -- 172.21.15.114:0/1368013608 shutdown_connections 2024-05-27T09:30:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.302+0000 7f1aeda00640 1 -- 172.21.15.114:0/1368013608 wait complete. 2024-05-27T09:30:57.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.304+0000 7f1aeda00640 1 Processor -- start 2024-05-27T09:30:57.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.305+0000 7f1aeda00640 1 -- start start 2024-05-27T09:30:57.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.306+0000 7f1aeda00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ae80b2c60 0x7f1ae81f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:57.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.306+0000 7f1aeda00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ae80ca140 0x7f1ae8154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:57.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.306+0000 7f1aeda00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ae80ca550 0x7f1ae806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:57.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.306+0000 7f1aeda00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae8160560 con 0x7f1ae80ca140 2024-05-27T09:30:57.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.306+0000 7f1aeda00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae8160280 con 0x7f1ae80b2c60 2024-05-27T09:30:57.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.306+0000 7f1aeda00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f1ae8160840 con 0x7f1ae80ca550 2024-05-27T09:30:57.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.307+0000 7f1ade000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ae80ca140 0x7f1ae8154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:57.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.307+0000 7f1ade000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ae80ca140 0x7f1ae8154940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46700/0 (socket says 172.21.15.114:46700) 2024-05-27T09:30:57.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.307+0000 7f1ade000640 1 -- 172.21.15.114:0/1087481327 learned_addr learned my addr 172.21.15.114:0/1087481327 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:30:57.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.307+0000 7f1ae6a00640 1 --2- 172.21.15.114:0/1087481327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ae80b2c60 0x7f1ae81f2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:57.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.307+0000 7f1ae7400640 1 --2- 172.21.15.114:0/1087481327 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ae80ca550 0x7f1ae806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:57.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.307+0000 7f1ade000640 1 -- 172.21.15.114:0/1087481327 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ae80ca550 msgr2=0x7f1ae806c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:57.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.307+0000 7f1ade000640 1 --2- 172.21.15.114:0/1087481327 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ae80ca550 0x7f1ae806c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:57.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.307+0000 7f1ade000640 1 -- 172.21.15.114:0/1087481327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ae80b2c60 msgr2=0x7f1ae81f2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:57.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.307+0000 7f1ade000640 1 --2- 172.21.15.114:0/1087481327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ae80b2c60 0x7f1ae81f2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:57.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.307+0000 7f1ade000640 1 -- 172.21.15.114:0/1087481327 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae81f3000 con 0x7f1ae80ca140 2024-05-27T09:30:57.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.307+0000 7f1ae6a00640 1 --2- 172.21.15.114:0/1087481327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ae80b2c60 0x7f1ae81f2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:30:57.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.307+0000 7f1ae7400640 1 --2- 172.21.15.114:0/1087481327 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ae80ca550 0x7f1ae806c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:30:57.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.308+0000 7f1ade000640 1 --2- 172.21.15.114:0/1087481327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ae80ca140 0x7f1ae8154940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000c390 tx=0x7f1ad000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:57.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.308+0000 7f1adf400640 1 -- 172.21.15.114:0/1087481327 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0017070 con 0x7f1ae80ca140 2024-05-27T09:30:57.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.308+0000 7f1adf400640 1 -- 172.21.15.114:0/1087481327 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ad00035f0 con 0x7f1ae80ca140 2024-05-27T09:30:57.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.308+0000 7f1aeda00640 1 -- 172.21.15.114:0/1087481327 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae8153b30 con 0x7f1ae80ca140 2024-05-27T09:30:57.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.308+0000 7f1adf400640 1 -- 172.21.15.114:0/1087481327 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad00039c0 con 0x7f1ae80ca140 2024-05-27T09:30:57.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.308+0000 7f1aeda00640 1 -- 172.21.15.114:0/1087481327 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae81540f0 con 0x7f1ae80ca140 2024-05-27T09:30:57.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.309+0000 7f1aeda00640 1 -- 172.21.15.114:0/1087481327 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae806a380 con 0x7f1ae80ca140 2024-05-27T09:30:57.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.310+0000 7f1adf400640 1 -- 172.21.15.114:0/1087481327 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1ad001a070 con 0x7f1ae80ca140 2024-05-27T09:30:57.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.312+0000 7f1adf400640 1 --2- 172.21.15.114:0/1087481327 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1ac40762d0 0x7f1ac40787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:57.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.312+0000 7f1adf400640 1 -- 172.21.15.114:0/1087481327 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5824+0+0 (secure 0 0 0) 0x7f1ad00968a0 con 0x7f1ae80ca140 2024-05-27T09:30:57.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.312+0000 7f1ae6a00640 1 --2- 172.21.15.114:0/1087481327 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1ac40762d0 0x7f1ac40787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:57.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.313+0000 7f1ae6a00640 1 --2- 172.21.15.114:0/1087481327 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1ac40762d0 0x7f1ac40787e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8034000 tx=0x7f1ad802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:57.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.316+0000 7f1adf400640 1 -- 172.21.15.114:0/1087481327 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1ad005fb50 con 0x7f1ae80ca140 2024-05-27T09:30:57.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.569+0000 7f1aeda00640 1 -- 172.21.15.114:0/1087481327 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ae81f2770 con 0x7f1ae80ca140 2024-05-27T09:30:57.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.570+0000 7f1adf400640 1 -- 172.21.15.114:0/1087481327 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6883 (secure 0 0 0) 0x7f1ad0063e50 con 0x7f1ae80ca140 2024-05-27T09:30:57.570 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:30:57.570 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4828,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:30:05.919439+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"a","rank":2,"incarnation":13,"state":"up:active","state_seq":47,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4297,4256]},"id":1}]} 2024-05-27T09:30:57.571 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 18 2024-05-27T09:30:57.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.572+0000 7f1aeda00640 1 -- 172.21.15.114:0/1087481327 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1ac40762d0 msgr2=0x7f1ac40787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:57.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.572+0000 7f1aeda00640 1 --2- 172.21.15.114:0/1087481327 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1ac40762d0 0x7f1ac40787e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8034000 tx=0x7f1ad802e000 comp rx=0 tx=0).stop 2024-05-27T09:30:57.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.573+0000 7f1aeda00640 1 -- 172.21.15.114:0/1087481327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ae80ca140 msgr2=0x7f1ae8154940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:57.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.573+0000 7f1aeda00640 1 --2- 172.21.15.114:0/1087481327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ae80ca140 0x7f1ae8154940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000c390 tx=0x7f1ad000c850 comp rx=0 tx=0).stop 2024-05-27T09:30:57.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.573+0000 7f1aeda00640 1 -- 172.21.15.114:0/1087481327 shutdown_connections 2024-05-27T09:30:57.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.573+0000 7f1aeda00640 1 --2- 172.21.15.114:0/1087481327 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1ac40762d0 0x7f1ac40787e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:57.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.573+0000 7f1aeda00640 1 --2- 172.21.15.114:0/1087481327 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ae80ca550 0x7f1ae806c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:57.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.573+0000 7f1aeda00640 1 --2- 172.21.15.114:0/1087481327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ae80ca140 0x7f1ae8154940 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:57.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.573+0000 7f1aeda00640 1 --2- 172.21.15.114:0/1087481327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ae80b2c60 0x7f1ae81f2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:57.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.573+0000 7f1aeda00640 1 -- 172.21.15.114:0/1087481327 >> 172.21.15.114:0/1087481327 conn(0x7f1ae806f6e0 msgr2=0x7f1ae815a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:57.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.573+0000 7f1aeda00640 1 -- 172.21.15.114:0/1087481327 shutdown_connections 2024-05-27T09:30:57.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:57.573+0000 7f1aeda00640 1 -- 172.21.15.114:0/1087481327 wait complete. 2024-05-27T09:30:57.598 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-05-27T09:30:59.600 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:30:59.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.884+0000 7f0c47a00640 1 Processor -- start 2024-05-27T09:30:59.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.884+0000 7f0c47a00640 1 -- start start 2024-05-27T09:30:59.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.886+0000 7f0c47a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c40151060 0x7f0c4015af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:59.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.886+0000 7f0c47a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0c400c8ad0 0x7f0c4015dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:59.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.886+0000 7f0c47a00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c4006c690 con 0x7f0c400ca7b0 2024-05-27T09:30:59.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.886+0000 7f0c47a00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f0c4006c930 con 0x7f0c400c8ad0 2024-05-27T09:30:59.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.886+0000 7f0c47a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f0c4006cc10 con 0x7f0c40151060 2024-05-27T09:30:59.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.887+0000 7f0c45a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0c400c8ad0 0x7f0c4015dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:59.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.887+0000 7f0c45a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0c400c8ad0 0x7f0c4015dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46702/0 (socket says 172.21.15.114:46702) 2024-05-27T09:30:59.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.887+0000 7f0c45a00640 1 -- 172.21.15.114:0/1725940859 learned_addr learned my addr 172.21.15.114:0/1725940859 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:30:59.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.887+0000 7f0c3fe00640 1 --2- 172.21.15.114:0/1725940859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c40151060 0x7f0c4015af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:59.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.887+0000 7f0c3f400640 1 -- 172.21.15.114:0/1725940859 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4268347648 0 0) 0x7f0c4006c690 con 0x7f0c400ca7b0 2024-05-27T09:30:59.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.887+0000 7f0c45a00640 1 -- 172.21.15.114:0/1725940859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c40151060 msgr2=0x7f0c4015af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:30:59.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.887+0000 7f0c45a00640 1 --2- 172.21.15.114:0/1725940859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c40151060 0x7f0c4015af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:59.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.887+0000 7f0c45a00640 1 -- 172.21.15.114:0/1725940859 >> v1:172.21.15.163:6789/0 conn(0x7f0c400ca7b0 legacy=0x7f0c400c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:59.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.887+0000 7f0c45a00640 1 -- 172.21.15.114:0/1725940859 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4006d850 con 0x7f0c400c8ad0 2024-05-27T09:30:59.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.888+0000 7f0c45a00640 1 --2- 172.21.15.114:0/1725940859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c400c8ad0 0x7f0c4015dc90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009e50 tx=0x7f0c3802a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8160c80fef6516 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:59.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.889+0000 7f0c3f400640 1 -- 172.21.15.114:0/1725940859 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38036070 con 0x7f0c400c8ad0 2024-05-27T09:30:59.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.889+0000 7f0c3f400640 1 -- 172.21.15.114:0/1725940859 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c38031600 con 0x7f0c400c8ad0 2024-05-27T09:30:59.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.889+0000 7f0c3f400640 1 -- 172.21.15.114:0/1725940859 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c380319f0 con 0x7f0c400c8ad0 2024-05-27T09:30:59.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.890+0000 7f0c47a00640 1 -- 172.21.15.114:0/1725940859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c400c8ad0 msgr2=0x7f0c4015dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:59.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.890+0000 7f0c47a00640 1 --2- 172.21.15.114:0/1725940859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c400c8ad0 0x7f0c4015dc90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009e50 tx=0x7f0c3802a7c0 comp rx=0 tx=0).stop 2024-05-27T09:30:59.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.890+0000 7f0c47a00640 1 -- 172.21.15.114:0/1725940859 shutdown_connections 2024-05-27T09:30:59.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.890+0000 7f0c47a00640 1 --2- 172.21.15.114:0/1725940859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c400c8ad0 0x7f0c4015dc90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:59.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.890+0000 7f0c47a00640 1 --2- 172.21.15.114:0/1725940859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c40151060 0x7f0c4015af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:59.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.890+0000 7f0c47a00640 1 -- 172.21.15.114:0/1725940859 >> 172.21.15.114:0/1725940859 conn(0x7f0c400ce0d0 msgr2=0x7f0c400ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:59.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.890+0000 7f0c47a00640 1 -- 172.21.15.114:0/1725940859 shutdown_connections 2024-05-27T09:30:59.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.890+0000 7f0c47a00640 1 -- 172.21.15.114:0/1725940859 wait complete. 2024-05-27T09:30:59.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.892+0000 7f0c47a00640 1 Processor -- start 2024-05-27T09:30:59.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.892+0000 7f0c47a00640 1 -- start start 2024-05-27T09:30:59.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.894+0000 7f0c47a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c400c8ad0 0x7f0c40151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:59.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.894+0000 7f0c47a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c40151060 0x7f0c401576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:59.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.894+0000 7f0c47a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c40159560 0x7f0c40159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:59.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.894+0000 7f0c47a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0c400c8ee0 con 0x7f0c40151060 2024-05-27T09:30:59.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.894+0000 7f0c47a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4006de10 con 0x7f0c40159560 2024-05-27T09:30:59.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.894+0000 7f0c47a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f0c400c7ef0 con 0x7f0c400c8ad0 2024-05-27T09:30:59.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.894+0000 7f0c3fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c40151060 0x7f0c401576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:59.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.894+0000 7f0c3fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c40151060 0x7f0c401576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46706/0 (socket says 172.21.15.114:46706) 2024-05-27T09:30:59.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.894+0000 7f0c3fe00640 1 -- 172.21.15.114:0/3312727337 learned_addr learned my addr 172.21.15.114:0/3312727337 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:30:59.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.894+0000 7f0c45a00640 1 --2- 172.21.15.114:0/3312727337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c40159560 0x7f0c40159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:59.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.894+0000 7f0c45000640 1 --2- 172.21.15.114:0/3312727337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c400c8ad0 0x7f0c40151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:59.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.895+0000 7f0c3fe00640 1 -- 172.21.15.114:0/3312727337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c400c8ad0 msgr2=0x7f0c40151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:59.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.895+0000 7f0c3fe00640 1 --2- 172.21.15.114:0/3312727337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c400c8ad0 0x7f0c40151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:59.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.895+0000 7f0c3fe00640 1 -- 172.21.15.114:0/3312727337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c40159560 msgr2=0x7f0c40159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:59.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.895+0000 7f0c3fe00640 1 --2- 172.21.15.114:0/3312727337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c40159560 0x7f0c40159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:59.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.895+0000 7f0c3fe00640 1 -- 172.21.15.114:0/3312727337 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c40157c10 con 0x7f0c40151060 2024-05-27T09:30:59.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.895+0000 7f0c45a00640 1 --2- 172.21.15.114:0/3312727337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c40159560 0x7f0c40159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:30:59.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.895+0000 7f0c3fe00640 1 --2- 172.21.15.114:0/3312727337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c40151060 0x7f0c401576d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00e8a0 tx=0x7f0c2c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:59.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.895+0000 7f0c3cc00640 1 -- 172.21.15.114:0/3312727337 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c017070 con 0x7f0c40151060 2024-05-27T09:30:59.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.895+0000 7f0c3cc00640 1 -- 172.21.15.114:0/3312727337 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c2c00c940 con 0x7f0c40151060 2024-05-27T09:30:59.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.895+0000 7f0c47a00640 1 -- 172.21.15.114:0/3312727337 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c40152dc0 con 0x7f0c40151060 2024-05-27T09:30:59.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.895+0000 7f0c3cc00640 1 -- 172.21.15.114:0/3312727337 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c00cd30 con 0x7f0c40151060 2024-05-27T09:30:59.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.895+0000 7f0c47a00640 1 -- 172.21.15.114:0/3312727337 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c40156630 con 0x7f0c40151060 2024-05-27T09:30:59.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.897+0000 7f0c47a00640 1 -- 172.21.15.114:0/3312727337 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c4006a380 con 0x7f0c40151060 2024-05-27T09:30:59.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.898+0000 7f0c3cc00640 1 -- 172.21.15.114:0/3312727337 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0c2c01a070 con 0x7f0c40151060 2024-05-27T09:30:59.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.899+0000 7f0c3cc00640 1 --2- 172.21.15.114:0/3312727337 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0c200762d0 0x7f0c200787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:59.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.899+0000 7f0c3cc00640 1 -- 172.21.15.114:0/3312727337 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5824+0+0 (secure 0 0 0) 0x7f0c2c097460 con 0x7f0c40151060 2024-05-27T09:30:59.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.899+0000 7f0c45000640 1 --2- 172.21.15.114:0/3312727337 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0c200762d0 0x7f0c200787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:59.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.900+0000 7f0c45000640 1 --2- 172.21.15.114:0/3312727337 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0c200762d0 0x7f0c200787e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0c34000f60 tx=0x7f0c34009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:59.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:30:59.903+0000 7f0c3cc00640 1 -- 172.21.15.114:0/3312727337 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0c2c060700 con 0x7f0c40151060 2024-05-27T09:31:00.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:00.157+0000 7f0c47a00640 1 -- 172.21.15.114:0/3312727337 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c4006cf40 con 0x7f0c40151060 2024-05-27T09:31:00.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:00.158+0000 7f0c3cc00640 1 -- 172.21.15.114:0/3312727337 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6883 (secure 0 0 0) 0x7f0c2c064a00 con 0x7f0c40151060 2024-05-27T09:31:00.158 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:31:00.159 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4828,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:30:05.919439+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"a","rank":2,"incarnation":13,"state":"up:active","state_seq":47,"addr":"172.21.15.114:6835/1325619883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":1325619883},{"type":"v1","addr":"172.21.15.114:6835","nonce":1325619883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4297,4256]},"id":1}]} 2024-05-27T09:31:00.159 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 18 2024-05-27T09:31:00.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:00.160+0000 7f0c47a00640 1 -- 172.21.15.114:0/3312727337 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0c200762d0 msgr2=0x7f0c200787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:00.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:00.160+0000 7f0c47a00640 1 --2- 172.21.15.114:0/3312727337 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0c200762d0 0x7f0c200787e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0c34000f60 tx=0x7f0c34009040 comp rx=0 tx=0).stop 2024-05-27T09:31:00.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:00.161+0000 7f0c47a00640 1 -- 172.21.15.114:0/3312727337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c40151060 msgr2=0x7f0c401576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:00.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:00.161+0000 7f0c47a00640 1 --2- 172.21.15.114:0/3312727337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c40151060 0x7f0c401576d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00e8a0 tx=0x7f0c2c00ed60 comp rx=0 tx=0).stop 2024-05-27T09:31:00.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:00.161+0000 7f0c47a00640 1 -- 172.21.15.114:0/3312727337 shutdown_connections 2024-05-27T09:31:00.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:00.161+0000 7f0c47a00640 1 --2- 172.21.15.114:0/3312727337 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0c200762d0 0x7f0c200787e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:00.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:00.161+0000 7f0c47a00640 1 --2- 172.21.15.114:0/3312727337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c40159560 0x7f0c40159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:00.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:00.161+0000 7f0c47a00640 1 --2- 172.21.15.114:0/3312727337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c40151060 0x7f0c401576d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:00.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:00.161+0000 7f0c47a00640 1 --2- 172.21.15.114:0/3312727337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c400c8ad0 0x7f0c40151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:00.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:00.161+0000 7f0c47a00640 1 -- 172.21.15.114:0/3312727337 >> 172.21.15.114:0/3312727337 conn(0x7f0c400ce0d0 msgr2=0x7f0c400cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:00.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:00.161+0000 7f0c47a00640 1 -- 172.21.15.114:0/3312727337 shutdown_connections 2024-05-27T09:31:00.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:00.161+0000 7f0c47a00640 1 -- 172.21.15.114:0/3312727337 wait complete. 2024-05-27T09:31:00.189 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-05-27T09:31:02.190 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:31:02.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.469+0000 7fc0bf200640 1 Processor -- start 2024-05-27T09:31:02.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.470+0000 7fc0bf200640 1 -- start start 2024-05-27T09:31:02.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.472+0000 7fc0bf200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc0b80741f0 0x7fc0b80c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:02.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.472+0000 7fc0bf200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc0b81547d0 0x7fc0b815ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:02.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.472+0000 7fc0bf200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0b8074840 0x7fc0b806edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:02.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.472+0000 7fc0bf200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc0b80703d0 con 0x7fc0b81547d0 2024-05-27T09:31:02.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.472+0000 7fc0bf200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fc0b815b260 con 0x7fc0b80741f0 2024-05-27T09:31:02.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.472+0000 7fc0bf200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fc0b806fe50 con 0x7fc0b8074840 2024-05-27T09:31:02.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.472+0000 7fc0bd400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc0b81547d0 0x7fc0b815ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:02.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.472+0000 7fc0bde00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc0b80741f0 0x7fc0b80c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:02.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.472+0000 7fc0be800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0b8074840 0x7fc0b806edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:02.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.472+0000 7fc0bd400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc0b81547d0 0x7fc0b815ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46710/0 (socket says 172.21.15.114:46710) 2024-05-27T09:31:02.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.472+0000 7fc0bde00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc0b80741f0 0x7fc0b80c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37224/0 (socket says 172.21.15.114:37224) 2024-05-27T09:31:02.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.472+0000 7fc0bd400640 1 -- 172.21.15.114:0/3271486287 learned_addr learned my addr 172.21.15.114:0/3271486287 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:31:02.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.473+0000 7fc0be800640 1 -- 172.21.15.114:0/3271486287 >> v2:172.21.15.163:3300/0 conn(0x7fc0b80741f0 msgr2=0x7fc0b80c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:31:02.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.473+0000 7fc0be800640 1 --2- 172.21.15.114:0/3271486287 >> v2:172.21.15.163:3300/0 conn(0x7fc0b80741f0 0x7fc0b80c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:02.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.473+0000 7fc0be800640 1 -- 172.21.15.114:0/3271486287 >> v2:172.21.15.114:3300/0 conn(0x7fc0b81547d0 msgr2=0x7fc0b815ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:31:02.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.473+0000 7fc0be800640 1 --2- 172.21.15.114:0/3271486287 >> v2:172.21.15.114:3300/0 conn(0x7fc0b81547d0 0x7fc0b815ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:02.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.473+0000 7fc0be800640 1 -- 172.21.15.114:0/3271486287 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b80700f0 con 0x7fc0b8074840 2024-05-27T09:31:02.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.473+0000 7fc0bde00640 1 --2- 172.21.15.114:0/3271486287 >> v2:172.21.15.163:3300/0 conn(0x7fc0b80741f0 0x7fc0b80c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:31:02.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.473+0000 7fc0bd400640 1 --2- 172.21.15.114:0/3271486287 >> v2:172.21.15.114:3300/0 conn(0x7fc0b81547d0 0x7fc0b815ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:31:02.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.474+0000 7fc0be800640 1 --2- 172.21.15.114:0/3271486287 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0b8074840 0x7fc0b806edf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80caf30 tx=0x7fc0b402a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d2aa879028eb58 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:02.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.474+0000 7fc0bca00640 1 -- 172.21.15.114:0/3271486287 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4007740 con 0x7fc0b8074840 2024-05-27T09:31:02.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.474+0000 7fc0bca00640 1 -- 172.21.15.114:0/3271486287 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b40078e0 con 0x7fc0b8074840 2024-05-27T09:31:02.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.474+0000 7fc0bca00640 1 -- 172.21.15.114:0/3271486287 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4007cb0 con 0x7fc0b8074840 2024-05-27T09:31:02.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.475+0000 7fc0bf200640 1 -- 172.21.15.114:0/3271486287 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0b8074840 msgr2=0x7fc0b806edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:02.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.475+0000 7fc0bf200640 1 --2- 172.21.15.114:0/3271486287 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0b8074840 0x7fc0b806edf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80caf30 tx=0x7fc0b402a6a0 comp rx=0 tx=0).stop 2024-05-27T09:31:02.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.475+0000 7fc0bf200640 1 -- 172.21.15.114:0/3271486287 shutdown_connections 2024-05-27T09:31:02.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.475+0000 7fc0bf200640 1 --2- 172.21.15.114:0/3271486287 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0b8074840 0x7fc0b806edf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:02.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.475+0000 7fc0bf200640 1 --2- 172.21.15.114:0/3271486287 >> v2:172.21.15.114:3300/0 conn(0x7fc0b81547d0 0x7fc0b815ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:02.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.475+0000 7fc0bf200640 1 --2- 172.21.15.114:0/3271486287 >> v2:172.21.15.163:3300/0 conn(0x7fc0b80741f0 0x7fc0b80c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:02.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.475+0000 7fc0bf200640 1 -- 172.21.15.114:0/3271486287 >> 172.21.15.114:0/3271486287 conn(0x7fc0b80ca320 msgr2=0x7fc0b80ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:02.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.475+0000 7fc0bf200640 1 -- 172.21.15.114:0/3271486287 shutdown_connections 2024-05-27T09:31:02.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.475+0000 7fc0bf200640 1 -- 172.21.15.114:0/3271486287 wait complete. 2024-05-27T09:31:02.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.478+0000 7fc0bf200640 1 Processor -- start 2024-05-27T09:31:02.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.478+0000 7fc0bf200640 1 -- start start 2024-05-27T09:31:02.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.479+0000 7fc0bf200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0b80741f0 0x7fc0b81f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:02.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.479+0000 7fc0bf200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0b8074840 0x7fc0b815ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:02.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.479+0000 7fc0bf200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0b81547d0 0x7fc0b80cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:02.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.479+0000 7fc0bf200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b8073790 con 0x7fc0b8074840 2024-05-27T09:31:02.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.479+0000 7fc0bf200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b80734b0 con 0x7fc0b81547d0 2024-05-27T09:31:02.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.479+0000 7fc0bf200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fc0b8073cb0 con 0x7fc0b80741f0 2024-05-27T09:31:02.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.480+0000 7fc0bd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0b8074840 0x7fc0b815ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:02.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.480+0000 7fc0bde00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0b80741f0 0x7fc0b81f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:02.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.480+0000 7fc0be800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0b81547d0 0x7fc0b80cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:02.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.480+0000 7fc0bd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0b8074840 0x7fc0b815ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46712/0 (socket says 172.21.15.114:46712) 2024-05-27T09:31:02.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.480+0000 7fc0bde00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0b80741f0 0x7fc0b81f3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:51000/0 (socket says 172.21.15.114:51000) 2024-05-27T09:31:02.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.480+0000 7fc0bd400640 1 -- 172.21.15.114:0/1408815186 learned_addr learned my addr 172.21.15.114:0/1408815186 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:31:02.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.480+0000 7fc0bde00640 1 -- 172.21.15.114:0/1408815186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0b81547d0 msgr2=0x7fc0b80cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:02.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.480+0000 7fc0bde00640 1 --2- 172.21.15.114:0/1408815186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0b81547d0 0x7fc0b80cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:02.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.480+0000 7fc0bde00640 1 -- 172.21.15.114:0/1408815186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0b8074840 msgr2=0x7fc0b815ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:02.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.480+0000 7fc0bde00640 1 --2- 172.21.15.114:0/1408815186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0b8074840 0x7fc0b815ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:02.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.480+0000 7fc0bde00640 1 -- 172.21.15.114:0/1408815186 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b81f5d30 con 0x7fc0b80741f0 2024-05-27T09:31:02.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.480+0000 7fc0bd400640 1 --2- 172.21.15.114:0/1408815186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0b8074840 0x7fc0b815ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:31:02.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.480+0000 7fc0bde00640 1 --2- 172.21.15.114:0/1408815186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0b80741f0 0x7fc0b81f3a90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00e870 tx=0x7fc0ac00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:02.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.481+0000 7fc0b2000640 1 -- 172.21.15.114:0/1408815186 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac004030 con 0x7fc0b80741f0 2024-05-27T09:31:02.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.481+0000 7fc0b2000640 1 -- 172.21.15.114:0/1408815186 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0ac0041d0 con 0x7fc0b80741f0 2024-05-27T09:31:02.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.481+0000 7fc0b2000640 1 -- 172.21.15.114:0/1408815186 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac00c8e0 con 0x7fc0b80741f0 2024-05-27T09:31:02.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.481+0000 7fc0bf200640 1 -- 172.21.15.114:0/1408815186 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b81f6010 con 0x7fc0b80741f0 2024-05-27T09:31:02.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.481+0000 7fc0bf200640 1 -- 172.21.15.114:0/1408815186 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b81f4060 con 0x7fc0b80741f0 2024-05-27T09:31:02.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.482+0000 7fc0bf200640 1 -- 172.21.15.114:0/1408815186 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0b81f42c0 con 0x7fc0b80741f0 2024-05-27T09:31:02.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.483+0000 7fc0b2000640 1 -- 172.21.15.114:0/1408815186 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc0ac00ca80 con 0x7fc0b80741f0 2024-05-27T09:31:02.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.484+0000 7fc0b2000640 1 --2- 172.21.15.114:0/1408815186 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc09c0762d0 0x7fc09c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:02.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.485+0000 7fc0b2000640 1 -- 172.21.15.114:0/1408815186 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 5910+0+0 (secure 0 0 0) 0x7fc0ac0980a0 con 0x7fc0b80741f0 2024-05-27T09:31:02.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.485+0000 7fc0bd400640 1 --2- 172.21.15.114:0/1408815186 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc09c0762d0 0x7fc09c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:02.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.485+0000 7fc0bd400640 1 --2- 172.21.15.114:0/1408815186 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc09c0762d0 0x7fc09c0787e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80112c0 tx=0x7fc0a8009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:02.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.488+0000 7fc0b2000640 1 -- 172.21.15.114:0/1408815186 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc0ac017020 con 0x7fc0b80741f0 2024-05-27T09:31:02.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.840+0000 7fc0bf200640 1 -- 172.21.15.114:0/1408815186 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0b814e8e0 con 0x7fc0b80741f0 2024-05-27T09:31:02.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.841+0000 7fc0b2000640 1 -- 172.21.15.114:0/1408815186 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+6049 (secure 0 0 0) 0x7fc0ac0612c0 con 0x7fc0b80741f0 2024-05-27T09:31:02.843 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:31:02.843 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:31:00.345840+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:replay","state_seq":1,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4297,4300]},"id":1}]} 2024-05-27T09:31:02.843 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 19 2024-05-27T09:31:02.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.844+0000 7fc0bf200640 1 -- 172.21.15.114:0/1408815186 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc09c0762d0 msgr2=0x7fc09c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:02.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.844+0000 7fc0bf200640 1 --2- 172.21.15.114:0/1408815186 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc09c0762d0 0x7fc09c0787e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80112c0 tx=0x7fc0a8009240 comp rx=0 tx=0).stop 2024-05-27T09:31:02.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.844+0000 7fc0bf200640 1 -- 172.21.15.114:0/1408815186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0b80741f0 msgr2=0x7fc0b81f3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:02.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.844+0000 7fc0bf200640 1 --2- 172.21.15.114:0/1408815186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0b80741f0 0x7fc0b81f3a90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00e870 tx=0x7fc0ac00ed30 comp rx=0 tx=0).stop 2024-05-27T09:31:02.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.844+0000 7fc0bf200640 1 -- 172.21.15.114:0/1408815186 shutdown_connections 2024-05-27T09:31:02.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.844+0000 7fc0bf200640 1 --2- 172.21.15.114:0/1408815186 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc09c0762d0 0x7fc09c0787e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:02.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.844+0000 7fc0bf200640 1 --2- 172.21.15.114:0/1408815186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0b81547d0 0x7fc0b80cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:02.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.844+0000 7fc0bf200640 1 --2- 172.21.15.114:0/1408815186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc0b8074840 0x7fc0b815ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:02.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.844+0000 7fc0bf200640 1 --2- 172.21.15.114:0/1408815186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0b80741f0 0x7fc0b81f3a90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:02.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.844+0000 7fc0bf200640 1 -- 172.21.15.114:0/1408815186 >> 172.21.15.114:0/1408815186 conn(0x7fc0b80ca320 msgr2=0x7fc0b814eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:02.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.844+0000 7fc0bf200640 1 -- 172.21.15.114:0/1408815186 shutdown_connections 2024-05-27T09:31:02.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:02.844+0000 7fc0bf200640 1 -- 172.21.15.114:0/1408815186 wait complete. 2024-05-27T09:31:02.870 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2024-05-27T09:31:02.870 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-05-27T09:31:02.870 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:31:03.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.108+0000 7f063f200640 1 Processor -- start 2024-05-27T09:31:03.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.108+0000 7f063f200640 1 -- start start 2024-05-27T09:31:03.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.110+0000 7f063f200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06400741f0 0x7f06400c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:03.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.110+0000 7f063f200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f06401547d0 0x7f064015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:03.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.110+0000 7f063f200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f064006fe50 con 0x7f0640074840 2024-05-27T09:31:03.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.110+0000 7f063f200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f06400703d0 con 0x7f06401547d0 2024-05-27T09:31:03.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.110+0000 7f063f200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f064015b260 con 0x7f06400741f0 2024-05-27T09:31:03.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.111+0000 7f063d400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f06401547d0 0x7f064015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:03.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.111+0000 7f063de00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06400741f0 0x7f06400c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:03.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.111+0000 7f063e800640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f0640074840 0x7f064006edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:40770/0 (socket says 172.21.15.114:40770) 2024-05-27T09:31:03.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.111+0000 7f063de00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06400741f0 0x7f06400c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:51004/0 (socket says 172.21.15.114:51004) 2024-05-27T09:31:03.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.111+0000 7f063e800640 1 -- 172.21.15.114:0/2429734650 learned_addr learned my addr 172.21.15.114:0/2429734650 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:31:03.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.111+0000 7f063d400640 1 -- 172.21.15.114:0/2429734650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06400741f0 msgr2=0x7f06400c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:31:03.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.111+0000 7f063d400640 1 --2- 172.21.15.114:0/2429734650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06400741f0 0x7f06400c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:03.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.111+0000 7f063d400640 1 -- 172.21.15.114:0/2429734650 >> v1:172.21.15.163:6789/0 conn(0x7f0640074840 legacy=0x7f064006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:03.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.111+0000 7f063d400640 1 -- 172.21.15.114:0/2429734650 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06400700f0 con 0x7f06401547d0 2024-05-27T09:31:03.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.112+0000 7f063d400640 1 --2- 172.21.15.114:0/2429734650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06401547d0 0x7f064015ad20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f062c0097a0 tx=0x7f062c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcf28923fa5227d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:03.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.112+0000 7f063ca00640 1 -- 172.21.15.114:0/2429734650 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c037070 con 0x7f06401547d0 2024-05-27T09:31:03.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.112+0000 7f063ca00640 1 -- 172.21.15.114:0/2429734650 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f062c032440 con 0x7f06401547d0 2024-05-27T09:31:03.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.112+0000 7f063ca00640 1 -- 172.21.15.114:0/2429734650 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c032810 con 0x7f06401547d0 2024-05-27T09:31:03.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.113+0000 7f063f200640 1 -- 172.21.15.114:0/2429734650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06401547d0 msgr2=0x7f064015ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:03.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.113+0000 7f063f200640 1 --2- 172.21.15.114:0/2429734650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06401547d0 0x7f064015ad20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f062c0097a0 tx=0x7f062c02a210 comp rx=0 tx=0).stop 2024-05-27T09:31:03.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.113+0000 7f063f200640 1 -- 172.21.15.114:0/2429734650 shutdown_connections 2024-05-27T09:31:03.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.113+0000 7f063f200640 1 --2- 172.21.15.114:0/2429734650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06401547d0 0x7f064015ad20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:03.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.113+0000 7f063f200640 1 --2- 172.21.15.114:0/2429734650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06400741f0 0x7f06400c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:03.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.113+0000 7f063f200640 1 -- 172.21.15.114:0/2429734650 >> 172.21.15.114:0/2429734650 conn(0x7f06400ca320 msgr2=0x7f06400ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:03.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.113+0000 7f063f200640 1 -- 172.21.15.114:0/2429734650 shutdown_connections 2024-05-27T09:31:03.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.113+0000 7f063f200640 1 -- 172.21.15.114:0/2429734650 wait complete. 2024-05-27T09:31:03.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.115+0000 7f063f200640 1 Processor -- start 2024-05-27T09:31:03.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.116+0000 7f063f200640 1 -- start start 2024-05-27T09:31:03.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.117+0000 7f063f200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06400741f0 0x7f06400c3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:03.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.117+0000 7f063f200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0640074840 0x7f0640147500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:03.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.117+0000 7f063f200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06401547d0 0x7f06401e7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:03.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.117+0000 7f063f200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0640073790 con 0x7f06400741f0 2024-05-27T09:31:03.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.117+0000 7f063f200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f06400734b0 con 0x7f06401547d0 2024-05-27T09:31:03.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.117+0000 7f063f200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f064006f500 con 0x7f0640074840 2024-05-27T09:31:03.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.117+0000 7f063de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06400741f0 0x7f06400c3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:03.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.117+0000 7f063de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06400741f0 0x7f06400c3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46738/0 (socket says 172.21.15.114:46738) 2024-05-27T09:31:03.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.117+0000 7f063de00640 1 -- 172.21.15.114:0/1706971036 learned_addr learned my addr 172.21.15.114:0/1706971036 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:31:03.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.117+0000 7f063d400640 1 --2- 172.21.15.114:0/1706971036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0640074840 0x7f0640147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:03.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.117+0000 7f063e800640 1 --2- 172.21.15.114:0/1706971036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06401547d0 0x7f06401e7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:03.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.118+0000 7f063de00640 1 -- 172.21.15.114:0/1706971036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0640074840 msgr2=0x7f0640147500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:03.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.118+0000 7f063de00640 1 --2- 172.21.15.114:0/1706971036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0640074840 0x7f0640147500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:03.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.118+0000 7f063de00640 1 -- 172.21.15.114:0/1706971036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06401547d0 msgr2=0x7f06401e7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:03.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.118+0000 7f063de00640 1 --2- 172.21.15.114:0/1706971036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06401547d0 0x7f06401e7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:03.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.118+0000 7f063de00640 1 -- 172.21.15.114:0/1706971036 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06401554a0 con 0x7f06400741f0 2024-05-27T09:31:03.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.118+0000 7f063de00640 1 --2- 172.21.15.114:0/1706971036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06400741f0 0x7f06400c3320 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f063000e870 tx=0x7f063000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:03.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.118+0000 7f0636000640 1 -- 172.21.15.114:0/1706971036 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630017070 con 0x7f06400741f0 2024-05-27T09:31:03.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.118+0000 7f0636000640 1 -- 172.21.15.114:0/1706971036 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f063000c940 con 0x7f06400741f0 2024-05-27T09:31:03.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.118+0000 7f063f200640 1 -- 172.21.15.114:0/1706971036 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0640155780 con 0x7f06400741f0 2024-05-27T09:31:03.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.118+0000 7f0636000640 1 -- 172.21.15.114:0/1706971036 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063000cd30 con 0x7f06400741f0 2024-05-27T09:31:03.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.118+0000 7f063f200640 1 -- 172.21.15.114:0/1706971036 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06400c2b20 con 0x7f06400741f0 2024-05-27T09:31:03.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.119+0000 7f063f200640 1 -- 172.21.15.114:0/1706971036 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06400b3910 con 0x7f06400741f0 2024-05-27T09:31:03.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.121+0000 7f0636000640 1 -- 172.21.15.114:0/1706971036 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f063001a070 con 0x7f06400741f0 2024-05-27T09:31:03.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.122+0000 7f0636000640 1 --2- 172.21.15.114:0/1706971036 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0620076300 0x7f0620078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:03.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.122+0000 7f0636000640 1 -- 172.21.15.114:0/1706971036 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5910+0+0 (secure 0 0 0) 0x7f0630097810 con 0x7f06400741f0 2024-05-27T09:31:03.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.123+0000 7f063d400640 1 --2- 172.21.15.114:0/1706971036 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0620076300 0x7f0620078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:03.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.123+0000 7f063d400640 1 --2- 172.21.15.114:0/1706971036 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0620076300 0x7f0620078810 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f062c034000 tx=0x7f062c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:03.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.128+0000 7f0636000640 1 -- 172.21.15.114:0/1706971036 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0630060a40 con 0x7f06400741f0 2024-05-27T09:31:03.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.355+0000 7f063f200640 1 -- 172.21.15.114:0/1706971036 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06400c65c0 con 0x7f06400741f0 2024-05-27T09:31:03.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.360+0000 7f0636000640 1 -- 172.21.15.114:0/1706971036 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+6049 (secure 0 0 0) 0x7f0630064d40 con 0x7f06400741f0 2024-05-27T09:31:03.362 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:31:03.362 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:31:00.345840+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:replay","state_seq":1,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4297,4300]},"id":1}]} 2024-05-27T09:31:03.362 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 19 2024-05-27T09:31:03.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.363+0000 7f063f200640 1 -- 172.21.15.114:0/1706971036 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0620076300 msgr2=0x7f0620078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:03.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.363+0000 7f063f200640 1 --2- 172.21.15.114:0/1706971036 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0620076300 0x7f0620078810 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f062c034000 tx=0x7f062c02e000 comp rx=0 tx=0).stop 2024-05-27T09:31:03.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.363+0000 7f063f200640 1 -- 172.21.15.114:0/1706971036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06400741f0 msgr2=0x7f06400c3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:03.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.363+0000 7f063f200640 1 --2- 172.21.15.114:0/1706971036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06400741f0 0x7f06400c3320 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f063000e870 tx=0x7f063000ed30 comp rx=0 tx=0).stop 2024-05-27T09:31:03.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.363+0000 7f063f200640 1 -- 172.21.15.114:0/1706971036 shutdown_connections 2024-05-27T09:31:03.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.363+0000 7f063f200640 1 --2- 172.21.15.114:0/1706971036 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0620076300 0x7f0620078810 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:03.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.363+0000 7f063f200640 1 --2- 172.21.15.114:0/1706971036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06401547d0 0x7f06401e7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:03.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.363+0000 7f063f200640 1 --2- 172.21.15.114:0/1706971036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0640074840 0x7f0640147500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:03.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.363+0000 7f063f200640 1 --2- 172.21.15.114:0/1706971036 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06400741f0 0x7f06400c3320 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:03.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.363+0000 7f063f200640 1 -- 172.21.15.114:0/1706971036 >> 172.21.15.114:0/1706971036 conn(0x7f06400ca320 msgr2=0x7f06400cb6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:03.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.364+0000 7f063f200640 1 -- 172.21.15.114:0/1706971036 shutdown_connections 2024-05-27T09:31:03.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:03.364+0000 7f063f200640 1 -- 172.21.15.114:0/1706971036 wait complete. 2024-05-27T09:31:03.390 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 19, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-05-27T09:27:18.079214+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 19, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4297": { "addr": "172.21.15.114:6834/657027519", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6832", "nonce": 657027519, "type": "v2" }, { "addr": "172.21.15.114:6834", "nonce": 657027519, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4297, "incarnation": 9, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4300": { "addr": "172.21.15.114:6837/1448950017", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6836", "nonce": 1448950017, "type": "v2" }, { "addr": "172.21.15.114:6837", "nonce": 1448950017, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4300, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 4 }, "gid_4828": { "addr": "172.21.15.163:6839/3371890103", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6837", "nonce": 3371890103, "type": "v2" }, { "addr": "172.21.15.163:6839", "nonce": 3371890103, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4828, "incarnation": 19, "join_fscid": -1, "name": "f", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-05-27T09:31:00.345840+0000", "qdb_cluster": [ 4297, 4300 ], "qdb_leader": 4300, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4300, "mds_1": 4297, "mds_2": 4828 } } } ], "standbys": [ { "addr": "172.21.15.163:6835/3513522318", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 3513522318, "type": "v2" }, { "addr": "172.21.15.163:6835", "nonce": 3513522318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6838/1753596112", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6836", "nonce": 1753596112, "type": "v2" }, { "addr": "172.21.15.163:6838", "nonce": 1753596112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-05-27T09:31:05.392 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:31:05.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.670+0000 7f226dc00640 1 Processor -- start 2024-05-27T09:31:05.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.670+0000 7f226dc00640 1 -- start start 2024-05-27T09:31:05.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.672+0000 7f226dc00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2268074140 0x7f2268074550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:05.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.672+0000 7f226dc00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22680c96d0 con 0x7f22680c2a00 2024-05-27T09:31:05.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.672+0000 7f226dc00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22680c8930 con 0x7f22681534c0 2024-05-27T09:31:05.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.672+0000 7f226dc00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f2268159190 con 0x7f2268074140 2024-05-27T09:31:05.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.673+0000 7f2267e00640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f22681534c0 0x7f22680c8220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:40140/0 (socket says 172.21.15.114:40140) 2024-05-27T09:31:05.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.673+0000 7f2267e00640 1 -- 172.21.15.114:0/1643990238 learned_addr learned my addr 172.21.15.114:0/1643990238 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:31:05.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.673+0000 7f225f400640 1 --1- 172.21.15.114:0/1643990238 >> v1:172.21.15.114:6789/0 conn(0x7f22680c2a00 0x7f2268159900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:45504/0 (socket says 172.21.15.114:45504) 2024-05-27T09:31:05.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.673+0000 7f226d200640 1 --2- 172.21.15.114:0/1643990238 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2268074140 0x7f2268074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:05.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.673+0000 7f2267400640 1 -- 172.21.15.114:0/1643990238 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 301283523 0 0) 0x7f22680c8930 con 0x7f22681534c0 2024-05-27T09:31:05.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.673+0000 7f2267400640 1 -- 172.21.15.114:0/1643990238 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f224c003520 con 0x7f22681534c0 2024-05-27T09:31:05.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.673+0000 7f2267400640 1 -- 172.21.15.114:0/1643990238 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1522927856 0 0) 0x7f22680c96d0 con 0x7f22680c2a00 2024-05-27T09:31:05.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.673+0000 7f2267400640 1 -- 172.21.15.114:0/1643990238 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22680c8930 con 0x7f22680c2a00 2024-05-27T09:31:05.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.673+0000 7f226d200640 1 -- 172.21.15.114:0/1643990238 >> v1:172.21.15.163:6789/0 conn(0x7f22681534c0 legacy=0x7f22680c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:05.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.673+0000 7f226d200640 1 -- 172.21.15.114:0/1643990238 >> v1:172.21.15.114:6789/0 conn(0x7f22680c2a00 legacy=0x7f2268159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:05.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.673+0000 7f226d200640 1 -- 172.21.15.114:0/1643990238 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22680c9970 con 0x7f2268074140 2024-05-27T09:31:05.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.674+0000 7f226d200640 1 --2- 172.21.15.114:0/1643990238 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2268074140 0x7f2268074550 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f22680c8b00 tx=0x7f226002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a00d808b2cbcc66 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:05.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.675+0000 7f2267400640 1 -- 172.21.15.114:0/1643990238 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2260007740 con 0x7f2268074140 2024-05-27T09:31:05.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.675+0000 7f2267400640 1 -- 172.21.15.114:0/1643990238 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22600078e0 con 0x7f2268074140 2024-05-27T09:31:05.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.675+0000 7f2267400640 1 -- 172.21.15.114:0/1643990238 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2260007cb0 con 0x7f2268074140 2024-05-27T09:31:05.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.675+0000 7f226dc00640 1 -- 172.21.15.114:0/1643990238 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2268074140 msgr2=0x7f2268074550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:05.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.675+0000 7f226dc00640 1 --2- 172.21.15.114:0/1643990238 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2268074140 0x7f2268074550 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f22680c8b00 tx=0x7f226002a6d0 comp rx=0 tx=0).stop 2024-05-27T09:31:05.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.675+0000 7f226dc00640 1 -- 172.21.15.114:0/1643990238 shutdown_connections 2024-05-27T09:31:05.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.675+0000 7f226dc00640 1 --2- 172.21.15.114:0/1643990238 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2268074140 0x7f2268074550 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:05.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.675+0000 7f226dc00640 1 -- 172.21.15.114:0/1643990238 >> 172.21.15.114:0/1643990238 conn(0x7f22680ca320 msgr2=0x7f22680ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:05.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.676+0000 7f226dc00640 1 -- 172.21.15.114:0/1643990238 shutdown_connections 2024-05-27T09:31:05.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.676+0000 7f226dc00640 1 -- 172.21.15.114:0/1643990238 wait complete. 2024-05-27T09:31:05.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.678+0000 7f226dc00640 1 Processor -- start 2024-05-27T09:31:05.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.678+0000 7f226dc00640 1 -- start start 2024-05-27T09:31:05.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.679+0000 7f226dc00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2268074140 0x7f22681e2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:05.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.679+0000 7f226dc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22680c2a00 0x7f22681f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:05.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.680+0000 7f226dc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22681534c0 0x7f22681e42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:05.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.680+0000 7f226dc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f22680c9f30 con 0x7f22680c2a00 2024-05-27T09:31:05.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.680+0000 7f226dc00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f22681538d0 con 0x7f22681534c0 2024-05-27T09:31:05.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.680+0000 7f226dc00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f22681593e0 con 0x7f2268074140 2024-05-27T09:31:05.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.680+0000 7f225f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22680c2a00 0x7f22681f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:05.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.680+0000 7f225f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22680c2a00 0x7f22681f1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37646/0 (socket says 172.21.15.114:37646) 2024-05-27T09:31:05.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.680+0000 7f225f400640 1 -- 172.21.15.114:0/3632367125 learned_addr learned my addr 172.21.15.114:0/3632367125 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:31:05.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.680+0000 7f2267e00640 1 --2- 172.21.15.114:0/3632367125 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2268074140 0x7f22681e2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:05.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.680+0000 7f226d200640 1 --2- 172.21.15.114:0/3632367125 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22681534c0 0x7f22681e42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:05.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.680+0000 7f225f400640 1 -- 172.21.15.114:0/3632367125 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2268074140 msgr2=0x7f22681e2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:05.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.680+0000 7f225f400640 1 --2- 172.21.15.114:0/3632367125 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2268074140 0x7f22681e2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:05.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.680+0000 7f225f400640 1 -- 172.21.15.114:0/3632367125 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22681534c0 msgr2=0x7f22681e42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:05.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.680+0000 7f225f400640 1 --2- 172.21.15.114:0/3632367125 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22681534c0 0x7f22681e42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:05.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.680+0000 7f225f400640 1 -- 172.21.15.114:0/3632367125 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22681e35a0 con 0x7f22680c2a00 2024-05-27T09:31:05.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.680+0000 7f226d200640 1 --2- 172.21.15.114:0/3632367125 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22681534c0 0x7f22681e42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:31:05.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.681+0000 7f225f400640 1 --2- 172.21.15.114:0/3632367125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22680c2a00 0x7f22681f1d40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f225000c430 tx=0x7f225000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:05.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.681+0000 7f2264c00640 1 -- 172.21.15.114:0/3632367125 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22500074e0 con 0x7f22680c2a00 2024-05-27T09:31:05.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.681+0000 7f2264c00640 1 -- 172.21.15.114:0/3632367125 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22500047e0 con 0x7f22680c2a00 2024-05-27T09:31:05.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.681+0000 7f226dc00640 1 -- 172.21.15.114:0/3632367125 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22681e4830 con 0x7f22680c2a00 2024-05-27T09:31:05.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.681+0000 7f2264c00640 1 -- 172.21.15.114:0/3632367125 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2250004bd0 con 0x7f22680c2a00 2024-05-27T09:31:05.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.681+0000 7f226dc00640 1 -- 172.21.15.114:0/3632367125 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22681e3af0 con 0x7f22680c2a00 2024-05-27T09:31:05.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.682+0000 7f226dc00640 1 -- 172.21.15.114:0/3632367125 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2268050860 con 0x7f22680c2a00 2024-05-27T09:31:05.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.683+0000 7f2264c00640 1 -- 172.21.15.114:0/3632367125 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f22500123f0 con 0x7f22680c2a00 2024-05-27T09:31:05.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.684+0000 7f2264c00640 1 --2- 172.21.15.114:0/3632367125 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f22440762d0 0x7f22440787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:05.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.685+0000 7f2264c00640 1 -- 172.21.15.114:0/3632367125 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5910+0+0 (secure 0 0 0) 0x7f2250096bd0 con 0x7f22680c2a00 2024-05-27T09:31:05.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.685+0000 7f2267e00640 1 --2- 172.21.15.114:0/3632367125 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f22440762d0 0x7f22440787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:05.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.685+0000 7f2267e00640 1 --2- 172.21.15.114:0/3632367125 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f22440762d0 0x7f22440787e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2258005880 tx=0x7f2258006aa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:05.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.690+0000 7f2264c00640 1 -- 172.21.15.114:0/3632367125 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f225005fd70 con 0x7f22680c2a00 2024-05-27T09:31:05.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.928+0000 7f226dc00640 1 -- 172.21.15.114:0/3632367125 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22681e2fc0 con 0x7f22680c2a00 2024-05-27T09:31:05.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.929+0000 7f2264c00640 1 -- 172.21.15.114:0/3632367125 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+6051 (secure 0 0 0) 0x7f22500683b0 con 0x7f22680c2a00 2024-05-27T09:31:05.931 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:31:05.931 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:31:05.359513+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:resolve","state_seq":14,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4297]},"id":1}]} 2024-05-27T09:31:05.931 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 20 2024-05-27T09:31:05.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.932+0000 7f226dc00640 1 -- 172.21.15.114:0/3632367125 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f22440762d0 msgr2=0x7f22440787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:05.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.932+0000 7f226dc00640 1 --2- 172.21.15.114:0/3632367125 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f22440762d0 0x7f22440787e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2258005880 tx=0x7f2258006aa0 comp rx=0 tx=0).stop 2024-05-27T09:31:05.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.932+0000 7f226dc00640 1 -- 172.21.15.114:0/3632367125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22680c2a00 msgr2=0x7f22681f1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:05.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.932+0000 7f226dc00640 1 --2- 172.21.15.114:0/3632367125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22680c2a00 0x7f22681f1d40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f225000c430 tx=0x7f225000c8f0 comp rx=0 tx=0).stop 2024-05-27T09:31:05.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.932+0000 7f226dc00640 1 -- 172.21.15.114:0/3632367125 shutdown_connections 2024-05-27T09:31:05.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.932+0000 7f226dc00640 1 --2- 172.21.15.114:0/3632367125 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f22440762d0 0x7f22440787e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:05.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.933+0000 7f226dc00640 1 --2- 172.21.15.114:0/3632367125 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22681534c0 0x7f22681e42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:05.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.933+0000 7f226dc00640 1 --2- 172.21.15.114:0/3632367125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f22680c2a00 0x7f22681f1d40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:05.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.933+0000 7f226dc00640 1 --2- 172.21.15.114:0/3632367125 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2268074140 0x7f22681e2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:05.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.933+0000 7f226dc00640 1 -- 172.21.15.114:0/3632367125 >> 172.21.15.114:0/3632367125 conn(0x7f22680ca320 msgr2=0x7f22680c7510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:05.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.933+0000 7f226dc00640 1 -- 172.21.15.114:0/3632367125 shutdown_connections 2024-05-27T09:31:05.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:05.933+0000 7f226dc00640 1 -- 172.21.15.114:0/3632367125 wait complete. 2024-05-27T09:31:05.961 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:31:07.963 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:31:08.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.233+0000 7f9eade00640 1 Processor -- start 2024-05-27T09:31:08.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.234+0000 7f9eade00640 1 -- start start 2024-05-27T09:31:08.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.236+0000 7f9eade00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9ea8164db0 0x7f9ea80c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:08.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.236+0000 7f9eade00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9ea8074860 0x7f9ea815aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:08.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.236+0000 7f9eade00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ea815b000 con 0x7f9ea8160b40 2024-05-27T09:31:08.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.236+0000 7f9eade00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f9ea80cade0 con 0x7f9ea8164db0 2024-05-27T09:31:08.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.236+0000 7f9eade00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f9ea80c3d60 con 0x7f9ea8074860 2024-05-27T09:31:08.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.236+0000 7f9ea6a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9ea8164db0 0x7f9ea80c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:08.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.236+0000 7f9ea6a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9ea8164db0 0x7f9ea80c3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37650/0 (socket says 172.21.15.114:37650) 2024-05-27T09:31:08.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.236+0000 7f9ea6a00640 1 -- 172.21.15.114:0/3770317189 learned_addr learned my addr 172.21.15.114:0/3770317189 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:31:08.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.236+0000 7f9ea6000640 1 --2- 172.21.15.114:0/3770317189 >> v2:172.21.15.163:3300/0 conn(0x7f9ea8074860 0x7f9ea815aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:08.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.237+0000 7f9ea6a00640 1 -- 172.21.15.114:0/3770317189 >> v2:172.21.15.163:3300/0 conn(0x7f9ea8074860 msgr2=0x7f9ea815aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:31:08.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.237+0000 7f9ea6a00640 1 --2- 172.21.15.114:0/3770317189 >> v2:172.21.15.163:3300/0 conn(0x7f9ea8074860 0x7f9ea815aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:08.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.237+0000 7f9ea6a00640 1 -- 172.21.15.114:0/3770317189 >> v1:172.21.15.163:6789/0 conn(0x7f9ea8160b40 legacy=0x7f9ea8073310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:08.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.237+0000 7f9ea6a00640 1 -- 172.21.15.114:0/3770317189 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea81615d0 con 0x7f9ea8164db0 2024-05-27T09:31:08.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.237+0000 7f9ea5600640 1 -- 172.21.15.114:0/3770317189 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2846875705 0 0) 0x7f9ea815b000 con 0x7f9ea8160b40 2024-05-27T09:31:08.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.238+0000 7f9ea6a00640 1 --2- 172.21.15.114:0/3770317189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ea8164db0 0x7f9ea80c3820 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8071e10 tx=0x7f9e9802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb4bc23e8868628 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:08.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.239+0000 7f9ea5600640 1 -- 172.21.15.114:0/3770317189 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e98037070 con 0x7f9ea8164db0 2024-05-27T09:31:08.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.239+0000 7f9ea5600640 1 -- 172.21.15.114:0/3770317189 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e9802ae50 con 0x7f9ea8164db0 2024-05-27T09:31:08.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.239+0000 7f9ea5600640 1 -- 172.21.15.114:0/3770317189 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e98032860 con 0x7f9ea8164db0 2024-05-27T09:31:08.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.239+0000 7f9eade00640 1 -- 172.21.15.114:0/3770317189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ea8164db0 msgr2=0x7f9ea80c3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:08.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.239+0000 7f9eade00640 1 --2- 172.21.15.114:0/3770317189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ea8164db0 0x7f9ea80c3820 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8071e10 tx=0x7f9e9802a300 comp rx=0 tx=0).stop 2024-05-27T09:31:08.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.239+0000 7f9eade00640 1 -- 172.21.15.114:0/3770317189 shutdown_connections 2024-05-27T09:31:08.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.239+0000 7f9eade00640 1 --2- 172.21.15.114:0/3770317189 >> v2:172.21.15.163:3300/0 conn(0x7f9ea8074860 0x7f9ea815aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:08.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.239+0000 7f9eade00640 1 --2- 172.21.15.114:0/3770317189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ea8164db0 0x7f9ea80c3820 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:08.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.239+0000 7f9eade00640 1 -- 172.21.15.114:0/3770317189 >> 172.21.15.114:0/3770317189 conn(0x7f9ea806f6e0 msgr2=0x7f9ea8070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:08.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.240+0000 7f9eade00640 1 -- 172.21.15.114:0/3770317189 shutdown_connections 2024-05-27T09:31:08.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.240+0000 7f9eade00640 1 -- 172.21.15.114:0/3770317189 wait complete. 2024-05-27T09:31:08.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.242+0000 7f9eade00640 1 Processor -- start 2024-05-27T09:31:08.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.242+0000 7f9eade00640 1 -- start start 2024-05-27T09:31:08.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.244+0000 7f9eade00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ea8074860 0x7f9ea8157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:08.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.244+0000 7f9eade00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea8160b40 0x7f9ea814cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:08.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.244+0000 7f9eade00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8164db0 0x7f9ea81ed050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:08.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.244+0000 7f9eade00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea81645b0 con 0x7f9ea8074860 2024-05-27T09:31:08.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.244+0000 7f9eade00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea81642d0 con 0x7f9ea8164db0 2024-05-27T09:31:08.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.244+0000 7f9eade00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f9ea8164890 con 0x7f9ea8160b40 2024-05-27T09:31:08.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.244+0000 7f9ea6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ea8074860 0x7f9ea8157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:08.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.244+0000 7f9ea6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ea8074860 0x7f9ea8157cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37666/0 (socket says 172.21.15.114:37666) 2024-05-27T09:31:08.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.244+0000 7f9ea6a00640 1 -- 172.21.15.114:0/177338624 learned_addr learned my addr 172.21.15.114:0/177338624 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:31:08.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.244+0000 7f9ea6000640 1 --2- 172.21.15.114:0/177338624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea8160b40 0x7f9ea814cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:08.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.244+0000 7f9ea7400640 1 --2- 172.21.15.114:0/177338624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8164db0 0x7f9ea81ed050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:08.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.244+0000 7f9ea6a00640 1 -- 172.21.15.114:0/177338624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea8160b40 msgr2=0x7f9ea814cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:08.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.244+0000 7f9ea6a00640 1 --2- 172.21.15.114:0/177338624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea8160b40 0x7f9ea814cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:08.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.244+0000 7f9ea6a00640 1 -- 172.21.15.114:0/177338624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8164db0 msgr2=0x7f9ea81ed050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:08.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.244+0000 7f9ea6a00640 1 --2- 172.21.15.114:0/177338624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8164db0 0x7f9ea81ed050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:08.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.244+0000 7f9ea6a00640 1 -- 172.21.15.114:0/177338624 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea8158a60 con 0x7f9ea8074860 2024-05-27T09:31:08.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.245+0000 7f9ea6a00640 1 --2- 172.21.15.114:0/177338624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ea8074860 0x7f9ea8157cf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9e98000c00 tx=0x7f9e980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:08.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.245+0000 7f9e9f400640 1 -- 172.21.15.114:0/177338624 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e98041070 con 0x7f9ea8074860 2024-05-27T09:31:08.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.245+0000 7f9e9f400640 1 -- 172.21.15.114:0/177338624 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e9803c660 con 0x7f9ea8074860 2024-05-27T09:31:08.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.245+0000 7f9eade00640 1 -- 172.21.15.114:0/177338624 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea80cbc30 con 0x7f9ea8074860 2024-05-27T09:31:08.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.245+0000 7f9e9f400640 1 -- 172.21.15.114:0/177338624 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e98037040 con 0x7f9ea8074860 2024-05-27T09:31:08.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.245+0000 7f9eade00640 1 -- 172.21.15.114:0/177338624 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea8158780 con 0x7f9ea8074860 2024-05-27T09:31:08.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.246+0000 7f9eade00640 1 -- 172.21.15.114:0/177338624 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea806a380 con 0x7f9ea8074860 2024-05-27T09:31:08.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.248+0000 7f9e9f400640 1 -- 172.21.15.114:0/177338624 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9e98044070 con 0x7f9ea8074860 2024-05-27T09:31:08.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.249+0000 7f9e9f400640 1 --2- 172.21.15.114:0/177338624 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9e840762d0 0x7f9e840787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:08.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.249+0000 7f9e9f400640 1 -- 172.21.15.114:0/177338624 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5910+0+0 (secure 0 0 0) 0x7f9e980b7ce0 con 0x7f9ea8074860 2024-05-27T09:31:08.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.249+0000 7f9ea6000640 1 --2- 172.21.15.114:0/177338624 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9e840762d0 0x7f9e840787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:08.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.250+0000 7f9ea6000640 1 --2- 172.21.15.114:0/177338624 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9e840762d0 0x7f9e840787e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8159520 tx=0x7f9e90009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:08.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.253+0000 7f9e9f400640 1 -- 172.21.15.114:0/177338624 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9e98080e80 con 0x7f9ea8074860 2024-05-27T09:31:08.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.496+0000 7f9eade00640 1 -- 172.21.15.114:0/177338624 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ea8156670 con 0x7f9ea8074860 2024-05-27T09:31:08.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.497+0000 7f9e9f400640 1 -- 172.21.15.114:0/177338624 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+6051 (secure 0 0 0) 0x7f9e98085180 con 0x7f9ea8074860 2024-05-27T09:31:08.498 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:31:08.499 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:31:05.359513+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:resolve","state_seq":14,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4297]},"id":1}]} 2024-05-27T09:31:08.499 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 20 2024-05-27T09:31:08.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.500+0000 7f9eade00640 1 -- 172.21.15.114:0/177338624 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9e840762d0 msgr2=0x7f9e840787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:08.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.500+0000 7f9eade00640 1 --2- 172.21.15.114:0/177338624 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9e840762d0 0x7f9e840787e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8159520 tx=0x7f9e90009190 comp rx=0 tx=0).stop 2024-05-27T09:31:08.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.500+0000 7f9eade00640 1 -- 172.21.15.114:0/177338624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ea8074860 msgr2=0x7f9ea8157cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:08.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.500+0000 7f9eade00640 1 --2- 172.21.15.114:0/177338624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ea8074860 0x7f9ea8157cf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9e98000c00 tx=0x7f9e980046c0 comp rx=0 tx=0).stop 2024-05-27T09:31:08.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.501+0000 7f9eade00640 1 -- 172.21.15.114:0/177338624 shutdown_connections 2024-05-27T09:31:08.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.501+0000 7f9eade00640 1 --2- 172.21.15.114:0/177338624 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9e840762d0 0x7f9e840787e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:08.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.501+0000 7f9eade00640 1 --2- 172.21.15.114:0/177338624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8164db0 0x7f9ea81ed050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:08.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.501+0000 7f9eade00640 1 --2- 172.21.15.114:0/177338624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea8160b40 0x7f9ea814cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:08.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.501+0000 7f9eade00640 1 --2- 172.21.15.114:0/177338624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ea8074860 0x7f9ea8157cf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:08.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.501+0000 7f9eade00640 1 -- 172.21.15.114:0/177338624 >> 172.21.15.114:0/177338624 conn(0x7f9ea806f6e0 msgr2=0x7f9ea80732b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:08.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.501+0000 7f9eade00640 1 -- 172.21.15.114:0/177338624 shutdown_connections 2024-05-27T09:31:08.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:08.501+0000 7f9eade00640 1 -- 172.21.15.114:0/177338624 wait complete. 2024-05-27T09:31:08.529 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:31:10.530 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:31:10.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.809+0000 7fd5abc00640 1 Processor -- start 2024-05-27T09:31:10.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.809+0000 7fd5abc00640 1 -- start start 2024-05-27T09:31:10.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.812+0000 7fd5abc00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd5a4156ad0 0x7fd5a406d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:10.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.812+0000 7fd5abc00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5a40cb330 0x7fd5a40cb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:10.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.812+0000 7fd5abc00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5a40ca650 con 0x7fd5a40c3aa0 2024-05-27T09:31:10.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.812+0000 7fd5abc00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fd5a406d890 con 0x7fd5a4156ad0 2024-05-27T09:31:10.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.812+0000 7fd5abc00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fd5a406cf60 con 0x7fd5a40cb330 2024-05-27T09:31:10.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.813+0000 7fd5a9200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd5a4156ad0 0x7fd5a406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:10.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.813+0000 7fd5a9c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5a40cb330 0x7fd5a40cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:10.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.813+0000 7fd5a3e00640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fd5a40c3aa0 0x7fd5a415fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:40164/0 (socket says 172.21.15.114:40164) 2024-05-27T09:31:10.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.813+0000 7fd5a3e00640 1 -- 172.21.15.114:0/2180870812 learned_addr learned my addr 172.21.15.114:0/2180870812 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:31:10.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.813+0000 7fd5a9200640 1 -- 172.21.15.114:0/2180870812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5a40cb330 msgr2=0x7fd5a40cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:31:10.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.813+0000 7fd5a9200640 1 --2- 172.21.15.114:0/2180870812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5a40cb330 0x7fd5a40cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:10.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.813+0000 7fd5a9200640 1 -- 172.21.15.114:0/2180870812 >> v1:172.21.15.163:6789/0 conn(0x7fd5a40c3aa0 legacy=0x7fd5a415fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:10.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.813+0000 7fd5a9200640 1 -- 172.21.15.114:0/2180870812 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a40ca930 con 0x7fd5a4156ad0 2024-05-27T09:31:10.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.813+0000 7fd5a3400640 1 -- 172.21.15.114:0/2180870812 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2976086624 0 0) 0x7fd5a40ca650 con 0x7fd5a40c3aa0 2024-05-27T09:31:10.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.813+0000 7fd5a9c00640 1 --2- 172.21.15.114:0/2180870812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5a40cb330 0x7fd5a40cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:31:10.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.814+0000 7fd5a9200640 1 --2- 172.21.15.114:0/2180870812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5a4156ad0 0x7fd5a406d350 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd5980097a0 tx=0x7fd59802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e595a63bf161029 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:10.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.815+0000 7fd5a3400640 1 -- 172.21.15.114:0/2180870812 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598037070 con 0x7fd5a4156ad0 2024-05-27T09:31:10.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.815+0000 7fd5a3400640 1 -- 172.21.15.114:0/2180870812 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd598032440 con 0x7fd5a4156ad0 2024-05-27T09:31:10.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.815+0000 7fd5a3400640 1 -- 172.21.15.114:0/2180870812 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598032810 con 0x7fd5a4156ad0 2024-05-27T09:31:10.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.816+0000 7fd5abc00640 1 -- 172.21.15.114:0/2180870812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5a4156ad0 msgr2=0x7fd5a406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:10.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.816+0000 7fd5abc00640 1 --2- 172.21.15.114:0/2180870812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5a4156ad0 0x7fd5a406d350 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd5980097a0 tx=0x7fd59802a210 comp rx=0 tx=0).stop 2024-05-27T09:31:10.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.816+0000 7fd5abc00640 1 -- 172.21.15.114:0/2180870812 shutdown_connections 2024-05-27T09:31:10.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.816+0000 7fd5abc00640 1 --2- 172.21.15.114:0/2180870812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5a40cb330 0x7fd5a40cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:10.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.816+0000 7fd5abc00640 1 --2- 172.21.15.114:0/2180870812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5a4156ad0 0x7fd5a406d350 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:10.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.816+0000 7fd5abc00640 1 -- 172.21.15.114:0/2180870812 >> 172.21.15.114:0/2180870812 conn(0x7fd5a4070560 msgr2=0x7fd5a4071e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:10.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.816+0000 7fd5abc00640 1 -- 172.21.15.114:0/2180870812 shutdown_connections 2024-05-27T09:31:10.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.816+0000 7fd5abc00640 1 -- 172.21.15.114:0/2180870812 wait complete. 2024-05-27T09:31:10.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.820+0000 7fd5abc00640 1 Processor -- start 2024-05-27T09:31:10.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.820+0000 7fd5abc00640 1 -- start start 2024-05-27T09:31:10.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.822+0000 7fd5abc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5a40c3aa0 0x7fd5a40cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:10.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.822+0000 7fd5abc00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5a40cb330 0x7fd5a41513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:10.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.822+0000 7fd5abc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5a4156ad0 0x7fd5a4151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:10.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.822+0000 7fd5abc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a406e190 con 0x7fd5a4156ad0 2024-05-27T09:31:10.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.822+0000 7fd5abc00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a406deb0 con 0x7fd5a40c3aa0 2024-05-27T09:31:10.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.822+0000 7fd5abc00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fd5a406e6b0 con 0x7fd5a40cb330 2024-05-27T09:31:10.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.823+0000 7fd5a9200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5a40c3aa0 0x7fd5a40cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:10.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.823+0000 7fd5a9200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5a40c3aa0 0x7fd5a40cce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:40900/0 (socket says 172.21.15.114:40900) 2024-05-27T09:31:10.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.823+0000 7fd5a9200640 1 -- 172.21.15.114:0/478418296 learned_addr learned my addr 172.21.15.114:0/478418296 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:31:10.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.823+0000 7fd5a3e00640 1 --2- 172.21.15.114:0/478418296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5a40cb330 0x7fd5a41513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:10.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.823+0000 7fd5a9c00640 1 --2- 172.21.15.114:0/478418296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5a4156ad0 0x7fd5a4151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:10.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.823+0000 7fd5a9200640 1 -- 172.21.15.114:0/478418296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5a40cb330 msgr2=0x7fd5a41513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:10.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.823+0000 7fd5a9200640 1 --2- 172.21.15.114:0/478418296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5a40cb330 0x7fd5a41513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:10.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.823+0000 7fd5a9200640 1 -- 172.21.15.114:0/478418296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5a4156ad0 msgr2=0x7fd5a4151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:10.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.823+0000 7fd5a9200640 1 --2- 172.21.15.114:0/478418296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5a4156ad0 0x7fd5a4151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:10.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.823+0000 7fd5a9200640 1 -- 172.21.15.114:0/478418296 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a41e9c90 con 0x7fd5a40c3aa0 2024-05-27T09:31:10.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.823+0000 7fd5a9c00640 1 --2- 172.21.15.114:0/478418296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5a4156ad0 0x7fd5a4151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:31:10.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.824+0000 7fd5a9200640 1 --2- 172.21.15.114:0/478418296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5a40c3aa0 0x7fd5a40cce10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd598000c00 tx=0x7fd5980046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:10.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.824+0000 7fd5a0c00640 1 -- 172.21.15.114:0/478418296 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598037050 con 0x7fd5a40c3aa0 2024-05-27T09:31:10.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.824+0000 7fd5a0c00640 1 -- 172.21.15.114:0/478418296 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd598038040 con 0x7fd5a40c3aa0 2024-05-27T09:31:10.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.824+0000 7fd5abc00640 1 -- 172.21.15.114:0/478418296 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a40cd350 con 0x7fd5a40c3aa0 2024-05-27T09:31:10.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.824+0000 7fd5a0c00640 1 -- 172.21.15.114:0/478418296 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598004470 con 0x7fd5a40c3aa0 2024-05-27T09:31:10.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.824+0000 7fd5abc00640 1 -- 172.21.15.114:0/478418296 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a40cf270 con 0x7fd5a40c3aa0 2024-05-27T09:31:10.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.826+0000 7fd5abc00640 1 -- 172.21.15.114:0/478418296 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5a406a380 con 0x7fd5a40c3aa0 2024-05-27T09:31:10.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.827+0000 7fd5a0c00640 1 -- 172.21.15.114:0/478418296 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd59803c6d0 con 0x7fd5a40c3aa0 2024-05-27T09:31:10.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.828+0000 7fd5a0c00640 1 --2- 172.21.15.114:0/478418296 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd5840762d0 0x7fd5840787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:10.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.828+0000 7fd5a0c00640 1 -- 172.21.15.114:0/478418296 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5910+0+0 (secure 0 0 0) 0x7fd5980b81b0 con 0x7fd5a40c3aa0 2024-05-27T09:31:10.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.828+0000 7fd5a3e00640 1 --2- 172.21.15.114:0/478418296 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd5840762d0 0x7fd5840787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:10.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.829+0000 7fd5a3e00640 1 --2- 172.21.15.114:0/478418296 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd5840762d0 0x7fd5840787e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd5a40cfa60 tx=0x7fd590007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:10.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:10.833+0000 7fd5a0c00640 1 -- 172.21.15.114:0/478418296 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd598081910 con 0x7fd5a40c3aa0 2024-05-27T09:31:11.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:11.093+0000 7fd5abc00640 1 -- 172.21.15.114:0/478418296 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5a415a090 con 0x7fd5a40c3aa0 2024-05-27T09:31:11.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:11.094+0000 7fd5a0c00640 1 -- 172.21.15.114:0/478418296 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+6053 (secure 0 0 0) 0x7fd598081210 con 0x7fd5a40c3aa0 2024-05-27T09:31:11.095 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:31:11.095 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:31:10.376876+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:reconnect","state_seq":16,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4297,4300]},"id":1}]} 2024-05-27T09:31:11.096 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 21 2024-05-27T09:31:11.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:11.096+0000 7fd5abc00640 1 -- 172.21.15.114:0/478418296 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd5840762d0 msgr2=0x7fd5840787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:11.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:11.096+0000 7fd5abc00640 1 --2- 172.21.15.114:0/478418296 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd5840762d0 0x7fd5840787e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd5a40cfa60 tx=0x7fd590007de0 comp rx=0 tx=0).stop 2024-05-27T09:31:11.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:11.096+0000 7fd5abc00640 1 -- 172.21.15.114:0/478418296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5a40c3aa0 msgr2=0x7fd5a40cce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:11.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:11.096+0000 7fd5abc00640 1 --2- 172.21.15.114:0/478418296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5a40c3aa0 0x7fd5a40cce10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd598000c00 tx=0x7fd5980046c0 comp rx=0 tx=0).stop 2024-05-27T09:31:11.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:11.096+0000 7fd5abc00640 1 -- 172.21.15.114:0/478418296 shutdown_connections 2024-05-27T09:31:11.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:11.096+0000 7fd5abc00640 1 --2- 172.21.15.114:0/478418296 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd5840762d0 0x7fd5840787e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:11.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:11.096+0000 7fd5abc00640 1 --2- 172.21.15.114:0/478418296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5a4156ad0 0x7fd5a4151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:11.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:11.096+0000 7fd5abc00640 1 --2- 172.21.15.114:0/478418296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5a40cb330 0x7fd5a41513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:11.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:11.096+0000 7fd5abc00640 1 --2- 172.21.15.114:0/478418296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5a40c3aa0 0x7fd5a40cce10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:11.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:11.096+0000 7fd5abc00640 1 -- 172.21.15.114:0/478418296 >> 172.21.15.114:0/478418296 conn(0x7fd5a4070560 msgr2=0x7fd5a4154240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:11.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:11.097+0000 7fd5abc00640 1 -- 172.21.15.114:0/478418296 shutdown_connections 2024-05-27T09:31:11.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:11.097+0000 7fd5abc00640 1 -- 172.21.15.114:0/478418296 wait complete. 2024-05-27T09:31:11.124 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:31:13.124 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:31:13.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.403+0000 7f8153800640 1 Processor -- start 2024-05-27T09:31:13.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.404+0000 7f8153800640 1 -- start start 2024-05-27T09:31:13.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.405+0000 7f8153800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f814c0c8370 0x7f814c152f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:13.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.405+0000 7f8153800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f814c0ca400 0x7f814c0ca810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:13.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.405+0000 7f8153800640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f814c0c42f0 con 0x7f814c0c9170 2024-05-27T09:31:13.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.405+0000 7f8153800640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f814c0c9b40 con 0x7f814c0c8370 2024-05-27T09:31:13.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.405+0000 7f8153800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f814c0c2b20 con 0x7f814c0ca400 2024-05-27T09:31:13.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.406+0000 7f8150e00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f814c0c9170 0x7f814c15c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:45512/0 (socket says 172.21.15.114:45512) 2024-05-27T09:31:13.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.406+0000 7f8150e00640 1 -- 172.21.15.114:0/1259842670 learned_addr learned my addr 172.21.15.114:0/1259842670 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:31:13.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.406+0000 7f8151800640 1 --2- 172.21.15.114:0/1259842670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f814c0ca400 0x7f814c0ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:13.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.406+0000 7f814be00640 1 --2- 172.21.15.114:0/1259842670 >> v2:172.21.15.114:3300/0 conn(0x7f814c0c8370 0x7f814c152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:13.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.406+0000 7f814b400640 1 -- 172.21.15.114:0/1259842670 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2187694609 0 0) 0x7f814c0c42f0 con 0x7f814c0c9170 2024-05-27T09:31:13.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.406+0000 7f814b400640 1 -- 172.21.15.114:0/1259842670 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8130003520 con 0x7f814c0c9170 2024-05-27T09:31:13.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.406+0000 7f8151800640 1 -- 172.21.15.114:0/1259842670 >> v2:172.21.15.114:3300/0 conn(0x7f814c0c8370 msgr2=0x7f814c152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:31:13.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.406+0000 7f8151800640 1 --2- 172.21.15.114:0/1259842670 >> v2:172.21.15.114:3300/0 conn(0x7f814c0c8370 0x7f814c152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:13.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.406+0000 7f8151800640 1 -- 172.21.15.114:0/1259842670 >> v1:172.21.15.114:6789/0 conn(0x7f814c0c9170 legacy=0x7f814c15c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:13.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.406+0000 7f8151800640 1 -- 172.21.15.114:0/1259842670 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814c0c45d0 con 0x7f814c0ca400 2024-05-27T09:31:13.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.407+0000 7f8151800640 1 --2- 172.21.15.114:0/1259842670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f814c0ca400 0x7f814c0ca810 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f814400aea0 tx=0x7f814402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18d5ad08854fb915 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:13.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.408+0000 7f814b400640 1 -- 172.21.15.114:0/1259842670 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8144038070 con 0x7f814c0ca400 2024-05-27T09:31:13.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.408+0000 7f814b400640 1 -- 172.21.15.114:0/1259842670 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8144008640 con 0x7f814c0ca400 2024-05-27T09:31:13.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.408+0000 7f814b400640 1 -- 172.21.15.114:0/1259842670 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8144008a10 con 0x7f814c0ca400 2024-05-27T09:31:13.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.408+0000 7f8153800640 1 -- 172.21.15.114:0/1259842670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f814c0ca400 msgr2=0x7f814c0ca810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:13.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.408+0000 7f8153800640 1 --2- 172.21.15.114:0/1259842670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f814c0ca400 0x7f814c0ca810 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f814400aea0 tx=0x7f814402b840 comp rx=0 tx=0).stop 2024-05-27T09:31:13.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.409+0000 7f8153800640 1 -- 172.21.15.114:0/1259842670 shutdown_connections 2024-05-27T09:31:13.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.409+0000 7f8153800640 1 --2- 172.21.15.114:0/1259842670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f814c0ca400 0x7f814c0ca810 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:13.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.409+0000 7f8153800640 1 --2- 172.21.15.114:0/1259842670 >> v2:172.21.15.114:3300/0 conn(0x7f814c0c8370 0x7f814c152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:13.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.409+0000 7f8153800640 1 -- 172.21.15.114:0/1259842670 >> 172.21.15.114:0/1259842670 conn(0x7f814c06f6e0 msgr2=0x7f814c070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:13.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.409+0000 7f8153800640 1 -- 172.21.15.114:0/1259842670 shutdown_connections 2024-05-27T09:31:13.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.409+0000 7f8153800640 1 -- 172.21.15.114:0/1259842670 wait complete. 2024-05-27T09:31:13.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.411+0000 7f8153800640 1 Processor -- start 2024-05-27T09:31:13.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.411+0000 7f8153800640 1 -- start start 2024-05-27T09:31:13.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.413+0000 7f8153800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f814c0c8370 0x7f814c1e4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:13.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.413+0000 7f8153800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f814c0c9170 0x7f814c15b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:13.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.413+0000 7f8153800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f814c0ca400 0x7f814c14e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:13.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.413+0000 7f8153800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f814c0c9fd0 con 0x7f814c0c9170 2024-05-27T09:31:13.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.413+0000 7f8153800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f814c0c97d0 con 0x7f814c0c8370 2024-05-27T09:31:13.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.413+0000 7f8153800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f814c0ca150 con 0x7f814c0ca400 2024-05-27T09:31:13.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.413+0000 7f814be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f814c0c9170 0x7f814c15b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:13.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.413+0000 7f8150e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f814c0c8370 0x7f814c1e4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:13.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.413+0000 7f814be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f814c0c9170 0x7f814c15b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37686/0 (socket says 172.21.15.114:37686) 2024-05-27T09:31:13.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.413+0000 7f814be00640 1 -- 172.21.15.114:0/577119874 learned_addr learned my addr 172.21.15.114:0/577119874 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:31:13.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.413+0000 7f8151800640 1 --2- 172.21.15.114:0/577119874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f814c0ca400 0x7f814c14e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:13.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.414+0000 7f814be00640 1 -- 172.21.15.114:0/577119874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f814c0ca400 msgr2=0x7f814c14e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:13.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.414+0000 7f814be00640 1 --2- 172.21.15.114:0/577119874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f814c0ca400 0x7f814c14e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:13.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.414+0000 7f814be00640 1 -- 172.21.15.114:0/577119874 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f814c0c8370 msgr2=0x7f814c1e4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:13.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.414+0000 7f814be00640 1 --2- 172.21.15.114:0/577119874 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f814c0c8370 0x7f814c1e4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:13.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.414+0000 7f814be00640 1 -- 172.21.15.114:0/577119874 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814c15bab0 con 0x7f814c0c9170 2024-05-27T09:31:13.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.414+0000 7f814be00640 1 --2- 172.21.15.114:0/577119874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f814c0c9170 0x7f814c15b570 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f813800e870 tx=0x7f813800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:13.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.414+0000 7f8148c00640 1 -- 172.21.15.114:0/577119874 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138017070 con 0x7f814c0c9170 2024-05-27T09:31:13.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.414+0000 7f8148c00640 1 -- 172.21.15.114:0/577119874 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f813800c8e0 con 0x7f814c0c9170 2024-05-27T09:31:13.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.414+0000 7f8153800640 1 -- 172.21.15.114:0/577119874 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814c15a230 con 0x7f814c0c9170 2024-05-27T09:31:13.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.414+0000 7f8148c00640 1 -- 172.21.15.114:0/577119874 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813800ccb0 con 0x7f814c0c9170 2024-05-27T09:31:13.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.415+0000 7f8153800640 1 -- 172.21.15.114:0/577119874 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f814c15a7b0 con 0x7f814c0c9170 2024-05-27T09:31:13.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.416+0000 7f8153800640 1 -- 172.21.15.114:0/577119874 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f814c06a380 con 0x7f814c0c9170 2024-05-27T09:31:13.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.417+0000 7f8148c00640 1 -- 172.21.15.114:0/577119874 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f813801a070 con 0x7f814c0c9170 2024-05-27T09:31:13.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.418+0000 7f8148c00640 1 --2- 172.21.15.114:0/577119874 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f812c0762d0 0x7f812c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:13.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.418+0000 7f8148c00640 1 -- 172.21.15.114:0/577119874 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5910+0+0 (secure 0 0 0) 0x7f8138097c50 con 0x7f814c0c9170 2024-05-27T09:31:13.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.419+0000 7f8150e00640 1 --2- 172.21.15.114:0/577119874 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f812c0762d0 0x7f812c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:13.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.419+0000 7f8150e00640 1 --2- 172.21.15.114:0/577119874 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f812c0762d0 0x7f812c0787e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f813c005880 tx=0x7f813c006aa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:13.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.424+0000 7f8148c00640 1 -- 172.21.15.114:0/577119874 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8138060f70 con 0x7f814c0c9170 2024-05-27T09:31:13.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.667+0000 7f8153800640 1 -- 172.21.15.114:0/577119874 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f814c14ff20 con 0x7f814c0c9170 2024-05-27T09:31:13.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.669+0000 7f8148c00640 1 -- 172.21.15.114:0/577119874 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+6053 (secure 0 0 0) 0x7f8138065270 con 0x7f814c0c9170 2024-05-27T09:31:13.671 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:31:13.671 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:31:10.376876+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:reconnect","state_seq":16,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4297,4300]},"id":1}]} 2024-05-27T09:31:13.671 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 21 2024-05-27T09:31:13.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.672+0000 7f8153800640 1 -- 172.21.15.114:0/577119874 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f812c0762d0 msgr2=0x7f812c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:13.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.672+0000 7f8153800640 1 --2- 172.21.15.114:0/577119874 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f812c0762d0 0x7f812c0787e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f813c005880 tx=0x7f813c006aa0 comp rx=0 tx=0).stop 2024-05-27T09:31:13.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.672+0000 7f8153800640 1 -- 172.21.15.114:0/577119874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f814c0c9170 msgr2=0x7f814c15b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:13.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.672+0000 7f8153800640 1 --2- 172.21.15.114:0/577119874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f814c0c9170 0x7f814c15b570 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f813800e870 tx=0x7f813800ed30 comp rx=0 tx=0).stop 2024-05-27T09:31:13.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.672+0000 7f8153800640 1 -- 172.21.15.114:0/577119874 shutdown_connections 2024-05-27T09:31:13.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.672+0000 7f8153800640 1 --2- 172.21.15.114:0/577119874 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f812c0762d0 0x7f812c0787e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:13.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.672+0000 7f8153800640 1 --2- 172.21.15.114:0/577119874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f814c0ca400 0x7f814c14e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:13.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.672+0000 7f8153800640 1 --2- 172.21.15.114:0/577119874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f814c0c9170 0x7f814c15b570 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:13.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.672+0000 7f8153800640 1 --2- 172.21.15.114:0/577119874 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f814c0c8370 0x7f814c1e4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:13.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.672+0000 7f8153800640 1 -- 172.21.15.114:0/577119874 >> 172.21.15.114:0/577119874 conn(0x7f814c06f6e0 msgr2=0x7f814c157890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:13.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.672+0000 7f8153800640 1 -- 172.21.15.114:0/577119874 shutdown_connections 2024-05-27T09:31:13.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:13.672+0000 7f8153800640 1 -- 172.21.15.114:0/577119874 wait complete. 2024-05-27T09:31:13.698 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:31:15.699 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:31:15.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.980+0000 7f607a400640 1 Processor -- start 2024-05-27T09:31:15.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.981+0000 7f607a400640 1 -- start start 2024-05-27T09:31:15.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.983+0000 7f607a400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6074074a60 0x7f6074154820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:15.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.983+0000 7f607a400640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f607415a810 con 0x7f6074073260 2024-05-27T09:31:15.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.983+0000 7f607a400640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f607415aaf0 con 0x7f60740ca9e0 2024-05-27T09:31:15.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.983+0000 7f607a400640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f607415db60 con 0x7f6074074a60 2024-05-27T09:31:15.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.983+0000 7f6072a00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f6074073260 0x7f60740ca2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:49278/0 (socket says 172.21.15.114:49278) 2024-05-27T09:31:15.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.983+0000 7f6072a00640 1 -- 172.21.15.114:0/1356870861 learned_addr learned my addr 172.21.15.114:0/1356870861 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:31:15.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.983+0000 7f6073400640 1 --2- 172.21.15.114:0/1356870861 >> v2:172.21.15.163:3300/0 conn(0x7f6074074a60 0x7f6074154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:15.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.984+0000 7f6072000640 1 -- 172.21.15.114:0/1356870861 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3897356157 0 0) 0x7f607415aaf0 con 0x7f60740ca9e0 2024-05-27T09:31:15.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.984+0000 7f6073400640 1 -- 172.21.15.114:0/1356870861 >> v1:172.21.15.163:6789/0 conn(0x7f60740ca9e0 legacy=0x7f607415dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:15.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.984+0000 7f6073400640 1 -- 172.21.15.114:0/1356870861 >> v1:172.21.15.114:6789/0 conn(0x7f6074073260 legacy=0x7f60740ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:15.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.984+0000 7f6073400640 1 -- 172.21.15.114:0/1356870861 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f607415add0 con 0x7f6074074a60 2024-05-27T09:31:15.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.985+0000 7f6073400640 1 --2- 172.21.15.114:0/1356870861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6074074a60 0x7f6074154820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6064009d90 tx=0x7f606402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6542db3e7b89a2d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:15.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.985+0000 7f6072000640 1 -- 172.21.15.114:0/1356870861 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6064007740 con 0x7f6074074a60 2024-05-27T09:31:15.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.985+0000 7f6072000640 1 -- 172.21.15.114:0/1356870861 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60640078e0 con 0x7f6074074a60 2024-05-27T09:31:15.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.985+0000 7f6072000640 1 -- 172.21.15.114:0/1356870861 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6064007cb0 con 0x7f6074074a60 2024-05-27T09:31:15.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.986+0000 7f607a400640 1 -- 172.21.15.114:0/1356870861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6074074a60 msgr2=0x7f6074154820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:15.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.986+0000 7f607a400640 1 --2- 172.21.15.114:0/1356870861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6074074a60 0x7f6074154820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6064009d90 tx=0x7f606402a680 comp rx=0 tx=0).stop 2024-05-27T09:31:15.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.986+0000 7f607a400640 1 -- 172.21.15.114:0/1356870861 shutdown_connections 2024-05-27T09:31:15.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.986+0000 7f607a400640 1 --2- 172.21.15.114:0/1356870861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6074074a60 0x7f6074154820 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:15.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.986+0000 7f607a400640 1 -- 172.21.15.114:0/1356870861 >> 172.21.15.114:0/1356870861 conn(0x7f607406f6e0 msgr2=0x7f6074070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:15.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.986+0000 7f607a400640 1 -- 172.21.15.114:0/1356870861 shutdown_connections 2024-05-27T09:31:15.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.986+0000 7f607a400640 1 -- 172.21.15.114:0/1356870861 wait complete. 2024-05-27T09:31:15.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.988+0000 7f607a400640 1 Processor -- start 2024-05-27T09:31:15.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.988+0000 7f607a400640 1 -- start start 2024-05-27T09:31:15.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.990+0000 7f607a400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6074073260 0x7f60741575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:15.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.990+0000 7f607a400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6074074a60 0x7f60740c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.990+0000 7f607a400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60740ca9e0 0x7f607414a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.990+0000 7f607a400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f607415a4e0 con 0x7f6074074a60 2024-05-27T09:31:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.990+0000 7f607a400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f607415b390 con 0x7f60740ca9e0 2024-05-27T09:31:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.990+0000 7f607a400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f607415e700 con 0x7f6074073260 2024-05-27T09:31:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.990+0000 7f6072a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6074074a60 0x7f60740c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.990+0000 7f6072a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6074074a60 0x7f60740c6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37424/0 (socket says 172.21.15.114:37424) 2024-05-27T09:31:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.990+0000 7f6072a00640 1 -- 172.21.15.114:0/1126317891 learned_addr learned my addr 172.21.15.114:0/1126317891 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:31:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.990+0000 7f6073400640 1 --2- 172.21.15.114:0/1126317891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6074073260 0x7f60741575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.990+0000 7f6073e00640 1 --2- 172.21.15.114:0/1126317891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60740ca9e0 0x7f607414a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:15.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.991+0000 7f6072a00640 1 -- 172.21.15.114:0/1126317891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6074073260 msgr2=0x7f60741575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:15.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.991+0000 7f6072a00640 1 --2- 172.21.15.114:0/1126317891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6074073260 0x7f60741575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:15.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.991+0000 7f6072a00640 1 -- 172.21.15.114:0/1126317891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60740ca9e0 msgr2=0x7f607414a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:15.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.991+0000 7f6072a00640 1 --2- 172.21.15.114:0/1126317891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60740ca9e0 0x7f607414a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:15.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.991+0000 7f6072a00640 1 -- 172.21.15.114:0/1126317891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60741593f0 con 0x7f6074074a60 2024-05-27T09:31:15.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.991+0000 7f6073400640 1 --2- 172.21.15.114:0/1126317891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6074073260 0x7f60741575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:31:15.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.991+0000 7f6073e00640 1 --2- 172.21.15.114:0/1126317891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60740ca9e0 0x7f607414a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:31:15.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.991+0000 7f6072a00640 1 --2- 172.21.15.114:0/1126317891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6074074a60 0x7f60740c6580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f606000c490 tx=0x7f606000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:15.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.991+0000 7f606b400640 1 -- 172.21.15.114:0/1126317891 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60600074e0 con 0x7f6074074a60 2024-05-27T09:31:15.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.991+0000 7f606b400640 1 -- 172.21.15.114:0/1126317891 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60600047e0 con 0x7f6074074a60 2024-05-27T09:31:15.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.991+0000 7f607a400640 1 -- 172.21.15.114:0/1126317891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6074155b70 con 0x7f6074074a60 2024-05-27T09:31:15.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.991+0000 7f606b400640 1 -- 172.21.15.114:0/1126317891 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060004b90 con 0x7f6074074a60 2024-05-27T09:31:15.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.992+0000 7f607a400640 1 -- 172.21.15.114:0/1126317891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6074155640 con 0x7f6074074a60 2024-05-27T09:31:15.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.993+0000 7f607a400640 1 -- 172.21.15.114:0/1126317891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f607406a380 con 0x7f6074074a60 2024-05-27T09:31:15.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.994+0000 7f606b400640 1 -- 172.21.15.114:0/1126317891 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6060011070 con 0x7f6074074a60 2024-05-27T09:31:15.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.995+0000 7f606b400640 1 --2- 172.21.15.114:0/1126317891 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f60540762d0 0x7f60540787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:15.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.995+0000 7f606b400640 1 -- 172.21.15.114:0/1126317891 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5910+0+0 (secure 0 0 0) 0x7f6060097170 con 0x7f6074074a60 2024-05-27T09:31:15.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.996+0000 7f6073400640 1 --2- 172.21.15.114:0/1126317891 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f60540762d0 0x7f60540787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:15.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:15.996+0000 7f6073400640 1 --2- 172.21.15.114:0/1126317891 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f60540762d0 0x7f60540787e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6064034000 tx=0x7f606402f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:16.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:16.000+0000 7f606b400640 1 -- 172.21.15.114:0/1126317891 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f60600603a0 con 0x7f6074074a60 2024-05-27T09:31:16.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:16.243+0000 7f607a400640 1 -- 172.21.15.114:0/1126317891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6074073ca0 con 0x7f6074074a60 2024-05-27T09:31:16.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:16.245+0000 7f606b400640 1 -- 172.21.15.114:0/1126317891 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+6050 (secure 0 0 0) 0x7f60600646a0 con 0x7f6074074a60 2024-05-27T09:31:16.246 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:31:16.246 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:31:15.406010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:rejoin","state_seq":18,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4297]},"id":1}]} 2024-05-27T09:31:16.247 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 22 2024-05-27T09:31:16.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:16.247+0000 7f607a400640 1 -- 172.21.15.114:0/1126317891 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f60540762d0 msgr2=0x7f60540787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:16.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:16.247+0000 7f607a400640 1 --2- 172.21.15.114:0/1126317891 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f60540762d0 0x7f60540787e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6064034000 tx=0x7f606402f000 comp rx=0 tx=0).stop 2024-05-27T09:31:16.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:16.247+0000 7f607a400640 1 -- 172.21.15.114:0/1126317891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6074074a60 msgr2=0x7f60740c6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:16.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:16.247+0000 7f607a400640 1 --2- 172.21.15.114:0/1126317891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6074074a60 0x7f60740c6580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f606000c490 tx=0x7f606000c950 comp rx=0 tx=0).stop 2024-05-27T09:31:16.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:16.247+0000 7f607a400640 1 -- 172.21.15.114:0/1126317891 shutdown_connections 2024-05-27T09:31:16.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:16.247+0000 7f607a400640 1 --2- 172.21.15.114:0/1126317891 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f60540762d0 0x7f60540787e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:16.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:16.247+0000 7f607a400640 1 --2- 172.21.15.114:0/1126317891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60740ca9e0 0x7f607414a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:16.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:16.247+0000 7f607a400640 1 --2- 172.21.15.114:0/1126317891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6074074a60 0x7f60740c6580 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:16.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:16.247+0000 7f607a400640 1 --2- 172.21.15.114:0/1126317891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6074073260 0x7f60741575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:16.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:16.247+0000 7f607a400640 1 -- 172.21.15.114:0/1126317891 >> 172.21.15.114:0/1126317891 conn(0x7f607406f6e0 msgr2=0x7f6074073ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:16.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:16.247+0000 7f607a400640 1 -- 172.21.15.114:0/1126317891 shutdown_connections 2024-05-27T09:31:16.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:16.247+0000 7f607a400640 1 -- 172.21.15.114:0/1126317891 wait complete. 2024-05-27T09:31:16.275 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 22, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-05-27T09:27:18.079214+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 22, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4297": { "addr": "172.21.15.114:6834/657027519", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6832", "nonce": 657027519, "type": "v2" }, { "addr": "172.21.15.114:6834", "nonce": 657027519, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4297, "incarnation": 9, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4300": { "addr": "172.21.15.114:6837/1448950017", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6836", "nonce": 1448950017, "type": "v2" }, { "addr": "172.21.15.114:6837", "nonce": 1448950017, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4300, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 4 }, "gid_4828": { "addr": "172.21.15.163:6839/3371890103", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6837", "nonce": 3371890103, "type": "v2" }, { "addr": "172.21.15.163:6839", "nonce": 3371890103, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4828, "incarnation": 19, "join_fscid": -1, "name": "f", "rank": 2, "state": "up:rejoin", "state_seq": 18 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-05-27T09:31:15.406010+0000", "qdb_cluster": [ 4300, 4297 ], "qdb_leader": 4300, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4300, "mds_1": 4297, "mds_2": 4828 } } } ], "standbys": [ { "addr": "172.21.15.163:6835/3513522318", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 3513522318, "type": "v2" }, { "addr": "172.21.15.163:6835", "nonce": 3513522318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6838/1753596112", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6836", "nonce": 1753596112, "type": "v2" }, { "addr": "172.21.15.163:6838", "nonce": 1753596112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-05-27T09:31:18.275 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:31:18.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.553+0000 7f691a200640 1 Processor -- start 2024-05-27T09:31:18.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.554+0000 7f691a200640 1 -- start start 2024-05-27T09:31:18.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.556+0000 7f691a200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6914142720 0x7f6914073260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:18.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.556+0000 7f691a200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f69140c2990 0x7f69140c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:18.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.556+0000 7f691a200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69140739a0 con 0x7f69141593b0 2024-05-27T09:31:18.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.556+0000 7f691a200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f6914074d20 con 0x7f6914142720 2024-05-27T09:31:18.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.556+0000 7f691a200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f6914142b30 con 0x7f69140c2990 2024-05-27T09:31:18.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.557+0000 7f690aa00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6914142720 0x7f6914073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:18.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.557+0000 7f690aa00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f6914142720 0x7f6914073260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37426/0 (socket says 172.21.15.114:37426) 2024-05-27T09:31:18.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.557+0000 7f6913400640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f69141593b0 0x7f691415a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:55170/0 (socket says 172.21.15.114:55170) 2024-05-27T09:31:18.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.557+0000 7f690aa00640 1 -- 172.21.15.114:0/560264934 learned_addr learned my addr 172.21.15.114:0/560264934 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:31:18.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.557+0000 7f6913e00640 1 --2- 172.21.15.114:0/560264934 >> v2:172.21.15.163:3300/0 conn(0x7f69140c2990 0x7f69140c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:18.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.557+0000 7f690aa00640 1 -- 172.21.15.114:0/560264934 >> v2:172.21.15.163:3300/0 conn(0x7f69140c2990 msgr2=0x7f69140c2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:31:18.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.557+0000 7f690aa00640 1 --2- 172.21.15.114:0/560264934 >> v2:172.21.15.163:3300/0 conn(0x7f69140c2990 0x7f69140c2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:18.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.557+0000 7f6912a00640 1 -- 172.21.15.114:0/560264934 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1698984412 0 0) 0x7f69140739a0 con 0x7f69141593b0 2024-05-27T09:31:18.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.557+0000 7f690aa00640 1 -- 172.21.15.114:0/560264934 >> v1:172.21.15.163:6789/0 conn(0x7f69141593b0 legacy=0x7f691415a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:18.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.557+0000 7f690aa00640 1 -- 172.21.15.114:0/560264934 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6914073c80 con 0x7f6914142720 2024-05-27T09:31:18.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.558+0000 7f690aa00640 1 --2- 172.21.15.114:0/560264934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6914142720 0x7f6914073260 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f69140731f0 tx=0x7f68fc02c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de44bb54ebbe1a0d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:18.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.559+0000 7f6912a00640 1 -- 172.21.15.114:0/560264934 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc038070 con 0x7f6914142720 2024-05-27T09:31:18.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.559+0000 7f6912a00640 1 -- 172.21.15.114:0/560264934 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68fc031710 con 0x7f6914142720 2024-05-27T09:31:18.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.559+0000 7f6912a00640 1 -- 172.21.15.114:0/560264934 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc031ae0 con 0x7f6914142720 2024-05-27T09:31:18.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.559+0000 7f691a200640 1 -- 172.21.15.114:0/560264934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6914142720 msgr2=0x7f6914073260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:18.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.559+0000 7f691a200640 1 --2- 172.21.15.114:0/560264934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6914142720 0x7f6914073260 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f69140731f0 tx=0x7f68fc02c510 comp rx=0 tx=0).stop 2024-05-27T09:31:18.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.559+0000 7f691a200640 1 -- 172.21.15.114:0/560264934 shutdown_connections 2024-05-27T09:31:18.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.559+0000 7f691a200640 1 --2- 172.21.15.114:0/560264934 >> v2:172.21.15.163:3300/0 conn(0x7f69140c2990 0x7f69140c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:18.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.559+0000 7f691a200640 1 --2- 172.21.15.114:0/560264934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6914142720 0x7f6914073260 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:18.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.559+0000 7f691a200640 1 -- 172.21.15.114:0/560264934 >> 172.21.15.114:0/560264934 conn(0x7f691406f6e0 msgr2=0x7f6914070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:18.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.560+0000 7f691a200640 1 -- 172.21.15.114:0/560264934 shutdown_connections 2024-05-27T09:31:18.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.560+0000 7f691a200640 1 -- 172.21.15.114:0/560264934 wait complete. 2024-05-27T09:31:18.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.562+0000 7f691a200640 1 Processor -- start 2024-05-27T09:31:18.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.562+0000 7f691a200640 1 -- start start 2024-05-27T09:31:18.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.563+0000 7f691a200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69140c2990 0x7f691413ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:18.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.563+0000 7f691a200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6914142720 0x7f691413a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:18.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.563+0000 7f691a200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69141593b0 0x7f69141e2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:18.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.564+0000 7f691a200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6914074520 con 0x7f69140c2990 2024-05-27T09:31:18.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.564+0000 7f691a200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6914074240 con 0x7f69141593b0 2024-05-27T09:31:18.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.564+0000 7f691a200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f6914074800 con 0x7f6914142720 2024-05-27T09:31:18.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.564+0000 7f6913400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69140c2990 0x7f691413ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:18.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.564+0000 7f690aa00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6914142720 0x7f691413a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:18.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.564+0000 7f6913e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69141593b0 0x7f69141e2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:18.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.564+0000 7f6913e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69141593b0 0x7f69141e2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:38884/0 (socket says 172.21.15.114:38884) 2024-05-27T09:31:18.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.564+0000 7f6913400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69140c2990 0x7f691413ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37434/0 (socket says 172.21.15.114:37434) 2024-05-27T09:31:18.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.564+0000 7f6913e00640 1 -- 172.21.15.114:0/2607139004 learned_addr learned my addr 172.21.15.114:0/2607139004 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:31:18.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.564+0000 7f690aa00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6914142720 0x7f691413a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:38160/0 (socket says 172.21.15.114:38160) 2024-05-27T09:31:18.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.564+0000 7f690aa00640 1 -- 172.21.15.114:0/2607139004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69141593b0 msgr2=0x7f69141e2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:18.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.564+0000 7f690aa00640 1 --2- 172.21.15.114:0/2607139004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69141593b0 0x7f69141e2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:18.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.564+0000 7f690aa00640 1 -- 172.21.15.114:0/2607139004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69140c2990 msgr2=0x7f691413ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:18.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.564+0000 7f690aa00640 1 --2- 172.21.15.114:0/2607139004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69140c2990 0x7f691413ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:18.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.564+0000 7f690aa00640 1 -- 172.21.15.114:0/2607139004 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691415a570 con 0x7f6914142720 2024-05-27T09:31:18.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.564+0000 7f6913400640 1 --2- 172.21.15.114:0/2607139004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69140c2990 0x7f691413ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:31:18.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.565+0000 7f690aa00640 1 --2- 172.21.15.114:0/2607139004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6914142720 0x7f691413a5f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f68fc033000 tx=0x7f68fc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:18.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.565+0000 7f690be00640 1 -- 172.21.15.114:0/2607139004 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc038040 con 0x7f6914142720 2024-05-27T09:31:18.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.565+0000 7f691a200640 1 -- 172.21.15.114:0/2607139004 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69141533f0 con 0x7f6914142720 2024-05-27T09:31:18.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.565+0000 7f690be00640 1 -- 172.21.15.114:0/2607139004 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68fc004230 con 0x7f6914142720 2024-05-27T09:31:18.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.565+0000 7f690be00640 1 -- 172.21.15.114:0/2607139004 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc03d670 con 0x7f6914142720 2024-05-27T09:31:18.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.565+0000 7f691a200640 1 -- 172.21.15.114:0/2607139004 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f691415a830 con 0x7f6914142720 2024-05-27T09:31:18.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.567+0000 7f691a200640 1 -- 172.21.15.114:0/2607139004 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f691406a380 con 0x7f6914142720 2024-05-27T09:31:18.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.568+0000 7f690be00640 1 -- 172.21.15.114:0/2607139004 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f68fc03b070 con 0x7f6914142720 2024-05-27T09:31:18.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.569+0000 7f690be00640 1 --2- 172.21.15.114:0/2607139004 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f68f00762d0 0x7f68f00787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:18.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.569+0000 7f690be00640 1 -- 172.21.15.114:0/2607139004 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 5910+0+0 (secure 0 0 0) 0x7f68fc0ba5c0 con 0x7f6914142720 2024-05-27T09:31:18.571 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:31:18.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.570+0000 7f6913400640 1 --2- 172.21.15.114:0/2607139004 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f68f00762d0 0x7f68f00787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:18.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.570+0000 7f6913400640 1 --2- 172.21.15.114:0/2607139004 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f68f00762d0 0x7f68f00787e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6904007fd0 tx=0x7f6904007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:18.576 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:31:18.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.573+0000 7f690be00640 1 -- 172.21.15.114:0/2607139004 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f68fc042020 con 0x7f6914142720 2024-05-27T09:31:18.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.822+0000 7f691a200640 1 -- 172.21.15.114:0/2607139004 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f691415d690 con 0x7f6914142720 2024-05-27T09:31:18.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.824+0000 7f690be00640 1 -- 172.21.15.114:0/2607139004 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+6050 (secure 0 0 0) 0x7f68fc0837f0 con 0x7f6914142720 2024-05-27T09:31:18.825 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:31:18.825 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:31:15.406010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:rejoin","state_seq":18,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4297]},"id":1}]} 2024-05-27T09:31:18.825 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 22 2024-05-27T09:31:18.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.826+0000 7f691a200640 1 -- 172.21.15.114:0/2607139004 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f68f00762d0 msgr2=0x7f68f00787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:18.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.826+0000 7f691a200640 1 --2- 172.21.15.114:0/2607139004 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f68f00762d0 0x7f68f00787e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6904007fd0 tx=0x7f6904007de0 comp rx=0 tx=0).stop 2024-05-27T09:31:18.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.826+0000 7f691a200640 1 -- 172.21.15.114:0/2607139004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6914142720 msgr2=0x7f691413a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:18.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.826+0000 7f691a200640 1 --2- 172.21.15.114:0/2607139004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6914142720 0x7f691413a5f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f68fc033000 tx=0x7f68fc0046c0 comp rx=0 tx=0).stop 2024-05-27T09:31:18.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.826+0000 7f691a200640 1 -- 172.21.15.114:0/2607139004 shutdown_connections 2024-05-27T09:31:18.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.826+0000 7f691a200640 1 --2- 172.21.15.114:0/2607139004 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f68f00762d0 0x7f68f00787e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:18.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.826+0000 7f691a200640 1 --2- 172.21.15.114:0/2607139004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69141593b0 0x7f69141e2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:18.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.826+0000 7f691a200640 1 --2- 172.21.15.114:0/2607139004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6914142720 0x7f691413a5f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:18.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.826+0000 7f691a200640 1 --2- 172.21.15.114:0/2607139004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69140c2990 0x7f691413ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:18.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.826+0000 7f691a200640 1 -- 172.21.15.114:0/2607139004 >> 172.21.15.114:0/2607139004 conn(0x7f691406f6e0 msgr2=0x7f6914070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:18.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.826+0000 7f691a200640 1 -- 172.21.15.114:0/2607139004 shutdown_connections 2024-05-27T09:31:18.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:18.826+0000 7f691a200640 1 -- 172.21.15.114:0/2607139004 wait complete. 2024-05-27T09:31:18.854 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:31:20.855 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:31:21.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.127+0000 7f6b0c000640 1 Processor -- start 2024-05-27T09:31:21.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.127+0000 7f6b0c000640 1 -- start start 2024-05-27T09:31:21.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.130+0000 7f6b0c000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b0415d280 0x7f6b04073310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:21.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.130+0000 7f6b0c000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6b04073850 0x7f6b040c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:21.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.130+0000 7f6b0c000640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b04156f80 con 0x7f6b040b2c60 2024-05-27T09:31:21.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.130+0000 7f6b0c000640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f6b0415fd50 con 0x7f6b04073850 2024-05-27T09:31:21.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.130+0000 7f6b0c000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f6b040cac50 con 0x7f6b0415d280 2024-05-27T09:31:21.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.130+0000 7f6b09600640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f6b040b2c60 0x7f6b040ca140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:49288/0 (socket says 172.21.15.114:49288) 2024-05-27T09:31:21.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.130+0000 7f6b09600640 1 -- 172.21.15.114:0/4063581122 learned_addr learned my addr 172.21.15.114:0/4063581122 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:31:21.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.130+0000 7f6b08c00640 1 --2- 172.21.15.114:0/4063581122 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b0415d280 0x7f6b04073310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:21.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.130+0000 7f6b0a000640 1 --2- 172.21.15.114:0/4063581122 >> v2:172.21.15.163:3300/0 conn(0x7f6b04073850 0x7f6b040c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:21.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.131+0000 7f6b08c00640 1 -- 172.21.15.114:0/4063581122 >> v2:172.21.15.163:3300/0 conn(0x7f6b04073850 msgr2=0x7f6b040c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:31:21.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.131+0000 7f6b08c00640 1 --2- 172.21.15.114:0/4063581122 >> v2:172.21.15.163:3300/0 conn(0x7f6b04073850 0x7f6b040c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:21.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.131+0000 7f6b08c00640 1 -- 172.21.15.114:0/4063581122 >> v1:172.21.15.114:6789/0 conn(0x7f6b040b2c60 legacy=0x7f6b040ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:21.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.131+0000 7f6b08c00640 1 -- 172.21.15.114:0/4063581122 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b0415ffa0 con 0x7f6b0415d280 2024-05-27T09:31:21.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.131+0000 7f6b0a000640 1 --2- 172.21.15.114:0/4063581122 >> v2:172.21.15.163:3300/0 conn(0x7f6b04073850 0x7f6b040c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:31:21.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.132+0000 7f6b08c00640 1 --2- 172.21.15.114:0/4063581122 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b0415d280 0x7f6b04073310 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6af00097a0 tx=0x7f6af002a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e91dceada366d87 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:21.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.132+0000 7f6b03e00640 1 -- 172.21.15.114:0/4063581122 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0037070 con 0x7f6b0415d280 2024-05-27T09:31:21.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.132+0000 7f6b03e00640 1 -- 172.21.15.114:0/4063581122 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6af0032440 con 0x7f6b0415d280 2024-05-27T09:31:21.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.132+0000 7f6b03e00640 1 -- 172.21.15.114:0/4063581122 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0032810 con 0x7f6b0415d280 2024-05-27T09:31:21.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.133+0000 7f6b0c000640 1 -- 172.21.15.114:0/4063581122 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b0415d280 msgr2=0x7f6b04073310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:21.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.133+0000 7f6b0c000640 1 --2- 172.21.15.114:0/4063581122 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b0415d280 0x7f6b04073310 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6af00097a0 tx=0x7f6af002a230 comp rx=0 tx=0).stop 2024-05-27T09:31:21.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.133+0000 7f6b0c000640 1 -- 172.21.15.114:0/4063581122 shutdown_connections 2024-05-27T09:31:21.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.133+0000 7f6b0c000640 1 --2- 172.21.15.114:0/4063581122 >> v2:172.21.15.163:3300/0 conn(0x7f6b04073850 0x7f6b040c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:21.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.133+0000 7f6b0c000640 1 --2- 172.21.15.114:0/4063581122 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b0415d280 0x7f6b04073310 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:21.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.133+0000 7f6b0c000640 1 -- 172.21.15.114:0/4063581122 >> 172.21.15.114:0/4063581122 conn(0x7f6b0406f6e0 msgr2=0x7f6b04070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:21.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.133+0000 7f6b0c000640 1 -- 172.21.15.114:0/4063581122 shutdown_connections 2024-05-27T09:31:21.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.133+0000 7f6b0c000640 1 -- 172.21.15.114:0/4063581122 wait complete. 2024-05-27T09:31:21.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.135+0000 7f6b0c000640 1 Processor -- start 2024-05-27T09:31:21.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.136+0000 7f6b0c000640 1 -- start start 2024-05-27T09:31:21.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.137+0000 7f6b0c000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b04073850 0x7f6b04153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:21.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.137+0000 7f6b0c000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b040b2c60 0x7f6b0415aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:21.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.137+0000 7f6b0c000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b0415d280 0x7f6b04152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:21.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.137+0000 7f6b0c000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6b04160840 con 0x7f6b040b2c60 2024-05-27T09:31:21.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.137+0000 7f6b0c000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6b04160560 con 0x7f6b0415d280 2024-05-27T09:31:21.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.137+0000 7f6b0c000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f6b0415d690 con 0x7f6b04073850 2024-05-27T09:31:21.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.137+0000 7f6b08c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b040b2c60 0x7f6b0415aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:21.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.137+0000 7f6b08c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b040b2c60 0x7f6b0415aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37450/0 (socket says 172.21.15.114:37450) 2024-05-27T09:31:21.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.138+0000 7f6b08c00640 1 -- 172.21.15.114:0/4004583429 learned_addr learned my addr 172.21.15.114:0/4004583429 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:31:21.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.137+0000 7f6b09600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b04073850 0x7f6b04153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:21.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.138+0000 7f6b0a000640 1 --2- 172.21.15.114:0/4004583429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b0415d280 0x7f6b04152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:21.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.138+0000 7f6b08c00640 1 -- 172.21.15.114:0/4004583429 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b04073850 msgr2=0x7f6b04153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:21.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.138+0000 7f6b08c00640 1 --2- 172.21.15.114:0/4004583429 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b04073850 0x7f6b04153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:21.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.138+0000 7f6b08c00640 1 -- 172.21.15.114:0/4004583429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b0415d280 msgr2=0x7f6b04152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:21.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.138+0000 7f6b08c00640 1 --2- 172.21.15.114:0/4004583429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b0415d280 0x7f6b04152390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:21.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.138+0000 7f6b08c00640 1 -- 172.21.15.114:0/4004583429 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b041f3a20 con 0x7f6b040b2c60 2024-05-27T09:31:21.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.138+0000 7f6b08c00640 1 --2- 172.21.15.114:0/4004583429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b040b2c60 0x7f6b0415aae0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6af00098c0 tx=0x7f6af00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:21.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.139+0000 7f6b01600640 1 -- 172.21.15.114:0/4004583429 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0037040 con 0x7f6b040b2c60 2024-05-27T09:31:21.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.139+0000 7f6b01600640 1 -- 172.21.15.114:0/4004583429 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6af0030440 con 0x7f6b040b2c60 2024-05-27T09:31:21.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.139+0000 7f6b01600640 1 -- 172.21.15.114:0/4004583429 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0030810 con 0x7f6b040b2c60 2024-05-27T09:31:21.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.139+0000 7f6b0c000640 1 -- 172.21.15.114:0/4004583429 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b04153b50 con 0x7f6b040b2c60 2024-05-27T09:31:21.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.139+0000 7f6b0c000640 1 -- 172.21.15.114:0/4004583429 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b04154b20 con 0x7f6b040b2c60 2024-05-27T09:31:21.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.140+0000 7f6b0c000640 1 -- 172.21.15.114:0/4004583429 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b0415bd10 con 0x7f6b040b2c60 2024-05-27T09:31:21.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.141+0000 7f6b01600640 1 -- 172.21.15.114:0/4004583429 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6af0044020 con 0x7f6b040b2c60 2024-05-27T09:31:21.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.142+0000 7f6b01600640 1 --2- 172.21.15.114:0/4004583429 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6ae40762d0 0x7f6ae40787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:21.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.143+0000 7f6b01600640 1 -- 172.21.15.114:0/4004583429 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5910+0+0 (secure 0 0 0) 0x7f6af00b76d0 con 0x7f6b040b2c60 2024-05-27T09:31:21.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.143+0000 7f6b09600640 1 --2- 172.21.15.114:0/4004583429 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6ae40762d0 0x7f6ae40787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:21.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.144+0000 7f6b09600640 1 --2- 172.21.15.114:0/4004583429 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6ae40762d0 0x7f6ae40787e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6af8000f60 tx=0x7f6af8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:21.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.148+0000 7f6b01600640 1 -- 172.21.15.114:0/4004583429 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6af0080ec0 con 0x7f6b040b2c60 2024-05-27T09:31:21.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.391+0000 7f6b0c000640 1 -- 172.21.15.114:0/4004583429 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b0406a380 con 0x7f6b040b2c60 2024-05-27T09:31:21.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.392+0000 7f6b01600640 1 -- 172.21.15.114:0/4004583429 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+6055 (secure 0 0 0) 0x7f6af00807c0 con 0x7f6b040b2c60 2024-05-27T09:31:21.393 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:31:21.394 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:31:20.888182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4297,4300,4828]},"id":1}]} 2024-05-27T09:31:21.394 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 23 2024-05-27T09:31:21.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.394+0000 7f6b0c000640 1 -- 172.21.15.114:0/4004583429 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6ae40762d0 msgr2=0x7f6ae40787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:21.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.394+0000 7f6b0c000640 1 --2- 172.21.15.114:0/4004583429 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6ae40762d0 0x7f6ae40787e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6af8000f60 tx=0x7f6af8009040 comp rx=0 tx=0).stop 2024-05-27T09:31:21.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.394+0000 7f6b0c000640 1 -- 172.21.15.114:0/4004583429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b040b2c60 msgr2=0x7f6b0415aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:21.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.394+0000 7f6b0c000640 1 --2- 172.21.15.114:0/4004583429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b040b2c60 0x7f6b0415aae0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6af00098c0 tx=0x7f6af00046c0 comp rx=0 tx=0).stop 2024-05-27T09:31:21.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.395+0000 7f6b0c000640 1 -- 172.21.15.114:0/4004583429 shutdown_connections 2024-05-27T09:31:21.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.395+0000 7f6b0c000640 1 --2- 172.21.15.114:0/4004583429 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6ae40762d0 0x7f6ae40787e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:21.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.395+0000 7f6b0c000640 1 --2- 172.21.15.114:0/4004583429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b0415d280 0x7f6b04152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:21.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.395+0000 7f6b0c000640 1 --2- 172.21.15.114:0/4004583429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b040b2c60 0x7f6b0415aae0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:21.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.395+0000 7f6b0c000640 1 --2- 172.21.15.114:0/4004583429 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b04073850 0x7f6b04153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:21.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.395+0000 7f6b0c000640 1 -- 172.21.15.114:0/4004583429 >> 172.21.15.114:0/4004583429 conn(0x7f6b0406f6e0 msgr2=0x7f6b04155500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:21.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.395+0000 7f6b0c000640 1 -- 172.21.15.114:0/4004583429 shutdown_connections 2024-05-27T09:31:21.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:21.395+0000 7f6b0c000640 1 -- 172.21.15.114:0/4004583429 wait complete. 2024-05-27T09:31:21.420 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=2, replacing gid=4256 2024-05-27T09:31:21.421 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 3 secs before reviving mds.a 2024-05-27T09:31:23.049 INFO:tasks.ceph.mds.e.smithi114.stderr:2024-05-27T09:31:23.048+0000 7f4ffb600640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-27T09:31:24.422 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2024-05-27T09:31:24.422 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-27T09:31:24.422 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-27T09:31:24.430 INFO:tasks.ceph.mds.a:Started 2024-05-27T09:31:24.431 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:31:24.659 INFO:tasks.ceph.mds.a.smithi114.stdout:starting mds.a at 2024-05-27T09:31:24.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.683+0000 7f177b400640 1 Processor -- start 2024-05-27T09:31:24.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.683+0000 7f177b400640 1 -- start start 2024-05-27T09:31:24.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.684+0000 7f177b400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1774074a60 0x7f1774154820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:24.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.684+0000 7f177b400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1774073260 0x7f1774073670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:24.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.684+0000 7f177b400640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f177415a810 con 0x7f17740ca2d0 2024-05-27T09:31:24.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.685+0000 7f177b400640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f177415db60 con 0x7f1774073260 2024-05-27T09:31:24.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.685+0000 7f177b400640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f177415aaf0 con 0x7f1774074a60 2024-05-27T09:31:24.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.685+0000 7f176be00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1774073260 0x7f1774073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:24.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.685+0000 7f176be00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1774073260 0x7f1774073670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37476/0 (socket says 172.21.15.114:37476) 2024-05-27T09:31:24.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.685+0000 7f176be00640 1 -- 172.21.15.114:0/238644606 learned_addr learned my addr 172.21.15.114:0/238644606 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:31:24.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.685+0000 7f1779400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f17740ca2d0 0x7f17740ca6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:49302/0 (socket says 172.21.15.114:49302) 2024-05-27T09:31:24.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.685+0000 7f1778a00640 1 --2- 172.21.15.114:0/238644606 >> v2:172.21.15.163:3300/0 conn(0x7f1774074a60 0x7f1774154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:24.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.685+0000 7f176be00640 1 -- 172.21.15.114:0/238644606 >> v2:172.21.15.163:3300/0 conn(0x7f1774074a60 msgr2=0x7f1774154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:31:24.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.685+0000 7f176be00640 1 --2- 172.21.15.114:0/238644606 >> v2:172.21.15.163:3300/0 conn(0x7f1774074a60 0x7f1774154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:24.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.685+0000 7f176be00640 1 -- 172.21.15.114:0/238644606 >> v1:172.21.15.114:6789/0 conn(0x7f17740ca2d0 legacy=0x7f17740ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:24.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.685+0000 7f176be00640 1 -- 172.21.15.114:0/238644606 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177415add0 con 0x7f1774073260 2024-05-27T09:31:24.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.686+0000 7f176be00640 1 --2- 172.21.15.114:0/238644606 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1774073260 0x7f1774073670 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f17740c4ce0 tx=0x7f175c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d9e25afbd2c4fa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:24.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.687+0000 7f1773e00640 1 -- 172.21.15.114:0/238644606 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c037070 con 0x7f1774073260 2024-05-27T09:31:24.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.687+0000 7f1773e00640 1 -- 172.21.15.114:0/238644606 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f175c02ae50 con 0x7f1774073260 2024-05-27T09:31:24.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.687+0000 7f1773e00640 1 -- 172.21.15.114:0/238644606 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c032860 con 0x7f1774073260 2024-05-27T09:31:24.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.687+0000 7f177b400640 1 -- 172.21.15.114:0/238644606 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1774073260 msgr2=0x7f1774073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:24.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.687+0000 7f177b400640 1 --2- 172.21.15.114:0/238644606 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1774073260 0x7f1774073670 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f17740c4ce0 tx=0x7f175c02a300 comp rx=0 tx=0).stop 2024-05-27T09:31:24.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.687+0000 7f177b400640 1 -- 172.21.15.114:0/238644606 shutdown_connections 2024-05-27T09:31:24.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.687+0000 7f177b400640 1 --2- 172.21.15.114:0/238644606 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1774073260 0x7f1774073670 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:24.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.687+0000 7f177b400640 1 --2- 172.21.15.114:0/238644606 >> v2:172.21.15.163:3300/0 conn(0x7f1774074a60 0x7f1774154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:24.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.687+0000 7f177b400640 1 -- 172.21.15.114:0/238644606 >> 172.21.15.114:0/238644606 conn(0x7f177406f6e0 msgr2=0x7f1774070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:24.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.688+0000 7f177b400640 1 -- 172.21.15.114:0/238644606 shutdown_connections 2024-05-27T09:31:24.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.688+0000 7f177b400640 1 -- 172.21.15.114:0/238644606 wait complete. 2024-05-27T09:31:24.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.690+0000 7f177b400640 1 Processor -- start 2024-05-27T09:31:24.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.690+0000 7f177b400640 1 -- start start 2024-05-27T09:31:24.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.691+0000 7f177b400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1774073260 0x7f17740c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:24.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.691+0000 7f177b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1774074a60 0x7f177414a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:24.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.691+0000 7f177b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17740ca2d0 0x7f177414a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:24.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.691+0000 7f177b400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f177415a4e0 con 0x7f1774074a60 2024-05-27T09:31:24.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.691+0000 7f177b400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f177415b390 con 0x7f17740ca2d0 2024-05-27T09:31:24.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.691+0000 7f177b400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f177415dff0 con 0x7f1774073260 2024-05-27T09:31:24.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.692+0000 7f176be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1774074a60 0x7f177414a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:24.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.692+0000 7f176be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1774074a60 0x7f177414a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37488/0 (socket says 172.21.15.114:37488) 2024-05-27T09:31:24.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.692+0000 7f176be00640 1 -- 172.21.15.114:0/3926163294 learned_addr learned my addr 172.21.15.114:0/3926163294 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:31:24.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.692+0000 7f1779400640 1 --2- 172.21.15.114:0/3926163294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17740ca2d0 0x7f177414a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:24.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.692+0000 7f1778a00640 1 --2- 172.21.15.114:0/3926163294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1774073260 0x7f17740c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:24.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.692+0000 7f176be00640 1 -- 172.21.15.114:0/3926163294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1774073260 msgr2=0x7f17740c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:24.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.692+0000 7f176be00640 1 --2- 172.21.15.114:0/3926163294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1774073260 0x7f17740c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:24.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.692+0000 7f176be00640 1 -- 172.21.15.114:0/3926163294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17740ca2d0 msgr2=0x7f177414a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:24.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.692+0000 7f176be00640 1 --2- 172.21.15.114:0/3926163294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17740ca2d0 0x7f177414a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:24.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.692+0000 7f176be00640 1 -- 172.21.15.114:0/3926163294 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1774150200 con 0x7f1774074a60 2024-05-27T09:31:24.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.692+0000 7f176be00640 1 --2- 172.21.15.114:0/3926163294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1774074a60 0x7f177414a1e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f175c000c00 tx=0x7f175c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:24.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.693+0000 7f1771600640 1 -- 172.21.15.114:0/3926163294 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c041020 con 0x7f1774074a60 2024-05-27T09:31:24.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.693+0000 7f1771600640 1 -- 172.21.15.114:0/3926163294 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f175c03c890 con 0x7f1774074a60 2024-05-27T09:31:24.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.693+0000 7f1771600640 1 -- 172.21.15.114:0/3926163294 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c037040 con 0x7f1774074a60 2024-05-27T09:31:24.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.693+0000 7f177b400640 1 -- 172.21.15.114:0/3926163294 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17741f5f90 con 0x7f1774074a60 2024-05-27T09:31:24.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.693+0000 7f177b400640 1 -- 172.21.15.114:0/3926163294 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17740cb880 con 0x7f1774074a60 2024-05-27T09:31:24.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.694+0000 7f177b400640 1 -- 172.21.15.114:0/3926163294 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f177406a380 con 0x7f1774074a60 2024-05-27T09:31:24.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.695+0000 7f1771600640 1 -- 172.21.15.114:0/3926163294 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f175c044020 con 0x7f1774074a60 2024-05-27T09:31:24.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.696+0000 7f1771600640 1 --2- 172.21.15.114:0/3926163294 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1750076300 0x7f1750078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:24.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.696+0000 7f1771600640 1 -- 172.21.15.114:0/3926163294 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5910+0+0 (secure 0 0 0) 0x7f175c0b7dd0 con 0x7f1774074a60 2024-05-27T09:31:24.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.697+0000 7f1778a00640 1 --2- 172.21.15.114:0/3926163294 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1750076300 0x7f1750078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:24.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.697+0000 7f1778a00640 1 --2- 172.21.15.114:0/3926163294 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1750076300 0x7f1750078810 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1764009aa0 tx=0x7f1764009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:24.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.701+0000 7f1771600640 1 -- 172.21.15.114:0/3926163294 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f175c04b030 con 0x7f1774074a60 2024-05-27T09:31:24.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.937+0000 7f177b400640 1 -- 172.21.15.114:0/3926163294 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17740c7c50 con 0x7f1774074a60 2024-05-27T09:31:24.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.938+0000 7f1771600640 1 -- 172.21.15.114:0/3926163294 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+6883 (secure 0 0 0) 0x7f175c0810a0 con 0x7f1774074a60 2024-05-27T09:31:24.940 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:31:24.940 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4875,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:31:20.888182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4828,4300,4297]},"id":1}]} 2024-05-27T09:31:24.940 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 24 2024-05-27T09:31:24.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.941+0000 7f177b400640 1 -- 172.21.15.114:0/3926163294 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1750076300 msgr2=0x7f1750078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:24.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.941+0000 7f177b400640 1 --2- 172.21.15.114:0/3926163294 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1750076300 0x7f1750078810 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1764009aa0 tx=0x7f1764009190 comp rx=0 tx=0).stop 2024-05-27T09:31:24.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.941+0000 7f177b400640 1 -- 172.21.15.114:0/3926163294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1774074a60 msgr2=0x7f177414a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:24.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.941+0000 7f177b400640 1 --2- 172.21.15.114:0/3926163294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1774074a60 0x7f177414a1e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f175c000c00 tx=0x7f175c0046c0 comp rx=0 tx=0).stop 2024-05-27T09:31:24.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.941+0000 7f177b400640 1 -- 172.21.15.114:0/3926163294 shutdown_connections 2024-05-27T09:31:24.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.941+0000 7f177b400640 1 --2- 172.21.15.114:0/3926163294 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1750076300 0x7f1750078810 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:24.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.941+0000 7f177b400640 1 --2- 172.21.15.114:0/3926163294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17740ca2d0 0x7f177414a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:24.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.941+0000 7f177b400640 1 --2- 172.21.15.114:0/3926163294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1774074a60 0x7f177414a1e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:24.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.941+0000 7f177b400640 1 --2- 172.21.15.114:0/3926163294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1774073260 0x7f17740c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:24.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.941+0000 7f177b400640 1 -- 172.21.15.114:0/3926163294 >> 172.21.15.114:0/3926163294 conn(0x7f177406f6e0 msgr2=0x7f17741594f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:24.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.941+0000 7f177b400640 1 -- 172.21.15.114:0/3926163294 shutdown_connections 2024-05-27T09:31:24.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:31:24.942+0000 7f177b400640 1 -- 172.21.15.114:0/3926163294 wait complete. 2024-05-27T09:31:24.968 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2024-05-27T09:31:24.968 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 50 secs before thrashing 2024-05-27T09:31:25.925 INFO:tasks.workunit.client.0.smithi114.stdout:./246/.snap/snap-subdir-test 2024-05-27T09:31:25.938 INFO:tasks.workunit.client.0.smithi114.stdout:./219/.snap/snap-subdir-test 2024-05-27T09:31:25.952 INFO:tasks.workunit.client.0.smithi114.stdout:./198/.snap/snap-subdir-test 2024-05-27T09:31:25.965 INFO:tasks.workunit.client.0.smithi114.stdout:./136/.snap/snap-subdir-test 2024-05-27T09:31:25.977 INFO:tasks.workunit.client.0.smithi114.stdout:./270/.snap/snap-subdir-test 2024-05-27T09:31:25.987 INFO:tasks.workunit.client.0.smithi114.stdout:./125/.snap/snap-subdir-test 2024-05-27T09:31:25.999 INFO:tasks.workunit.client.0.smithi114.stdout:./194/.snap/snap-subdir-test 2024-05-27T09:31:26.008 INFO:tasks.workunit.client.0.smithi114.stdout:./119/.snap/snap-subdir-test 2024-05-27T09:31:26.017 INFO:tasks.workunit.client.0.smithi114.stdout:./297/.snap/snap-subdir-test 2024-05-27T09:31:26.026 INFO:tasks.workunit.client.0.smithi114.stdout:./230/.snap/snap-subdir-test 2024-05-27T09:31:26.035 INFO:tasks.workunit.client.0.smithi114.stdout:./260/.snap/snap-subdir-test 2024-05-27T09:31:26.045 INFO:tasks.workunit.client.0.smithi114.stdout:./157/.snap/snap-subdir-test 2024-05-27T09:31:26.057 INFO:tasks.workunit.client.0.smithi114.stdout:./172/.snap/snap-subdir-test 2024-05-27T09:31:26.067 INFO:tasks.workunit.client.0.smithi114.stdout:./277/.snap/snap-subdir-test 2024-05-27T09:31:26.079 INFO:tasks.workunit.client.0.smithi114.stdout:./186/.snap/snap-subdir-test 2024-05-27T09:31:26.089 INFO:tasks.workunit.client.0.smithi114.stdout:./237/.snap/snap-subdir-test 2024-05-27T09:31:26.098 INFO:tasks.workunit.client.0.smithi114.stdout:./190/.snap/snap-subdir-test 2024-05-27T09:31:26.105 INFO:tasks.workunit.client.0.smithi114.stdout:./213/.snap/snap-subdir-test 2024-05-27T09:31:26.114 INFO:tasks.workunit.client.0.smithi114.stdout:./104/.snap/snap-subdir-test 2024-05-27T09:31:26.126 INFO:tasks.workunit.client.0.smithi114.stdout:./164/.snap/snap-subdir-test 2024-05-27T09:31:26.133 INFO:tasks.workunit.client.0.smithi114.stdout:./112/.snap/snap-subdir-test 2024-05-27T09:31:26.142 INFO:tasks.workunit.client.0.smithi114.stdout:./295/.snap/snap-subdir-test 2024-05-27T09:31:26.152 INFO:tasks.workunit.client.0.smithi114.stdout:./272/.snap/snap-subdir-test 2024-05-27T09:31:26.165 INFO:tasks.workunit.client.0.smithi114.stdout:./100/.snap/snap-subdir-test 2024-05-27T09:31:26.178 INFO:tasks.workunit.client.0.smithi114.stdout:./134/.snap/snap-subdir-test 2024-05-27T09:31:26.188 INFO:tasks.workunit.client.0.smithi114.stdout:./200/.snap/snap-subdir-test 2024-05-27T09:31:26.197 INFO:tasks.workunit.client.0.smithi114.stdout:./115/.snap/snap-subdir-test 2024-05-27T09:31:26.206 INFO:tasks.workunit.client.0.smithi114.stdout:./284/.snap/snap-subdir-test 2024-05-27T09:31:26.217 INFO:tasks.workunit.client.0.smithi114.stdout:./280/.snap/snap-subdir-test 2024-05-27T09:31:26.228 INFO:tasks.workunit.client.0.smithi114.stdout:./116/.snap/snap-subdir-test 2024-05-27T09:31:26.238 INFO:tasks.workunit.client.0.smithi114.stdout:./245/.snap/snap-subdir-test 2024-05-27T09:31:26.248 INFO:tasks.workunit.client.0.smithi114.stdout:./195/.snap/snap-subdir-test 2024-05-27T09:31:26.261 INFO:tasks.workunit.client.0.smithi114.stdout:./138/.snap/snap-subdir-test 2024-05-27T09:31:26.269 INFO:tasks.workunit.client.0.smithi114.stdout:./135/.snap/snap-subdir-test 2024-05-27T09:31:26.277 INFO:tasks.workunit.client.0.smithi114.stdout:./117/.snap/snap-subdir-test 2024-05-27T09:31:26.284 INFO:tasks.workunit.client.0.smithi114.stdout:./241/.snap/snap-subdir-test 2024-05-27T09:31:26.292 INFO:tasks.workunit.client.0.smithi114.stdout:./143/.snap/snap-subdir-test 2024-05-27T09:31:26.300 INFO:tasks.workunit.client.0.smithi114.stdout:./180/.snap/snap-subdir-test 2024-05-27T09:31:26.311 INFO:tasks.workunit.client.0.smithi114.stdout:./126/.snap/snap-subdir-test 2024-05-27T09:31:26.322 INFO:tasks.workunit.client.0.smithi114.stdout:./161/.snap/snap-subdir-test 2024-05-27T09:31:26.333 INFO:tasks.workunit.client.0.smithi114.stdout:./204/.snap/snap-subdir-test 2024-05-27T09:31:26.344 INFO:tasks.workunit.client.0.smithi114.stdout:./176/.snap/snap-subdir-test 2024-05-27T09:31:26.354 INFO:tasks.workunit.client.0.smithi114.stdout:./255/.snap/snap-subdir-test 2024-05-27T09:31:26.366 INFO:tasks.workunit.client.0.smithi114.stdout:./215/.snap/snap-subdir-test 2024-05-27T09:31:26.374 INFO:tasks.workunit.client.0.smithi114.stdout:./111/.snap/snap-subdir-test 2024-05-27T09:31:26.386 INFO:tasks.workunit.client.0.smithi114.stdout:./238/.snap/snap-subdir-test 2024-05-27T09:31:26.393 INFO:tasks.workunit.client.0.smithi114.stdout:./191/.snap/snap-subdir-test 2024-05-27T09:31:26.402 INFO:tasks.workunit.client.0.smithi114.stdout:./184/.snap/snap-subdir-test 2024-05-27T09:31:26.412 INFO:tasks.workunit.client.0.smithi114.stdout:./192/.snap/snap-subdir-test 2024-05-27T09:31:26.421 INFO:tasks.workunit.client.0.smithi114.stdout:./103/.snap/snap-subdir-test 2024-05-27T09:31:26.431 INFO:tasks.workunit.client.0.smithi114.stdout:./128/.snap/snap-subdir-test 2024-05-27T09:31:26.443 INFO:tasks.workunit.client.0.smithi114.stdout:./229/.snap/snap-subdir-test 2024-05-27T09:31:26.454 INFO:tasks.workunit.client.0.smithi114.stdout:./144/.snap/snap-subdir-test 2024-05-27T09:31:26.468 INFO:tasks.workunit.client.0.smithi114.stdout:./146/.snap/snap-subdir-test 2024-05-27T09:31:26.479 INFO:tasks.workunit.client.0.smithi114.stdout:./227/.snap/snap-subdir-test 2024-05-27T09:31:26.489 INFO:tasks.workunit.client.0.smithi114.stdout:./278/.snap/snap-subdir-test 2024-05-27T09:31:26.501 INFO:tasks.workunit.client.0.smithi114.stdout:./141/.snap/snap-subdir-test 2024-05-27T09:31:26.514 INFO:tasks.workunit.client.0.smithi114.stdout:./234/.snap/snap-subdir-test 2024-05-27T09:31:26.525 INFO:tasks.workunit.client.0.smithi114.stdout:./225/.snap/snap-subdir-test 2024-05-27T09:31:26.539 INFO:tasks.workunit.client.0.smithi114.stdout:./182/.snap/snap-subdir-test 2024-05-27T09:31:26.554 INFO:tasks.workunit.client.0.smithi114.stdout:./107/.snap/snap-subdir-test 2024-05-27T09:31:26.568 INFO:tasks.workunit.client.0.smithi114.stdout:./178/.snap/snap-subdir-test 2024-05-27T09:31:26.581 INFO:tasks.workunit.client.0.smithi114.stdout:./266/.snap/snap-subdir-test 2024-05-27T09:31:26.593 INFO:tasks.workunit.client.0.smithi114.stdout:./240/.snap/snap-subdir-test 2024-05-27T09:31:26.602 INFO:tasks.workunit.client.0.smithi114.stdout:./156/.snap/snap-subdir-test 2024-05-27T09:31:26.611 INFO:tasks.workunit.client.0.smithi114.stdout:./131/.snap/snap-subdir-test 2024-05-27T09:31:26.620 INFO:tasks.workunit.client.0.smithi114.stdout:./123/.snap/snap-subdir-test 2024-05-27T09:31:26.628 INFO:tasks.workunit.client.0.smithi114.stdout:./160/.snap/snap-subdir-test 2024-05-27T09:31:26.637 INFO:tasks.workunit.client.0.smithi114.stdout:./165/.snap/snap-subdir-test 2024-05-27T09:31:26.644 INFO:tasks.workunit.client.0.smithi114.stdout:./101/.snap/snap-subdir-test 2024-05-27T09:31:26.655 INFO:tasks.workunit.client.0.smithi114.stdout:./148/.snap/snap-subdir-test 2024-05-27T09:31:26.670 INFO:tasks.workunit.client.0.smithi114.stdout:./168/.snap/snap-subdir-test 2024-05-27T09:31:26.683 INFO:tasks.workunit.client.0.smithi114.stdout:./179/.snap/snap-subdir-test 2024-05-27T09:31:26.696 INFO:tasks.workunit.client.0.smithi114.stdout:./257/.snap/snap-subdir-test 2024-05-27T09:31:26.708 INFO:tasks.workunit.client.0.smithi114.stdout:./252/.snap/snap-subdir-test 2024-05-27T09:31:26.722 INFO:tasks.workunit.client.0.smithi114.stdout:./132/.snap/snap-subdir-test 2024-05-27T09:31:26.734 INFO:tasks.workunit.client.0.smithi114.stdout:./236/.snap/snap-subdir-test 2024-05-27T09:31:26.744 INFO:tasks.workunit.client.0.smithi114.stdout:./211/.snap/snap-subdir-test 2024-05-27T09:31:26.752 INFO:tasks.workunit.client.0.smithi114.stdout:./199/.snap/snap-subdir-test 2024-05-27T09:31:26.759 INFO:tasks.workunit.client.0.smithi114.stdout:./205/.snap/snap-subdir-test 2024-05-27T09:31:26.767 INFO:tasks.workunit.client.0.smithi114.stdout:./163/.snap/snap-subdir-test 2024-05-27T09:31:26.773 INFO:tasks.workunit.client.0.smithi114.stdout:./261/.snap/snap-subdir-test 2024-05-27T09:31:26.784 INFO:tasks.workunit.client.0.smithi114.stdout:./201/.snap/snap-subdir-test 2024-05-27T09:31:26.794 INFO:tasks.workunit.client.0.smithi114.stdout:./282/.snap/snap-subdir-test 2024-05-27T09:31:26.805 INFO:tasks.workunit.client.0.smithi114.stdout:./296/.snap/snap-subdir-test 2024-05-27T09:31:26.812 INFO:tasks.workunit.client.0.smithi114.stdout:./217/.snap/snap-subdir-test 2024-05-27T09:31:26.820 INFO:tasks.workunit.client.0.smithi114.stdout:./153/.snap/snap-subdir-test 2024-05-27T09:31:26.829 INFO:tasks.workunit.client.0.smithi114.stdout:./206/.snap/snap-subdir-test 2024-05-27T09:31:26.841 INFO:tasks.workunit.client.0.smithi114.stdout:./122/.snap/snap-subdir-test 2024-05-27T09:31:26.852 INFO:tasks.workunit.client.0.smithi114.stdout:./162/.snap/snap-subdir-test 2024-05-27T09:31:26.864 INFO:tasks.workunit.client.0.smithi114.stdout:./197/.snap/snap-subdir-test 2024-05-27T09:31:26.873 INFO:tasks.workunit.client.0.smithi114.stdout:./216/.snap/snap-subdir-test 2024-05-27T09:31:27.850 INFO:tasks.workunit.client.0.smithi114.stdout:./254/.snap/snap-subdir-test 2024-05-27T09:31:27.859 INFO:tasks.workunit.client.0.smithi114.stdout:./113/.snap/snap-subdir-test 2024-05-27T09:31:27.873 INFO:tasks.workunit.client.0.smithi114.stdout:./152/.snap/snap-subdir-test 2024-05-27T09:31:27.887 INFO:tasks.workunit.client.0.smithi114.stdout:./121/.snap/snap-subdir-test 2024-05-27T09:31:27.901 INFO:tasks.workunit.client.0.smithi114.stdout:./118/.snap/snap-subdir-test 2024-05-27T09:31:27.914 INFO:tasks.workunit.client.0.smithi114.stdout:./196/.snap/snap-subdir-test 2024-05-27T09:31:27.927 INFO:tasks.workunit.client.0.smithi114.stdout:./124/.snap/snap-subdir-test 2024-05-27T09:31:27.938 INFO:tasks.workunit.client.0.smithi114.stdout:./239/.snap/snap-subdir-test 2024-05-27T09:31:27.951 INFO:tasks.workunit.client.0.smithi114.stdout:./188/.snap/snap-subdir-test 2024-05-27T09:31:27.963 INFO:tasks.workunit.client.0.smithi114.stdout:./287/.snap/snap-subdir-test 2024-05-27T09:31:27.972 INFO:tasks.workunit.client.0.smithi114.stdout:./275/.snap/snap-subdir-test 2024-05-27T09:31:27.981 INFO:tasks.workunit.client.0.smithi114.stdout:./259/.snap/snap-subdir-test 2024-05-27T09:31:27.992 INFO:tasks.workunit.client.0.smithi114.stdout:./224/.snap/snap-subdir-test 2024-05-27T09:31:28.004 INFO:tasks.workunit.client.0.smithi114.stdout:./264/.snap/snap-subdir-test 2024-05-27T09:31:28.017 INFO:tasks.workunit.client.0.smithi114.stdout:./145/.snap/snap-subdir-test 2024-05-27T09:31:28.030 INFO:tasks.workunit.client.0.smithi114.stdout:./279/.snap/snap-subdir-test 2024-05-27T09:31:28.038 INFO:tasks.workunit.client.0.smithi114.stdout:./263/.snap/snap-subdir-test 2024-05-27T09:31:28.047 INFO:tasks.workunit.client.0.smithi114.stdout:./130/.snap/snap-subdir-test 2024-05-27T09:31:28.058 INFO:tasks.workunit.client.0.smithi114.stdout:./193/.snap/snap-subdir-test 2024-05-27T09:31:28.067 INFO:tasks.workunit.client.0.smithi114.stdout:./223/.snap/snap-subdir-test 2024-05-27T09:31:28.074 INFO:tasks.workunit.client.0.smithi114.stdout:./207/.snap/snap-subdir-test 2024-05-27T09:31:28.081 INFO:tasks.workunit.client.0.smithi114.stdout:./276/.snap/snap-subdir-test 2024-05-27T09:31:28.089 INFO:tasks.workunit.client.0.smithi114.stdout:./271/.snap/snap-subdir-test 2024-05-27T09:31:28.097 INFO:tasks.workunit.client.0.smithi114.stdout:./251/.snap/snap-subdir-test 2024-05-27T09:31:28.107 INFO:tasks.workunit.client.0.smithi114.stdout:./175/.snap/snap-subdir-test 2024-05-27T09:31:28.116 INFO:tasks.workunit.client.0.smithi114.stdout:./222/.snap/snap-subdir-test 2024-05-27T09:31:28.125 INFO:tasks.workunit.client.0.smithi114.stdout:./220/.snap/snap-subdir-test 2024-05-27T09:31:28.136 INFO:tasks.workunit.client.0.smithi114.stdout:./203/.snap/snap-subdir-test 2024-05-27T09:31:28.148 INFO:tasks.workunit.client.0.smithi114.stdout:./181/.snap/snap-subdir-test 2024-05-27T09:31:28.157 INFO:tasks.workunit.client.0.smithi114.stdout:./286/.snap/snap-subdir-test 2024-05-27T09:31:28.166 INFO:tasks.workunit.client.0.smithi114.stdout:./127/.snap/snap-subdir-test 2024-05-27T09:31:28.177 INFO:tasks.workunit.client.0.smithi114.stdout:./108/.snap/snap-subdir-test 2024-05-27T09:31:28.189 INFO:tasks.workunit.client.0.smithi114.stdout:./183/.snap/snap-subdir-test 2024-05-27T09:31:28.200 INFO:tasks.workunit.client.0.smithi114.stdout:./294/.snap/snap-subdir-test 2024-05-27T09:31:28.211 INFO:tasks.workunit.client.0.smithi114.stdout:./235/.snap/snap-subdir-test 2024-05-27T09:31:28.225 INFO:tasks.workunit.client.0.smithi114.stdout:./177/.snap/snap-subdir-test 2024-05-27T09:31:28.238 INFO:tasks.workunit.client.0.smithi114.stdout:./169/.snap/snap-subdir-test 2024-05-27T09:31:28.250 INFO:tasks.workunit.client.0.smithi114.stdout:./212/.snap/snap-subdir-test 2024-05-27T09:31:28.263 INFO:tasks.workunit.client.0.smithi114.stdout:./154/.snap/snap-subdir-test 2024-05-27T09:31:28.275 INFO:tasks.workunit.client.0.smithi114.stdout:./171/.snap/snap-subdir-test 2024-05-27T09:31:28.287 INFO:tasks.workunit.client.0.smithi114.stdout:./139/.snap/snap-subdir-test 2024-05-27T09:31:28.298 INFO:tasks.workunit.client.0.smithi114.stdout:./102/.snap/snap-subdir-test 2024-05-27T09:31:28.310 INFO:tasks.workunit.client.0.smithi114.stdout:./129/.snap/snap-subdir-test 2024-05-27T09:31:31.858 INFO:tasks.workunit.client.0.smithi114.stdout:Delete all the files and directories ... 2024-05-27T09:31:33.190 INFO:tasks.workunit.client.0.smithi114.stdout:removed './100/19' 2024-05-27T09:31:33.191 INFO:tasks.workunit.client.0.smithi114.stdout:removed './100/15' 2024-05-27T09:31:33.191 INFO:tasks.workunit.client.0.smithi114.stdout:removed './100/31' 2024-05-27T09:31:33.191 INFO:tasks.workunit.client.0.smithi114.stdout:removed './100/36' 2024-05-27T09:31:33.191 INFO:tasks.workunit.client.0.smithi114.stdout:removed './100/20' 2024-05-27T09:31:33.191 INFO:tasks.workunit.client.0.smithi114.stdout:removed './100/12' 2024-05-27T09:31:33.191 INFO:tasks.workunit.client.0.smithi114.stdout:removed './100/39' 2024-05-27T09:31:33.191 INFO:tasks.workunit.client.0.smithi114.stdout:removed './100/10' 2024-05-27T09:31:33.191 INFO:tasks.workunit.client.0.smithi114.stdout:removed './100/34' 2024-05-27T09:31:33.191 INFO:tasks.workunit.client.0.smithi114.stdout:removed './100/38' 2024-05-27T09:31:33.191 INFO:tasks.workunit.client.0.smithi114.stdout:removed './100/11' 2024-05-27T09:31:33.191 INFO:tasks.workunit.client.0.smithi114.stdout:removed './100/16' 2024-05-27T09:31:33.191 INFO:tasks.workunit.client.0.smithi114.stdout:removed './100/32' 2024-05-27T09:31:33.191 INFO:tasks.workunit.client.0.smithi114.stdout:removed './100/30' 2024-05-27T09:31:33.191 INFO:tasks.workunit.client.0.smithi114.stdout:removed './100/14' 2024-05-27T09:31:33.192 INFO:tasks.workunit.client.0.smithi114.stdout:removed './100/18' 2024-05-27T09:31:33.192 INFO:tasks.workunit.client.0.smithi114.stdout:removed './100/13' 2024-05-27T09:31:33.192 INFO:tasks.workunit.client.0.smithi114.stdout:removed './100/33' 2024-05-27T09:31:33.192 INFO:tasks.workunit.client.0.smithi114.stdout:removed './100/17' 2024-05-27T09:31:33.192 INFO:tasks.workunit.client.0.smithi114.stdout:removed './100/35' 2024-05-27T09:31:33.192 INFO:tasks.workunit.client.0.smithi114.stdout:removed './100/37' 2024-05-27T09:31:33.192 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './100' 2024-05-27T09:31:33.192 INFO:tasks.workunit.client.0.smithi114.stdout:removed './101/19' 2024-05-27T09:31:33.192 INFO:tasks.workunit.client.0.smithi114.stdout:removed './101/15' 2024-05-27T09:31:33.192 INFO:tasks.workunit.client.0.smithi114.stdout:removed './101/31' 2024-05-27T09:31:33.192 INFO:tasks.workunit.client.0.smithi114.stdout:removed './101/36' 2024-05-27T09:31:33.192 INFO:tasks.workunit.client.0.smithi114.stdout:removed './101/20' 2024-05-27T09:31:33.192 INFO:tasks.workunit.client.0.smithi114.stdout:removed './101/12' 2024-05-27T09:31:33.192 INFO:tasks.workunit.client.0.smithi114.stdout:removed './101/39' 2024-05-27T09:31:33.192 INFO:tasks.workunit.client.0.smithi114.stdout:removed './101/10' 2024-05-27T09:31:33.193 INFO:tasks.workunit.client.0.smithi114.stdout:removed './101/34' 2024-05-27T09:31:33.193 INFO:tasks.workunit.client.0.smithi114.stdout:removed './101/38' 2024-05-27T09:31:33.193 INFO:tasks.workunit.client.0.smithi114.stdout:removed './101/11' 2024-05-27T09:31:33.193 INFO:tasks.workunit.client.0.smithi114.stdout:removed './101/16' 2024-05-27T09:31:33.193 INFO:tasks.workunit.client.0.smithi114.stdout:removed './101/32' 2024-05-27T09:31:33.193 INFO:tasks.workunit.client.0.smithi114.stdout:removed './101/30' 2024-05-27T09:31:33.193 INFO:tasks.workunit.client.0.smithi114.stdout:removed './101/14' 2024-05-27T09:31:33.193 INFO:tasks.workunit.client.0.smithi114.stdout:removed './101/18' 2024-05-27T09:31:33.193 INFO:tasks.workunit.client.0.smithi114.stdout:removed './101/13' 2024-05-27T09:31:33.193 INFO:tasks.workunit.client.0.smithi114.stdout:removed './101/33' 2024-05-27T09:31:33.193 INFO:tasks.workunit.client.0.smithi114.stdout:removed './101/17' 2024-05-27T09:31:33.193 INFO:tasks.workunit.client.0.smithi114.stdout:removed './101/35' 2024-05-27T09:31:33.193 INFO:tasks.workunit.client.0.smithi114.stdout:removed './101/37' 2024-05-27T09:31:33.193 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './101' 2024-05-27T09:31:33.193 INFO:tasks.workunit.client.0.smithi114.stdout:removed './102/19' 2024-05-27T09:31:33.194 INFO:tasks.workunit.client.0.smithi114.stdout:removed './102/15' 2024-05-27T09:31:33.194 INFO:tasks.workunit.client.0.smithi114.stdout:removed './102/31' 2024-05-27T09:31:33.194 INFO:tasks.workunit.client.0.smithi114.stdout:removed './102/36' 2024-05-27T09:31:33.194 INFO:tasks.workunit.client.0.smithi114.stdout:removed './102/20' 2024-05-27T09:31:33.194 INFO:tasks.workunit.client.0.smithi114.stdout:removed './102/12' 2024-05-27T09:31:33.194 INFO:tasks.workunit.client.0.smithi114.stdout:removed './102/39' 2024-05-27T09:31:33.194 INFO:tasks.workunit.client.0.smithi114.stdout:removed './102/10' 2024-05-27T09:31:33.194 INFO:tasks.workunit.client.0.smithi114.stdout:removed './102/34' 2024-05-27T09:31:33.194 INFO:tasks.workunit.client.0.smithi114.stdout:removed './102/38' 2024-05-27T09:31:33.194 INFO:tasks.workunit.client.0.smithi114.stdout:removed './102/11' 2024-05-27T09:31:33.194 INFO:tasks.workunit.client.0.smithi114.stdout:removed './102/16' 2024-05-27T09:31:33.194 INFO:tasks.workunit.client.0.smithi114.stdout:removed './102/32' 2024-05-27T09:31:33.194 INFO:tasks.workunit.client.0.smithi114.stdout:removed './102/30' 2024-05-27T09:31:33.194 INFO:tasks.workunit.client.0.smithi114.stdout:removed './102/14' 2024-05-27T09:31:33.195 INFO:tasks.workunit.client.0.smithi114.stdout:removed './102/18' 2024-05-27T09:31:33.195 INFO:tasks.workunit.client.0.smithi114.stdout:removed './102/13' 2024-05-27T09:31:33.195 INFO:tasks.workunit.client.0.smithi114.stdout:removed './102/33' 2024-05-27T09:31:33.195 INFO:tasks.workunit.client.0.smithi114.stdout:removed './102/17' 2024-05-27T09:31:33.195 INFO:tasks.workunit.client.0.smithi114.stdout:removed './102/35' 2024-05-27T09:31:33.195 INFO:tasks.workunit.client.0.smithi114.stdout:removed './102/37' 2024-05-27T09:31:33.195 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './102' 2024-05-27T09:31:33.195 INFO:tasks.workunit.client.0.smithi114.stdout:removed './103/19' 2024-05-27T09:31:33.195 INFO:tasks.workunit.client.0.smithi114.stdout:removed './103/15' 2024-05-27T09:31:33.195 INFO:tasks.workunit.client.0.smithi114.stdout:removed './103/31' 2024-05-27T09:31:33.195 INFO:tasks.workunit.client.0.smithi114.stdout:removed './103/36' 2024-05-27T09:31:33.195 INFO:tasks.workunit.client.0.smithi114.stdout:removed './103/20' 2024-05-27T09:31:33.195 INFO:tasks.workunit.client.0.smithi114.stdout:removed './103/12' 2024-05-27T09:31:33.195 INFO:tasks.workunit.client.0.smithi114.stdout:removed './103/39' 2024-05-27T09:31:33.195 INFO:tasks.workunit.client.0.smithi114.stdout:removed './103/10' 2024-05-27T09:31:33.195 INFO:tasks.workunit.client.0.smithi114.stdout:removed './103/34' 2024-05-27T09:31:33.196 INFO:tasks.workunit.client.0.smithi114.stdout:removed './103/38' 2024-05-27T09:31:33.196 INFO:tasks.workunit.client.0.smithi114.stdout:removed './103/11' 2024-05-27T09:31:33.196 INFO:tasks.workunit.client.0.smithi114.stdout:removed './103/16' 2024-05-27T09:31:33.196 INFO:tasks.workunit.client.0.smithi114.stdout:removed './103/32' 2024-05-27T09:31:33.196 INFO:tasks.workunit.client.0.smithi114.stdout:removed './103/30' 2024-05-27T09:31:33.196 INFO:tasks.workunit.client.0.smithi114.stdout:removed './103/14' 2024-05-27T09:31:33.196 INFO:tasks.workunit.client.0.smithi114.stdout:removed './103/18' 2024-05-27T09:31:33.196 INFO:tasks.workunit.client.0.smithi114.stdout:removed './103/13' 2024-05-27T09:31:33.196 INFO:tasks.workunit.client.0.smithi114.stdout:removed './103/33' 2024-05-27T09:31:33.196 INFO:tasks.workunit.client.0.smithi114.stdout:removed './103/17' 2024-05-27T09:31:33.196 INFO:tasks.workunit.client.0.smithi114.stdout:removed './103/35' 2024-05-27T09:31:33.196 INFO:tasks.workunit.client.0.smithi114.stdout:removed './103/37' 2024-05-27T09:31:33.196 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './103' 2024-05-27T09:31:33.196 INFO:tasks.workunit.client.0.smithi114.stdout:removed './104/19' 2024-05-27T09:31:33.196 INFO:tasks.workunit.client.0.smithi114.stdout:removed './104/15' 2024-05-27T09:31:33.197 INFO:tasks.workunit.client.0.smithi114.stdout:removed './104/31' 2024-05-27T09:31:33.197 INFO:tasks.workunit.client.0.smithi114.stdout:removed './104/36' 2024-05-27T09:31:33.197 INFO:tasks.workunit.client.0.smithi114.stdout:removed './104/20' 2024-05-27T09:31:33.197 INFO:tasks.workunit.client.0.smithi114.stdout:removed './104/12' 2024-05-27T09:31:33.197 INFO:tasks.workunit.client.0.smithi114.stdout:removed './104/39' 2024-05-27T09:31:33.197 INFO:tasks.workunit.client.0.smithi114.stdout:removed './104/10' 2024-05-27T09:31:33.197 INFO:tasks.workunit.client.0.smithi114.stdout:removed './104/34' 2024-05-27T09:31:33.197 INFO:tasks.workunit.client.0.smithi114.stdout:removed './104/38' 2024-05-27T09:31:33.197 INFO:tasks.workunit.client.0.smithi114.stdout:removed './104/11' 2024-05-27T09:31:33.197 INFO:tasks.workunit.client.0.smithi114.stdout:removed './104/16' 2024-05-27T09:31:33.197 INFO:tasks.workunit.client.0.smithi114.stdout:removed './104/32' 2024-05-27T09:31:33.197 INFO:tasks.workunit.client.0.smithi114.stdout:removed './104/30' 2024-05-27T09:31:33.197 INFO:tasks.workunit.client.0.smithi114.stdout:removed './104/14' 2024-05-27T09:31:33.197 INFO:tasks.workunit.client.0.smithi114.stdout:removed './104/18' 2024-05-27T09:31:33.197 INFO:tasks.workunit.client.0.smithi114.stdout:removed './104/13' 2024-05-27T09:31:33.198 INFO:tasks.workunit.client.0.smithi114.stdout:removed './104/33' 2024-05-27T09:31:33.198 INFO:tasks.workunit.client.0.smithi114.stdout:removed './104/17' 2024-05-27T09:31:33.198 INFO:tasks.workunit.client.0.smithi114.stdout:removed './104/35' 2024-05-27T09:31:33.198 INFO:tasks.workunit.client.0.smithi114.stdout:removed './104/37' 2024-05-27T09:31:33.198 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './104' 2024-05-27T09:31:33.198 INFO:tasks.workunit.client.0.smithi114.stdout:removed './105/19' 2024-05-27T09:31:33.198 INFO:tasks.workunit.client.0.smithi114.stdout:removed './105/15' 2024-05-27T09:31:33.198 INFO:tasks.workunit.client.0.smithi114.stdout:removed './105/31' 2024-05-27T09:31:33.198 INFO:tasks.workunit.client.0.smithi114.stdout:removed './105/36' 2024-05-27T09:31:33.198 INFO:tasks.workunit.client.0.smithi114.stdout:removed './105/20' 2024-05-27T09:31:33.198 INFO:tasks.workunit.client.0.smithi114.stdout:removed './105/12' 2024-05-27T09:31:33.198 INFO:tasks.workunit.client.0.smithi114.stdout:removed './105/39' 2024-05-27T09:31:33.198 INFO:tasks.workunit.client.0.smithi114.stdout:removed './105/10' 2024-05-27T09:31:33.198 INFO:tasks.workunit.client.0.smithi114.stdout:removed './105/34' 2024-05-27T09:31:33.199 INFO:tasks.workunit.client.0.smithi114.stdout:removed './105/38' 2024-05-27T09:31:33.199 INFO:tasks.workunit.client.0.smithi114.stdout:removed './105/11' 2024-05-27T09:31:33.199 INFO:tasks.workunit.client.0.smithi114.stdout:removed './105/16' 2024-05-27T09:31:33.199 INFO:tasks.workunit.client.0.smithi114.stdout:removed './105/32' 2024-05-27T09:31:33.199 INFO:tasks.workunit.client.0.smithi114.stdout:removed './105/30' 2024-05-27T09:31:33.199 INFO:tasks.workunit.client.0.smithi114.stdout:removed './105/14' 2024-05-27T09:31:33.199 INFO:tasks.workunit.client.0.smithi114.stdout:removed './105/18' 2024-05-27T09:31:33.199 INFO:tasks.workunit.client.0.smithi114.stdout:removed './105/13' 2024-05-27T09:31:33.199 INFO:tasks.workunit.client.0.smithi114.stdout:removed './105/33' 2024-05-27T09:31:33.199 INFO:tasks.workunit.client.0.smithi114.stdout:removed './105/17' 2024-05-27T09:31:33.199 INFO:tasks.workunit.client.0.smithi114.stdout:removed './105/35' 2024-05-27T09:31:33.199 INFO:tasks.workunit.client.0.smithi114.stdout:removed './105/37' 2024-05-27T09:31:33.199 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './105' 2024-05-27T09:31:33.199 INFO:tasks.workunit.client.0.smithi114.stdout:removed './106/19' 2024-05-27T09:31:33.199 INFO:tasks.workunit.client.0.smithi114.stdout:removed './106/15' 2024-05-27T09:31:33.200 INFO:tasks.workunit.client.0.smithi114.stdout:removed './106/31' 2024-05-27T09:31:33.200 INFO:tasks.workunit.client.0.smithi114.stdout:removed './106/36' 2024-05-27T09:31:33.200 INFO:tasks.workunit.client.0.smithi114.stdout:removed './106/20' 2024-05-27T09:31:33.200 INFO:tasks.workunit.client.0.smithi114.stdout:removed './106/12' 2024-05-27T09:31:33.200 INFO:tasks.workunit.client.0.smithi114.stdout:removed './106/39' 2024-05-27T09:31:33.200 INFO:tasks.workunit.client.0.smithi114.stdout:removed './106/10' 2024-05-27T09:31:33.200 INFO:tasks.workunit.client.0.smithi114.stdout:removed './106/34' 2024-05-27T09:31:33.200 INFO:tasks.workunit.client.0.smithi114.stdout:removed './106/38' 2024-05-27T09:31:33.200 INFO:tasks.workunit.client.0.smithi114.stdout:removed './106/11' 2024-05-27T09:31:33.200 INFO:tasks.workunit.client.0.smithi114.stdout:removed './106/16' 2024-05-27T09:31:33.200 INFO:tasks.workunit.client.0.smithi114.stdout:removed './106/32' 2024-05-27T09:31:33.200 INFO:tasks.workunit.client.0.smithi114.stdout:removed './106/30' 2024-05-27T09:31:33.200 INFO:tasks.workunit.client.0.smithi114.stdout:removed './106/14' 2024-05-27T09:31:33.200 INFO:tasks.workunit.client.0.smithi114.stdout:removed './106/18' 2024-05-27T09:31:33.200 INFO:tasks.workunit.client.0.smithi114.stdout:removed './106/13' 2024-05-27T09:31:33.201 INFO:tasks.workunit.client.0.smithi114.stdout:removed './106/33' 2024-05-27T09:31:33.201 INFO:tasks.workunit.client.0.smithi114.stdout:removed './106/17' 2024-05-27T09:31:33.201 INFO:tasks.workunit.client.0.smithi114.stdout:removed './106/35' 2024-05-27T09:31:33.201 INFO:tasks.workunit.client.0.smithi114.stdout:removed './106/37' 2024-05-27T09:31:33.201 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './106' 2024-05-27T09:31:33.201 INFO:tasks.workunit.client.0.smithi114.stdout:removed './107/19' 2024-05-27T09:31:33.201 INFO:tasks.workunit.client.0.smithi114.stdout:removed './107/15' 2024-05-27T09:31:33.201 INFO:tasks.workunit.client.0.smithi114.stdout:removed './107/31' 2024-05-27T09:31:33.201 INFO:tasks.workunit.client.0.smithi114.stdout:removed './107/36' 2024-05-27T09:31:33.201 INFO:tasks.workunit.client.0.smithi114.stdout:removed './107/20' 2024-05-27T09:31:33.201 INFO:tasks.workunit.client.0.smithi114.stdout:removed './107/12' 2024-05-27T09:31:33.201 INFO:tasks.workunit.client.0.smithi114.stdout:removed './107/39' 2024-05-27T09:31:33.201 INFO:tasks.workunit.client.0.smithi114.stdout:removed './107/10' 2024-05-27T09:31:33.201 INFO:tasks.workunit.client.0.smithi114.stdout:removed './107/34' 2024-05-27T09:31:33.201 INFO:tasks.workunit.client.0.smithi114.stdout:removed './107/38' 2024-05-27T09:31:33.201 INFO:tasks.workunit.client.0.smithi114.stdout:removed './107/11' 2024-05-27T09:31:33.202 INFO:tasks.workunit.client.0.smithi114.stdout:removed './107/16' 2024-05-27T09:31:33.202 INFO:tasks.workunit.client.0.smithi114.stdout:removed './107/32' 2024-05-27T09:31:33.202 INFO:tasks.workunit.client.0.smithi114.stdout:removed './107/30' 2024-05-27T09:31:33.202 INFO:tasks.workunit.client.0.smithi114.stdout:removed './107/14' 2024-05-27T09:31:33.202 INFO:tasks.workunit.client.0.smithi114.stdout:removed './107/18' 2024-05-27T09:31:33.202 INFO:tasks.workunit.client.0.smithi114.stdout:removed './107/13' 2024-05-27T09:31:33.202 INFO:tasks.workunit.client.0.smithi114.stdout:removed './107/33' 2024-05-27T09:31:33.202 INFO:tasks.workunit.client.0.smithi114.stdout:removed './107/17' 2024-05-27T09:31:33.202 INFO:tasks.workunit.client.0.smithi114.stdout:removed './107/35' 2024-05-27T09:31:33.202 INFO:tasks.workunit.client.0.smithi114.stdout:removed './107/37' 2024-05-27T09:31:33.202 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './107' 2024-05-27T09:31:33.202 INFO:tasks.workunit.client.0.smithi114.stdout:removed './108/19' 2024-05-27T09:31:33.202 INFO:tasks.workunit.client.0.smithi114.stdout:removed './108/15' 2024-05-27T09:31:33.202 INFO:tasks.workunit.client.0.smithi114.stdout:removed './108/31' 2024-05-27T09:31:33.203 INFO:tasks.workunit.client.0.smithi114.stdout:removed './108/36' 2024-05-27T09:31:33.203 INFO:tasks.workunit.client.0.smithi114.stdout:removed './108/20' 2024-05-27T09:31:33.203 INFO:tasks.workunit.client.0.smithi114.stdout:removed './108/12' 2024-05-27T09:31:33.203 INFO:tasks.workunit.client.0.smithi114.stdout:removed './108/39' 2024-05-27T09:31:33.203 INFO:tasks.workunit.client.0.smithi114.stdout:removed './108/10' 2024-05-27T09:31:33.203 INFO:tasks.workunit.client.0.smithi114.stdout:removed './108/34' 2024-05-27T09:31:33.203 INFO:tasks.workunit.client.0.smithi114.stdout:removed './108/38' 2024-05-27T09:31:33.203 INFO:tasks.workunit.client.0.smithi114.stdout:removed './108/11' 2024-05-27T09:31:33.203 INFO:tasks.workunit.client.0.smithi114.stdout:removed './108/16' 2024-05-27T09:31:33.203 INFO:tasks.workunit.client.0.smithi114.stdout:removed './108/32' 2024-05-27T09:31:33.203 INFO:tasks.workunit.client.0.smithi114.stdout:removed './108/30' 2024-05-27T09:31:33.203 INFO:tasks.workunit.client.0.smithi114.stdout:removed './108/14' 2024-05-27T09:31:33.203 INFO:tasks.workunit.client.0.smithi114.stdout:removed './108/18' 2024-05-27T09:31:33.203 INFO:tasks.workunit.client.0.smithi114.stdout:removed './108/13' 2024-05-27T09:31:33.203 INFO:tasks.workunit.client.0.smithi114.stdout:removed './108/33' 2024-05-27T09:31:33.204 INFO:tasks.workunit.client.0.smithi114.stdout:removed './108/17' 2024-05-27T09:31:33.204 INFO:tasks.workunit.client.0.smithi114.stdout:removed './108/35' 2024-05-27T09:31:33.204 INFO:tasks.workunit.client.0.smithi114.stdout:removed './108/37' 2024-05-27T09:31:33.204 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './108' 2024-05-27T09:31:33.204 INFO:tasks.workunit.client.0.smithi114.stdout:removed './109/19' 2024-05-27T09:31:33.204 INFO:tasks.workunit.client.0.smithi114.stdout:removed './109/15' 2024-05-27T09:31:33.204 INFO:tasks.workunit.client.0.smithi114.stdout:removed './109/31' 2024-05-27T09:31:33.204 INFO:tasks.workunit.client.0.smithi114.stdout:removed './109/36' 2024-05-27T09:31:33.204 INFO:tasks.workunit.client.0.smithi114.stdout:removed './109/20' 2024-05-27T09:31:33.204 INFO:tasks.workunit.client.0.smithi114.stdout:removed './109/12' 2024-05-27T09:31:33.204 INFO:tasks.workunit.client.0.smithi114.stdout:removed './109/39' 2024-05-27T09:31:33.204 INFO:tasks.workunit.client.0.smithi114.stdout:removed './109/10' 2024-05-27T09:31:33.204 INFO:tasks.workunit.client.0.smithi114.stdout:removed './109/34' 2024-05-27T09:31:33.204 INFO:tasks.workunit.client.0.smithi114.stdout:removed './109/38' 2024-05-27T09:31:33.204 INFO:tasks.workunit.client.0.smithi114.stdout:removed './109/11' 2024-05-27T09:31:33.205 INFO:tasks.workunit.client.0.smithi114.stdout:removed './109/16' 2024-05-27T09:31:33.205 INFO:tasks.workunit.client.0.smithi114.stdout:removed './109/32' 2024-05-27T09:31:33.205 INFO:tasks.workunit.client.0.smithi114.stdout:removed './109/30' 2024-05-27T09:31:34.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed './109/14' 2024-05-27T09:31:34.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed './109/18' 2024-05-27T09:31:34.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed './109/13' 2024-05-27T09:31:34.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed './109/33' 2024-05-27T09:31:34.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed './109/17' 2024-05-27T09:31:34.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed './109/35' 2024-05-27T09:31:34.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed './109/37' 2024-05-27T09:31:34.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './109' 2024-05-27T09:31:34.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed './110/19' 2024-05-27T09:31:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './110/15' 2024-05-27T09:31:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './110/31' 2024-05-27T09:31:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './110/36' 2024-05-27T09:31:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './110/20' 2024-05-27T09:31:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './110/12' 2024-05-27T09:31:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './110/39' 2024-05-27T09:31:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './110/10' 2024-05-27T09:31:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './110/34' 2024-05-27T09:31:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './110/38' 2024-05-27T09:31:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './110/11' 2024-05-27T09:31:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './110/16' 2024-05-27T09:31:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './110/32' 2024-05-27T09:31:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './110/30' 2024-05-27T09:31:34.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './110/14' 2024-05-27T09:31:34.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './110/18' 2024-05-27T09:31:34.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './110/13' 2024-05-27T09:31:34.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './110/33' 2024-05-27T09:31:34.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './110/17' 2024-05-27T09:31:34.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './110/35' 2024-05-27T09:31:34.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './110/37' 2024-05-27T09:31:34.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './110' 2024-05-27T09:31:34.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './111/19' 2024-05-27T09:31:34.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './111/15' 2024-05-27T09:31:34.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './111/31' 2024-05-27T09:31:34.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './111/36' 2024-05-27T09:31:34.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './111/20' 2024-05-27T09:31:34.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './111/12' 2024-05-27T09:31:34.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './111/39' 2024-05-27T09:31:34.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './111/10' 2024-05-27T09:31:34.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './111/34' 2024-05-27T09:31:34.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './111/38' 2024-05-27T09:31:34.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './111/11' 2024-05-27T09:31:34.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './111/16' 2024-05-27T09:31:34.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './111/32' 2024-05-27T09:31:34.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './111/30' 2024-05-27T09:31:34.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './111/14' 2024-05-27T09:31:34.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './111/18' 2024-05-27T09:31:34.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './111/13' 2024-05-27T09:31:34.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './111/33' 2024-05-27T09:31:34.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './111/17' 2024-05-27T09:31:34.535 INFO:tasks.workunit.client.0.smithi114.stdout:removed './111/35' 2024-05-27T09:31:34.535 INFO:tasks.workunit.client.0.smithi114.stdout:removed './111/37' 2024-05-27T09:31:34.535 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './111' 2024-05-27T09:31:34.535 INFO:tasks.workunit.client.0.smithi114.stdout:removed './112/19' 2024-05-27T09:31:34.535 INFO:tasks.workunit.client.0.smithi114.stdout:removed './112/15' 2024-05-27T09:31:34.535 INFO:tasks.workunit.client.0.smithi114.stdout:removed './112/31' 2024-05-27T09:31:34.535 INFO:tasks.workunit.client.0.smithi114.stdout:removed './112/36' 2024-05-27T09:31:34.535 INFO:tasks.workunit.client.0.smithi114.stdout:removed './112/20' 2024-05-27T09:31:34.535 INFO:tasks.workunit.client.0.smithi114.stdout:removed './112/12' 2024-05-27T09:31:34.535 INFO:tasks.workunit.client.0.smithi114.stdout:removed './112/39' 2024-05-27T09:31:34.535 INFO:tasks.workunit.client.0.smithi114.stdout:removed './112/10' 2024-05-27T09:31:34.535 INFO:tasks.workunit.client.0.smithi114.stdout:removed './112/34' 2024-05-27T09:31:34.535 INFO:tasks.workunit.client.0.smithi114.stdout:removed './112/38' 2024-05-27T09:31:34.535 INFO:tasks.workunit.client.0.smithi114.stdout:removed './112/11' 2024-05-27T09:31:34.536 INFO:tasks.workunit.client.0.smithi114.stdout:removed './112/16' 2024-05-27T09:31:34.536 INFO:tasks.workunit.client.0.smithi114.stdout:removed './112/32' 2024-05-27T09:31:34.536 INFO:tasks.workunit.client.0.smithi114.stdout:removed './112/30' 2024-05-27T09:31:34.536 INFO:tasks.workunit.client.0.smithi114.stdout:removed './112/14' 2024-05-27T09:31:34.536 INFO:tasks.workunit.client.0.smithi114.stdout:removed './112/18' 2024-05-27T09:31:34.536 INFO:tasks.workunit.client.0.smithi114.stdout:removed './112/13' 2024-05-27T09:31:34.536 INFO:tasks.workunit.client.0.smithi114.stdout:removed './112/33' 2024-05-27T09:31:34.536 INFO:tasks.workunit.client.0.smithi114.stdout:removed './112/17' 2024-05-27T09:31:34.536 INFO:tasks.workunit.client.0.smithi114.stdout:removed './112/35' 2024-05-27T09:31:34.536 INFO:tasks.workunit.client.0.smithi114.stdout:removed './112/37' 2024-05-27T09:31:34.536 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './112' 2024-05-27T09:31:34.536 INFO:tasks.workunit.client.0.smithi114.stdout:removed './113/19' 2024-05-27T09:31:34.536 INFO:tasks.workunit.client.0.smithi114.stdout:removed './113/15' 2024-05-27T09:31:34.536 INFO:tasks.workunit.client.0.smithi114.stdout:removed './113/31' 2024-05-27T09:31:34.537 INFO:tasks.workunit.client.0.smithi114.stdout:removed './113/36' 2024-05-27T09:31:34.537 INFO:tasks.workunit.client.0.smithi114.stdout:removed './113/20' 2024-05-27T09:31:34.537 INFO:tasks.workunit.client.0.smithi114.stdout:removed './113/12' 2024-05-27T09:31:34.537 INFO:tasks.workunit.client.0.smithi114.stdout:removed './113/39' 2024-05-27T09:31:34.537 INFO:tasks.workunit.client.0.smithi114.stdout:removed './113/10' 2024-05-27T09:31:34.537 INFO:tasks.workunit.client.0.smithi114.stdout:removed './113/34' 2024-05-27T09:31:34.537 INFO:tasks.workunit.client.0.smithi114.stdout:removed './113/38' 2024-05-27T09:31:34.537 INFO:tasks.workunit.client.0.smithi114.stdout:removed './113/11' 2024-05-27T09:31:34.537 INFO:tasks.workunit.client.0.smithi114.stdout:removed './113/16' 2024-05-27T09:31:34.537 INFO:tasks.workunit.client.0.smithi114.stdout:removed './113/32' 2024-05-27T09:31:34.537 INFO:tasks.workunit.client.0.smithi114.stdout:removed './113/30' 2024-05-27T09:31:34.537 INFO:tasks.workunit.client.0.smithi114.stdout:removed './113/14' 2024-05-27T09:31:34.537 INFO:tasks.workunit.client.0.smithi114.stdout:removed './113/18' 2024-05-27T09:31:34.537 INFO:tasks.workunit.client.0.smithi114.stdout:removed './113/13' 2024-05-27T09:31:34.537 INFO:tasks.workunit.client.0.smithi114.stdout:removed './113/33' 2024-05-27T09:31:34.537 INFO:tasks.workunit.client.0.smithi114.stdout:removed './113/17' 2024-05-27T09:31:34.538 INFO:tasks.workunit.client.0.smithi114.stdout:removed './113/35' 2024-05-27T09:31:34.538 INFO:tasks.workunit.client.0.smithi114.stdout:removed './113/37' 2024-05-27T09:31:34.538 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './113' 2024-05-27T09:31:34.538 INFO:tasks.workunit.client.0.smithi114.stdout:removed './114/19' 2024-05-27T09:31:34.538 INFO:tasks.workunit.client.0.smithi114.stdout:removed './114/15' 2024-05-27T09:31:34.538 INFO:tasks.workunit.client.0.smithi114.stdout:removed './114/31' 2024-05-27T09:31:34.538 INFO:tasks.workunit.client.0.smithi114.stdout:removed './114/36' 2024-05-27T09:31:34.538 INFO:tasks.workunit.client.0.smithi114.stdout:removed './114/20' 2024-05-27T09:31:34.538 INFO:tasks.workunit.client.0.smithi114.stdout:removed './114/12' 2024-05-27T09:31:34.538 INFO:tasks.workunit.client.0.smithi114.stdout:removed './114/39' 2024-05-27T09:31:34.538 INFO:tasks.workunit.client.0.smithi114.stdout:removed './114/10' 2024-05-27T09:31:34.538 INFO:tasks.workunit.client.0.smithi114.stdout:removed './114/34' 2024-05-27T09:31:34.538 INFO:tasks.workunit.client.0.smithi114.stdout:removed './114/38' 2024-05-27T09:31:34.538 INFO:tasks.workunit.client.0.smithi114.stdout:removed './114/11' 2024-05-27T09:31:34.538 INFO:tasks.workunit.client.0.smithi114.stdout:removed './114/16' 2024-05-27T09:31:34.539 INFO:tasks.workunit.client.0.smithi114.stdout:removed './114/32' 2024-05-27T09:31:34.539 INFO:tasks.workunit.client.0.smithi114.stdout:removed './114/30' 2024-05-27T09:31:34.539 INFO:tasks.workunit.client.0.smithi114.stdout:removed './114/14' 2024-05-27T09:31:34.539 INFO:tasks.workunit.client.0.smithi114.stdout:removed './114/18' 2024-05-27T09:31:34.539 INFO:tasks.workunit.client.0.smithi114.stdout:removed './114/13' 2024-05-27T09:31:34.539 INFO:tasks.workunit.client.0.smithi114.stdout:removed './114/33' 2024-05-27T09:31:34.539 INFO:tasks.workunit.client.0.smithi114.stdout:removed './114/17' 2024-05-27T09:31:34.539 INFO:tasks.workunit.client.0.smithi114.stdout:removed './114/35' 2024-05-27T09:31:34.539 INFO:tasks.workunit.client.0.smithi114.stdout:removed './114/37' 2024-05-27T09:31:34.539 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './114' 2024-05-27T09:31:34.539 INFO:tasks.workunit.client.0.smithi114.stdout:removed './115/19' 2024-05-27T09:31:34.539 INFO:tasks.workunit.client.0.smithi114.stdout:removed './115/15' 2024-05-27T09:31:34.539 INFO:tasks.workunit.client.0.smithi114.stdout:removed './115/31' 2024-05-27T09:31:34.539 INFO:tasks.workunit.client.0.smithi114.stdout:removed './115/36' 2024-05-27T09:31:34.539 INFO:tasks.workunit.client.0.smithi114.stdout:removed './115/20' 2024-05-27T09:31:34.540 INFO:tasks.workunit.client.0.smithi114.stdout:removed './115/12' 2024-05-27T09:31:34.540 INFO:tasks.workunit.client.0.smithi114.stdout:removed './115/39' 2024-05-27T09:31:34.540 INFO:tasks.workunit.client.0.smithi114.stdout:removed './115/10' 2024-05-27T09:31:34.540 INFO:tasks.workunit.client.0.smithi114.stdout:removed './115/34' 2024-05-27T09:31:34.540 INFO:tasks.workunit.client.0.smithi114.stdout:removed './115/38' 2024-05-27T09:31:34.540 INFO:tasks.workunit.client.0.smithi114.stdout:removed './115/11' 2024-05-27T09:31:34.540 INFO:tasks.workunit.client.0.smithi114.stdout:removed './115/16' 2024-05-27T09:31:34.540 INFO:tasks.workunit.client.0.smithi114.stdout:removed './115/32' 2024-05-27T09:31:34.540 INFO:tasks.workunit.client.0.smithi114.stdout:removed './115/30' 2024-05-27T09:31:34.540 INFO:tasks.workunit.client.0.smithi114.stdout:removed './115/14' 2024-05-27T09:31:34.540 INFO:tasks.workunit.client.0.smithi114.stdout:removed './115/18' 2024-05-27T09:31:34.540 INFO:tasks.workunit.client.0.smithi114.stdout:removed './115/13' 2024-05-27T09:31:34.540 INFO:tasks.workunit.client.0.smithi114.stdout:removed './115/33' 2024-05-27T09:31:34.540 INFO:tasks.workunit.client.0.smithi114.stdout:removed './115/17' 2024-05-27T09:31:34.541 INFO:tasks.workunit.client.0.smithi114.stdout:removed './115/35' 2024-05-27T09:31:34.541 INFO:tasks.workunit.client.0.smithi114.stdout:removed './115/37' 2024-05-27T09:31:34.541 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './115' 2024-05-27T09:31:34.541 INFO:tasks.workunit.client.0.smithi114.stdout:removed './116/19' 2024-05-27T09:31:34.541 INFO:tasks.workunit.client.0.smithi114.stdout:removed './116/15' 2024-05-27T09:31:34.541 INFO:tasks.workunit.client.0.smithi114.stdout:removed './116/31' 2024-05-27T09:31:34.541 INFO:tasks.workunit.client.0.smithi114.stdout:removed './116/36' 2024-05-27T09:31:34.541 INFO:tasks.workunit.client.0.smithi114.stdout:removed './116/20' 2024-05-27T09:31:34.541 INFO:tasks.workunit.client.0.smithi114.stdout:removed './116/12' 2024-05-27T09:31:34.541 INFO:tasks.workunit.client.0.smithi114.stdout:removed './116/39' 2024-05-27T09:31:34.541 INFO:tasks.workunit.client.0.smithi114.stdout:removed './116/10' 2024-05-27T09:31:34.541 INFO:tasks.workunit.client.0.smithi114.stdout:removed './116/34' 2024-05-27T09:31:34.541 INFO:tasks.workunit.client.0.smithi114.stdout:removed './116/38' 2024-05-27T09:31:34.541 INFO:tasks.workunit.client.0.smithi114.stdout:removed './116/11' 2024-05-27T09:31:34.541 INFO:tasks.workunit.client.0.smithi114.stdout:removed './116/16' 2024-05-27T09:31:34.542 INFO:tasks.workunit.client.0.smithi114.stdout:removed './116/32' 2024-05-27T09:31:34.542 INFO:tasks.workunit.client.0.smithi114.stdout:removed './116/30' 2024-05-27T09:31:34.542 INFO:tasks.workunit.client.0.smithi114.stdout:removed './116/14' 2024-05-27T09:31:34.542 INFO:tasks.workunit.client.0.smithi114.stdout:removed './116/18' 2024-05-27T09:31:34.542 INFO:tasks.workunit.client.0.smithi114.stdout:removed './116/13' 2024-05-27T09:31:34.542 INFO:tasks.workunit.client.0.smithi114.stdout:removed './116/33' 2024-05-27T09:31:34.542 INFO:tasks.workunit.client.0.smithi114.stdout:removed './116/17' 2024-05-27T09:31:34.542 INFO:tasks.workunit.client.0.smithi114.stdout:removed './116/35' 2024-05-27T09:31:34.542 INFO:tasks.workunit.client.0.smithi114.stdout:removed './116/37' 2024-05-27T09:31:34.542 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './116' 2024-05-27T09:31:34.542 INFO:tasks.workunit.client.0.smithi114.stdout:removed './117/19' 2024-05-27T09:31:34.542 INFO:tasks.workunit.client.0.smithi114.stdout:removed './117/15' 2024-05-27T09:31:34.542 INFO:tasks.workunit.client.0.smithi114.stdout:removed './117/31' 2024-05-27T09:31:34.542 INFO:tasks.workunit.client.0.smithi114.stdout:removed './117/36' 2024-05-27T09:31:34.542 INFO:tasks.workunit.client.0.smithi114.stdout:removed './117/20' 2024-05-27T09:31:34.543 INFO:tasks.workunit.client.0.smithi114.stdout:removed './117/12' 2024-05-27T09:31:34.543 INFO:tasks.workunit.client.0.smithi114.stdout:removed './117/39' 2024-05-27T09:31:34.543 INFO:tasks.workunit.client.0.smithi114.stdout:removed './117/10' 2024-05-27T09:31:34.543 INFO:tasks.workunit.client.0.smithi114.stdout:removed './117/34' 2024-05-27T09:31:34.543 INFO:tasks.workunit.client.0.smithi114.stdout:removed './117/38' 2024-05-27T09:31:34.543 INFO:tasks.workunit.client.0.smithi114.stdout:removed './117/11' 2024-05-27T09:31:34.543 INFO:tasks.workunit.client.0.smithi114.stdout:removed './117/16' 2024-05-27T09:31:34.543 INFO:tasks.workunit.client.0.smithi114.stdout:removed './117/32' 2024-05-27T09:31:34.543 INFO:tasks.workunit.client.0.smithi114.stdout:removed './117/30' 2024-05-27T09:31:34.543 INFO:tasks.workunit.client.0.smithi114.stdout:removed './117/14' 2024-05-27T09:31:34.543 INFO:tasks.workunit.client.0.smithi114.stdout:removed './117/18' 2024-05-27T09:31:34.543 INFO:tasks.workunit.client.0.smithi114.stdout:removed './117/13' 2024-05-27T09:31:34.543 INFO:tasks.workunit.client.0.smithi114.stdout:removed './117/33' 2024-05-27T09:31:34.543 INFO:tasks.workunit.client.0.smithi114.stdout:removed './117/17' 2024-05-27T09:31:34.543 INFO:tasks.workunit.client.0.smithi114.stdout:removed './117/35' 2024-05-27T09:31:34.543 INFO:tasks.workunit.client.0.smithi114.stdout:removed './117/37' 2024-05-27T09:31:34.544 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './117' 2024-05-27T09:31:34.544 INFO:tasks.workunit.client.0.smithi114.stdout:removed './118/19' 2024-05-27T09:31:34.544 INFO:tasks.workunit.client.0.smithi114.stdout:removed './118/15' 2024-05-27T09:31:34.544 INFO:tasks.workunit.client.0.smithi114.stdout:removed './118/31' 2024-05-27T09:31:34.544 INFO:tasks.workunit.client.0.smithi114.stdout:removed './118/36' 2024-05-27T09:31:34.544 INFO:tasks.workunit.client.0.smithi114.stdout:removed './118/20' 2024-05-27T09:31:34.544 INFO:tasks.workunit.client.0.smithi114.stdout:removed './118/12' 2024-05-27T09:31:34.544 INFO:tasks.workunit.client.0.smithi114.stdout:removed './118/39' 2024-05-27T09:31:34.544 INFO:tasks.workunit.client.0.smithi114.stdout:removed './118/10' 2024-05-27T09:31:34.544 INFO:tasks.workunit.client.0.smithi114.stdout:removed './118/34' 2024-05-27T09:31:34.544 INFO:tasks.workunit.client.0.smithi114.stdout:removed './118/38' 2024-05-27T09:31:34.544 INFO:tasks.workunit.client.0.smithi114.stdout:removed './118/11' 2024-05-27T09:31:34.544 INFO:tasks.workunit.client.0.smithi114.stdout:removed './118/16' 2024-05-27T09:31:34.544 INFO:tasks.workunit.client.0.smithi114.stdout:removed './118/32' 2024-05-27T09:31:34.545 INFO:tasks.workunit.client.0.smithi114.stdout:removed './118/30' 2024-05-27T09:31:34.545 INFO:tasks.workunit.client.0.smithi114.stdout:removed './118/14' 2024-05-27T09:31:34.545 INFO:tasks.workunit.client.0.smithi114.stdout:removed './118/18' 2024-05-27T09:31:34.545 INFO:tasks.workunit.client.0.smithi114.stdout:removed './118/13' 2024-05-27T09:31:34.545 INFO:tasks.workunit.client.0.smithi114.stdout:removed './118/33' 2024-05-27T09:31:34.545 INFO:tasks.workunit.client.0.smithi114.stdout:removed './118/17' 2024-05-27T09:31:34.545 INFO:tasks.workunit.client.0.smithi114.stdout:removed './118/35' 2024-05-27T09:31:34.545 INFO:tasks.workunit.client.0.smithi114.stdout:removed './118/37' 2024-05-27T09:31:34.545 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './118' 2024-05-27T09:31:34.545 INFO:tasks.workunit.client.0.smithi114.stdout:removed './119/19' 2024-05-27T09:31:34.545 INFO:tasks.workunit.client.0.smithi114.stdout:removed './119/15' 2024-05-27T09:31:34.545 INFO:tasks.workunit.client.0.smithi114.stdout:removed './119/31' 2024-05-27T09:31:34.545 INFO:tasks.workunit.client.0.smithi114.stdout:removed './119/36' 2024-05-27T09:31:34.546 INFO:tasks.workunit.client.0.smithi114.stdout:removed './119/20' 2024-05-27T09:31:34.546 INFO:tasks.workunit.client.0.smithi114.stdout:removed './119/12' 2024-05-27T09:31:35.733 INFO:tasks.workunit.client.0.smithi114.stdout:removed './119/39' 2024-05-27T09:31:35.733 INFO:tasks.workunit.client.0.smithi114.stdout:removed './119/10' 2024-05-27T09:31:35.733 INFO:tasks.workunit.client.0.smithi114.stdout:removed './119/34' 2024-05-27T09:31:35.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './119/38' 2024-05-27T09:31:35.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './119/11' 2024-05-27T09:31:35.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './119/16' 2024-05-27T09:31:35.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './119/32' 2024-05-27T09:31:35.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './119/30' 2024-05-27T09:31:35.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './119/14' 2024-05-27T09:31:35.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './119/18' 2024-05-27T09:31:35.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './119/13' 2024-05-27T09:31:35.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './119/33' 2024-05-27T09:31:35.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './119/17' 2024-05-27T09:31:35.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './119/35' 2024-05-27T09:31:35.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './119/37' 2024-05-27T09:31:35.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './119' 2024-05-27T09:31:35.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './120/19' 2024-05-27T09:31:35.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './120/15' 2024-05-27T09:31:35.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './120/31' 2024-05-27T09:31:35.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './120/36' 2024-05-27T09:31:35.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './120/20' 2024-05-27T09:31:35.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './120/12' 2024-05-27T09:31:35.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './120/39' 2024-05-27T09:31:35.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './120/10' 2024-05-27T09:31:35.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './120/34' 2024-05-27T09:31:35.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './120/38' 2024-05-27T09:31:35.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './120/11' 2024-05-27T09:31:35.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './120/16' 2024-05-27T09:31:35.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './120/32' 2024-05-27T09:31:35.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './120/30' 2024-05-27T09:31:35.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './120/14' 2024-05-27T09:31:35.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './120/18' 2024-05-27T09:31:35.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './120/13' 2024-05-27T09:31:35.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './120/33' 2024-05-27T09:31:35.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './120/17' 2024-05-27T09:31:35.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './120/35' 2024-05-27T09:31:35.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './120/37' 2024-05-27T09:31:35.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './120' 2024-05-27T09:31:35.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './121/19' 2024-05-27T09:31:35.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './121/15' 2024-05-27T09:31:35.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './121/31' 2024-05-27T09:31:35.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './121/36' 2024-05-27T09:31:35.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './121/20' 2024-05-27T09:31:35.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './121/12' 2024-05-27T09:31:35.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './121/39' 2024-05-27T09:31:35.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './121/10' 2024-05-27T09:31:35.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './121/34' 2024-05-27T09:31:35.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './121/38' 2024-05-27T09:31:35.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './121/11' 2024-05-27T09:31:35.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './121/16' 2024-05-27T09:31:35.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './121/32' 2024-05-27T09:31:35.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './121/30' 2024-05-27T09:31:35.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './121/14' 2024-05-27T09:31:35.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './121/18' 2024-05-27T09:31:35.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './121/13' 2024-05-27T09:31:35.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './121/33' 2024-05-27T09:31:35.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './121/17' 2024-05-27T09:31:35.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './121/35' 2024-05-27T09:31:35.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './121/37' 2024-05-27T09:31:35.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './121' 2024-05-27T09:31:35.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './122/19' 2024-05-27T09:31:35.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './122/15' 2024-05-27T09:31:35.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './122/31' 2024-05-27T09:31:35.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './122/36' 2024-05-27T09:31:35.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './122/20' 2024-05-27T09:31:35.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './122/12' 2024-05-27T09:31:35.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './122/39' 2024-05-27T09:31:35.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './122/10' 2024-05-27T09:31:35.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './122/34' 2024-05-27T09:31:35.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './122/38' 2024-05-27T09:31:35.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './122/11' 2024-05-27T09:31:35.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './122/16' 2024-05-27T09:31:35.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './122/32' 2024-05-27T09:31:35.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './122/30' 2024-05-27T09:31:35.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './122/14' 2024-05-27T09:31:35.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './122/18' 2024-05-27T09:31:35.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './122/13' 2024-05-27T09:31:35.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './122/33' 2024-05-27T09:31:35.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './122/17' 2024-05-27T09:31:35.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './122/35' 2024-05-27T09:31:35.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './122/37' 2024-05-27T09:31:35.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './122' 2024-05-27T09:31:35.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './123/19' 2024-05-27T09:31:35.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './123/15' 2024-05-27T09:31:35.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './123/31' 2024-05-27T09:31:35.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './123/36' 2024-05-27T09:31:35.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './123/20' 2024-05-27T09:31:35.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './123/12' 2024-05-27T09:31:35.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './123/39' 2024-05-27T09:31:35.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './123/10' 2024-05-27T09:31:35.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './123/34' 2024-05-27T09:31:35.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './123/38' 2024-05-27T09:31:35.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './123/11' 2024-05-27T09:31:35.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './123/16' 2024-05-27T09:31:35.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './123/32' 2024-05-27T09:31:35.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './123/30' 2024-05-27T09:31:35.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './123/14' 2024-05-27T09:31:35.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './123/18' 2024-05-27T09:31:35.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed './123/13' 2024-05-27T09:31:35.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed './123/33' 2024-05-27T09:31:35.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed './123/17' 2024-05-27T09:31:35.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed './123/35' 2024-05-27T09:31:35.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed './123/37' 2024-05-27T09:31:35.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './123' 2024-05-27T09:31:35.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed './124/19' 2024-05-27T09:31:35.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed './124/15' 2024-05-27T09:31:35.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed './124/31' 2024-05-27T09:31:35.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed './124/36' 2024-05-27T09:31:35.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed './124/20' 2024-05-27T09:31:35.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed './124/12' 2024-05-27T09:31:35.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed './124/39' 2024-05-27T09:31:35.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed './124/10' 2024-05-27T09:31:35.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed './124/34' 2024-05-27T09:31:35.742 INFO:tasks.workunit.client.0.smithi114.stdout:removed './124/38' 2024-05-27T09:31:35.742 INFO:tasks.workunit.client.0.smithi114.stdout:removed './124/11' 2024-05-27T09:31:35.742 INFO:tasks.workunit.client.0.smithi114.stdout:removed './124/16' 2024-05-27T09:31:35.742 INFO:tasks.workunit.client.0.smithi114.stdout:removed './124/32' 2024-05-27T09:31:35.742 INFO:tasks.workunit.client.0.smithi114.stdout:removed './124/30' 2024-05-27T09:31:35.742 INFO:tasks.workunit.client.0.smithi114.stdout:removed './124/14' 2024-05-27T09:31:35.742 INFO:tasks.workunit.client.0.smithi114.stdout:removed './124/18' 2024-05-27T09:31:35.742 INFO:tasks.workunit.client.0.smithi114.stdout:removed './124/13' 2024-05-27T09:31:35.742 INFO:tasks.workunit.client.0.smithi114.stdout:removed './124/33' 2024-05-27T09:31:35.742 INFO:tasks.workunit.client.0.smithi114.stdout:removed './124/17' 2024-05-27T09:31:35.742 INFO:tasks.workunit.client.0.smithi114.stdout:removed './124/35' 2024-05-27T09:31:35.742 INFO:tasks.workunit.client.0.smithi114.stdout:removed './124/37' 2024-05-27T09:31:35.742 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './124' 2024-05-27T09:31:35.742 INFO:tasks.workunit.client.0.smithi114.stdout:removed './125/19' 2024-05-27T09:31:35.743 INFO:tasks.workunit.client.0.smithi114.stdout:removed './125/15' 2024-05-27T09:31:35.743 INFO:tasks.workunit.client.0.smithi114.stdout:removed './125/31' 2024-05-27T09:31:35.743 INFO:tasks.workunit.client.0.smithi114.stdout:removed './125/36' 2024-05-27T09:31:35.743 INFO:tasks.workunit.client.0.smithi114.stdout:removed './125/20' 2024-05-27T09:31:35.743 INFO:tasks.workunit.client.0.smithi114.stdout:removed './125/12' 2024-05-27T09:31:35.743 INFO:tasks.workunit.client.0.smithi114.stdout:removed './125/39' 2024-05-27T09:31:35.743 INFO:tasks.workunit.client.0.smithi114.stdout:removed './125/10' 2024-05-27T09:31:35.743 INFO:tasks.workunit.client.0.smithi114.stdout:removed './125/34' 2024-05-27T09:31:35.743 INFO:tasks.workunit.client.0.smithi114.stdout:removed './125/38' 2024-05-27T09:31:35.743 INFO:tasks.workunit.client.0.smithi114.stdout:removed './125/11' 2024-05-27T09:31:35.743 INFO:tasks.workunit.client.0.smithi114.stdout:removed './125/16' 2024-05-27T09:31:35.743 INFO:tasks.workunit.client.0.smithi114.stdout:removed './125/32' 2024-05-27T09:31:35.743 INFO:tasks.workunit.client.0.smithi114.stdout:removed './125/30' 2024-05-27T09:31:35.743 INFO:tasks.workunit.client.0.smithi114.stdout:removed './125/14' 2024-05-27T09:31:35.744 INFO:tasks.workunit.client.0.smithi114.stdout:removed './125/18' 2024-05-27T09:31:35.744 INFO:tasks.workunit.client.0.smithi114.stdout:removed './125/13' 2024-05-27T09:31:35.744 INFO:tasks.workunit.client.0.smithi114.stdout:removed './125/33' 2024-05-27T09:31:35.744 INFO:tasks.workunit.client.0.smithi114.stdout:removed './125/17' 2024-05-27T09:31:35.744 INFO:tasks.workunit.client.0.smithi114.stdout:removed './125/35' 2024-05-27T09:31:35.744 INFO:tasks.workunit.client.0.smithi114.stdout:removed './125/37' 2024-05-27T09:31:35.744 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './125' 2024-05-27T09:31:35.744 INFO:tasks.workunit.client.0.smithi114.stdout:removed './126/19' 2024-05-27T09:31:35.744 INFO:tasks.workunit.client.0.smithi114.stdout:removed './126/15' 2024-05-27T09:31:35.744 INFO:tasks.workunit.client.0.smithi114.stdout:removed './126/31' 2024-05-27T09:31:35.744 INFO:tasks.workunit.client.0.smithi114.stdout:removed './126/36' 2024-05-27T09:31:35.744 INFO:tasks.workunit.client.0.smithi114.stdout:removed './126/20' 2024-05-27T09:31:35.744 INFO:tasks.workunit.client.0.smithi114.stdout:removed './126/12' 2024-05-27T09:31:35.744 INFO:tasks.workunit.client.0.smithi114.stdout:removed './126/39' 2024-05-27T09:31:35.744 INFO:tasks.workunit.client.0.smithi114.stdout:removed './126/10' 2024-05-27T09:31:35.745 INFO:tasks.workunit.client.0.smithi114.stdout:removed './126/34' 2024-05-27T09:31:35.745 INFO:tasks.workunit.client.0.smithi114.stdout:removed './126/38' 2024-05-27T09:31:35.745 INFO:tasks.workunit.client.0.smithi114.stdout:removed './126/11' 2024-05-27T09:31:35.745 INFO:tasks.workunit.client.0.smithi114.stdout:removed './126/16' 2024-05-27T09:31:35.745 INFO:tasks.workunit.client.0.smithi114.stdout:removed './126/32' 2024-05-27T09:31:35.745 INFO:tasks.workunit.client.0.smithi114.stdout:removed './126/30' 2024-05-27T09:31:35.745 INFO:tasks.workunit.client.0.smithi114.stdout:removed './126/14' 2024-05-27T09:31:35.745 INFO:tasks.workunit.client.0.smithi114.stdout:removed './126/18' 2024-05-27T09:31:35.745 INFO:tasks.workunit.client.0.smithi114.stdout:removed './126/13' 2024-05-27T09:31:35.745 INFO:tasks.workunit.client.0.smithi114.stdout:removed './126/33' 2024-05-27T09:31:35.745 INFO:tasks.workunit.client.0.smithi114.stdout:removed './126/17' 2024-05-27T09:31:35.745 INFO:tasks.workunit.client.0.smithi114.stdout:removed './126/35' 2024-05-27T09:31:35.745 INFO:tasks.workunit.client.0.smithi114.stdout:removed './126/37' 2024-05-27T09:31:35.745 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './126' 2024-05-27T09:31:35.745 INFO:tasks.workunit.client.0.smithi114.stdout:removed './127/19' 2024-05-27T09:31:35.745 INFO:tasks.workunit.client.0.smithi114.stdout:removed './127/15' 2024-05-27T09:31:35.746 INFO:tasks.workunit.client.0.smithi114.stdout:removed './127/31' 2024-05-27T09:31:35.746 INFO:tasks.workunit.client.0.smithi114.stdout:removed './127/36' 2024-05-27T09:31:35.746 INFO:tasks.workunit.client.0.smithi114.stdout:removed './127/20' 2024-05-27T09:31:35.746 INFO:tasks.workunit.client.0.smithi114.stdout:removed './127/12' 2024-05-27T09:31:35.746 INFO:tasks.workunit.client.0.smithi114.stdout:removed './127/39' 2024-05-27T09:31:35.746 INFO:tasks.workunit.client.0.smithi114.stdout:removed './127/10' 2024-05-27T09:31:35.746 INFO:tasks.workunit.client.0.smithi114.stdout:removed './127/34' 2024-05-27T09:31:35.746 INFO:tasks.workunit.client.0.smithi114.stdout:removed './127/38' 2024-05-27T09:31:35.746 INFO:tasks.workunit.client.0.smithi114.stdout:removed './127/11' 2024-05-27T09:31:35.746 INFO:tasks.workunit.client.0.smithi114.stdout:removed './127/16' 2024-05-27T09:31:35.746 INFO:tasks.workunit.client.0.smithi114.stdout:removed './127/32' 2024-05-27T09:31:35.746 INFO:tasks.workunit.client.0.smithi114.stdout:removed './127/30' 2024-05-27T09:31:35.746 INFO:tasks.workunit.client.0.smithi114.stdout:removed './127/14' 2024-05-27T09:31:35.746 INFO:tasks.workunit.client.0.smithi114.stdout:removed './127/18' 2024-05-27T09:31:35.747 INFO:tasks.workunit.client.0.smithi114.stdout:removed './127/13' 2024-05-27T09:31:35.747 INFO:tasks.workunit.client.0.smithi114.stdout:removed './127/33' 2024-05-27T09:31:35.747 INFO:tasks.workunit.client.0.smithi114.stdout:removed './127/17' 2024-05-27T09:31:35.747 INFO:tasks.workunit.client.0.smithi114.stdout:removed './127/35' 2024-05-27T09:31:35.747 INFO:tasks.workunit.client.0.smithi114.stdout:removed './127/37' 2024-05-27T09:31:35.747 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './127' 2024-05-27T09:31:35.747 INFO:tasks.workunit.client.0.smithi114.stdout:removed './128/19' 2024-05-27T09:31:35.747 INFO:tasks.workunit.client.0.smithi114.stdout:removed './128/15' 2024-05-27T09:31:35.747 INFO:tasks.workunit.client.0.smithi114.stdout:removed './128/31' 2024-05-27T09:31:35.747 INFO:tasks.workunit.client.0.smithi114.stdout:removed './128/36' 2024-05-27T09:31:35.747 INFO:tasks.workunit.client.0.smithi114.stdout:removed './128/20' 2024-05-27T09:31:35.747 INFO:tasks.workunit.client.0.smithi114.stdout:removed './128/12' 2024-05-27T09:31:35.747 INFO:tasks.workunit.client.0.smithi114.stdout:removed './128/39' 2024-05-27T09:31:35.747 INFO:tasks.workunit.client.0.smithi114.stdout:removed './128/10' 2024-05-27T09:31:35.748 INFO:tasks.workunit.client.0.smithi114.stdout:removed './128/34' 2024-05-27T09:31:35.748 INFO:tasks.workunit.client.0.smithi114.stdout:removed './128/38' 2024-05-27T09:31:35.748 INFO:tasks.workunit.client.0.smithi114.stdout:removed './128/11' 2024-05-27T09:31:35.748 INFO:tasks.workunit.client.0.smithi114.stdout:removed './128/16' 2024-05-27T09:31:35.748 INFO:tasks.workunit.client.0.smithi114.stdout:removed './128/32' 2024-05-27T09:31:35.748 INFO:tasks.workunit.client.0.smithi114.stdout:removed './128/30' 2024-05-27T09:31:35.748 INFO:tasks.workunit.client.0.smithi114.stdout:removed './128/14' 2024-05-27T09:31:35.748 INFO:tasks.workunit.client.0.smithi114.stdout:removed './128/18' 2024-05-27T09:31:35.748 INFO:tasks.workunit.client.0.smithi114.stdout:removed './128/13' 2024-05-27T09:31:35.748 INFO:tasks.workunit.client.0.smithi114.stdout:removed './128/33' 2024-05-27T09:31:35.748 INFO:tasks.workunit.client.0.smithi114.stdout:removed './128/17' 2024-05-27T09:31:35.748 INFO:tasks.workunit.client.0.smithi114.stdout:removed './128/35' 2024-05-27T09:31:36.909 INFO:tasks.workunit.client.0.smithi114.stdout:removed './128/37' 2024-05-27T09:31:36.909 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './128' 2024-05-27T09:31:36.909 INFO:tasks.workunit.client.0.smithi114.stdout:removed './129/19' 2024-05-27T09:31:36.909 INFO:tasks.workunit.client.0.smithi114.stdout:removed './129/15' 2024-05-27T09:31:36.910 INFO:tasks.workunit.client.0.smithi114.stdout:removed './129/31' 2024-05-27T09:31:36.910 INFO:tasks.workunit.client.0.smithi114.stdout:removed './129/36' 2024-05-27T09:31:36.910 INFO:tasks.workunit.client.0.smithi114.stdout:removed './129/20' 2024-05-27T09:31:36.910 INFO:tasks.workunit.client.0.smithi114.stdout:removed './129/12' 2024-05-27T09:31:36.910 INFO:tasks.workunit.client.0.smithi114.stdout:removed './129/39' 2024-05-27T09:31:36.910 INFO:tasks.workunit.client.0.smithi114.stdout:removed './129/10' 2024-05-27T09:31:36.910 INFO:tasks.workunit.client.0.smithi114.stdout:removed './129/34' 2024-05-27T09:31:36.910 INFO:tasks.workunit.client.0.smithi114.stdout:removed './129/38' 2024-05-27T09:31:36.910 INFO:tasks.workunit.client.0.smithi114.stdout:removed './129/11' 2024-05-27T09:31:36.910 INFO:tasks.workunit.client.0.smithi114.stdout:removed './129/16' 2024-05-27T09:31:36.910 INFO:tasks.workunit.client.0.smithi114.stdout:removed './129/32' 2024-05-27T09:31:36.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './129/30' 2024-05-27T09:31:36.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './129/14' 2024-05-27T09:31:36.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './129/18' 2024-05-27T09:31:36.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './129/13' 2024-05-27T09:31:36.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './129/33' 2024-05-27T09:31:36.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './129/17' 2024-05-27T09:31:36.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './129/35' 2024-05-27T09:31:36.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './129/37' 2024-05-27T09:31:36.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './129' 2024-05-27T09:31:36.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './130/19' 2024-05-27T09:31:36.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './130/15' 2024-05-27T09:31:36.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './130/31' 2024-05-27T09:31:36.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './130/36' 2024-05-27T09:31:36.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './130/20' 2024-05-27T09:31:36.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './130/12' 2024-05-27T09:31:36.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './130/39' 2024-05-27T09:31:36.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './130/10' 2024-05-27T09:31:36.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './130/34' 2024-05-27T09:31:36.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './130/38' 2024-05-27T09:31:36.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './130/11' 2024-05-27T09:31:36.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './130/16' 2024-05-27T09:31:36.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './130/32' 2024-05-27T09:31:36.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './130/30' 2024-05-27T09:31:36.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './130/14' 2024-05-27T09:31:36.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './130/18' 2024-05-27T09:31:36.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './130/13' 2024-05-27T09:31:36.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './130/33' 2024-05-27T09:31:36.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './130/17' 2024-05-27T09:31:36.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './130/35' 2024-05-27T09:31:36.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './130/37' 2024-05-27T09:31:36.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './130' 2024-05-27T09:31:36.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './131/19' 2024-05-27T09:31:36.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './131/15' 2024-05-27T09:31:36.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './131/31' 2024-05-27T09:31:36.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './131/36' 2024-05-27T09:31:36.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './131/20' 2024-05-27T09:31:36.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './131/12' 2024-05-27T09:31:36.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './131/39' 2024-05-27T09:31:36.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './131/10' 2024-05-27T09:31:36.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './131/34' 2024-05-27T09:31:36.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './131/38' 2024-05-27T09:31:36.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './131/11' 2024-05-27T09:31:36.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './131/16' 2024-05-27T09:31:36.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './131/32' 2024-05-27T09:31:36.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './131/30' 2024-05-27T09:31:36.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './131/14' 2024-05-27T09:31:36.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './131/18' 2024-05-27T09:31:36.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './131/13' 2024-05-27T09:31:36.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './131/33' 2024-05-27T09:31:36.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './131/17' 2024-05-27T09:31:36.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './131/35' 2024-05-27T09:31:36.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './131/37' 2024-05-27T09:31:36.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './131' 2024-05-27T09:31:36.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './132/19' 2024-05-27T09:31:36.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './132/15' 2024-05-27T09:31:36.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './132/31' 2024-05-27T09:31:36.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './132/36' 2024-05-27T09:31:36.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './132/20' 2024-05-27T09:31:36.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './132/12' 2024-05-27T09:31:36.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './132/39' 2024-05-27T09:31:36.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './132/10' 2024-05-27T09:31:36.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './132/34' 2024-05-27T09:31:36.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './132/38' 2024-05-27T09:31:36.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './132/11' 2024-05-27T09:31:36.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './132/16' 2024-05-27T09:31:36.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './132/32' 2024-05-27T09:31:36.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './132/30' 2024-05-27T09:31:36.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './132/14' 2024-05-27T09:31:36.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './132/18' 2024-05-27T09:31:36.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './132/13' 2024-05-27T09:31:36.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './132/33' 2024-05-27T09:31:36.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './132/17' 2024-05-27T09:31:36.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './132/35' 2024-05-27T09:31:36.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './132/37' 2024-05-27T09:31:36.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './132' 2024-05-27T09:31:36.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './133/19' 2024-05-27T09:31:36.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './133/15' 2024-05-27T09:31:36.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './133/31' 2024-05-27T09:31:36.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './133/36' 2024-05-27T09:31:36.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './133/20' 2024-05-27T09:31:36.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './133/12' 2024-05-27T09:31:36.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './133/39' 2024-05-27T09:31:36.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './133/10' 2024-05-27T09:31:36.917 INFO:tasks.workunit.client.0.smithi114.stdout:removed './133/34' 2024-05-27T09:31:36.917 INFO:tasks.workunit.client.0.smithi114.stdout:removed './133/38' 2024-05-27T09:31:36.917 INFO:tasks.workunit.client.0.smithi114.stdout:removed './133/11' 2024-05-27T09:31:36.917 INFO:tasks.workunit.client.0.smithi114.stdout:removed './133/16' 2024-05-27T09:31:36.917 INFO:tasks.workunit.client.0.smithi114.stdout:removed './133/32' 2024-05-27T09:31:36.917 INFO:tasks.workunit.client.0.smithi114.stdout:removed './133/30' 2024-05-27T09:31:36.917 INFO:tasks.workunit.client.0.smithi114.stdout:removed './133/14' 2024-05-27T09:31:36.917 INFO:tasks.workunit.client.0.smithi114.stdout:removed './133/18' 2024-05-27T09:31:36.917 INFO:tasks.workunit.client.0.smithi114.stdout:removed './133/13' 2024-05-27T09:31:36.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './133/33' 2024-05-27T09:31:36.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './133/17' 2024-05-27T09:31:36.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './133/35' 2024-05-27T09:31:36.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './133/37' 2024-05-27T09:31:36.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './133' 2024-05-27T09:31:36.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './134/19' 2024-05-27T09:31:36.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './134/15' 2024-05-27T09:31:36.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './134/31' 2024-05-27T09:31:36.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './134/36' 2024-05-27T09:31:36.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './134/20' 2024-05-27T09:31:36.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './134/12' 2024-05-27T09:31:36.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './134/39' 2024-05-27T09:31:36.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './134/10' 2024-05-27T09:31:36.919 INFO:tasks.workunit.client.0.smithi114.stdout:removed './134/34' 2024-05-27T09:31:36.919 INFO:tasks.workunit.client.0.smithi114.stdout:removed './134/38' 2024-05-27T09:31:36.919 INFO:tasks.workunit.client.0.smithi114.stdout:removed './134/11' 2024-05-27T09:31:36.919 INFO:tasks.workunit.client.0.smithi114.stdout:removed './134/16' 2024-05-27T09:31:36.919 INFO:tasks.workunit.client.0.smithi114.stdout:removed './134/32' 2024-05-27T09:31:36.919 INFO:tasks.workunit.client.0.smithi114.stdout:removed './134/30' 2024-05-27T09:31:36.919 INFO:tasks.workunit.client.0.smithi114.stdout:removed './134/14' 2024-05-27T09:31:36.919 INFO:tasks.workunit.client.0.smithi114.stdout:removed './134/18' 2024-05-27T09:31:36.919 INFO:tasks.workunit.client.0.smithi114.stdout:removed './134/13' 2024-05-27T09:31:36.919 INFO:tasks.workunit.client.0.smithi114.stdout:removed './134/33' 2024-05-27T09:31:36.919 INFO:tasks.workunit.client.0.smithi114.stdout:removed './134/17' 2024-05-27T09:31:36.919 INFO:tasks.workunit.client.0.smithi114.stdout:removed './134/35' 2024-05-27T09:31:36.919 INFO:tasks.workunit.client.0.smithi114.stdout:removed './134/37' 2024-05-27T09:31:36.919 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './134' 2024-05-27T09:31:36.920 INFO:tasks.workunit.client.0.smithi114.stdout:removed './135/19' 2024-05-27T09:31:36.920 INFO:tasks.workunit.client.0.smithi114.stdout:removed './135/15' 2024-05-27T09:31:36.920 INFO:tasks.workunit.client.0.smithi114.stdout:removed './135/31' 2024-05-27T09:31:36.920 INFO:tasks.workunit.client.0.smithi114.stdout:removed './135/36' 2024-05-27T09:31:36.920 INFO:tasks.workunit.client.0.smithi114.stdout:removed './135/20' 2024-05-27T09:31:36.920 INFO:tasks.workunit.client.0.smithi114.stdout:removed './135/12' 2024-05-27T09:31:36.920 INFO:tasks.workunit.client.0.smithi114.stdout:removed './135/39' 2024-05-27T09:31:36.920 INFO:tasks.workunit.client.0.smithi114.stdout:removed './135/10' 2024-05-27T09:31:36.920 INFO:tasks.workunit.client.0.smithi114.stdout:removed './135/34' 2024-05-27T09:31:36.920 INFO:tasks.workunit.client.0.smithi114.stdout:removed './135/38' 2024-05-27T09:31:36.920 INFO:tasks.workunit.client.0.smithi114.stdout:removed './135/11' 2024-05-27T09:31:36.920 INFO:tasks.workunit.client.0.smithi114.stdout:removed './135/16' 2024-05-27T09:31:36.920 INFO:tasks.workunit.client.0.smithi114.stdout:removed './135/32' 2024-05-27T09:31:36.920 INFO:tasks.workunit.client.0.smithi114.stdout:removed './135/30' 2024-05-27T09:31:36.921 INFO:tasks.workunit.client.0.smithi114.stdout:removed './135/14' 2024-05-27T09:31:36.921 INFO:tasks.workunit.client.0.smithi114.stdout:removed './135/18' 2024-05-27T09:31:36.921 INFO:tasks.workunit.client.0.smithi114.stdout:removed './135/13' 2024-05-27T09:31:36.921 INFO:tasks.workunit.client.0.smithi114.stdout:removed './135/33' 2024-05-27T09:31:36.921 INFO:tasks.workunit.client.0.smithi114.stdout:removed './135/17' 2024-05-27T09:31:36.921 INFO:tasks.workunit.client.0.smithi114.stdout:removed './135/35' 2024-05-27T09:31:36.921 INFO:tasks.workunit.client.0.smithi114.stdout:removed './135/37' 2024-05-27T09:31:36.921 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './135' 2024-05-27T09:31:36.921 INFO:tasks.workunit.client.0.smithi114.stdout:removed './136/19' 2024-05-27T09:31:36.921 INFO:tasks.workunit.client.0.smithi114.stdout:removed './136/15' 2024-05-27T09:31:36.921 INFO:tasks.workunit.client.0.smithi114.stdout:removed './136/31' 2024-05-27T09:31:36.921 INFO:tasks.workunit.client.0.smithi114.stdout:removed './136/36' 2024-05-27T09:31:36.921 INFO:tasks.workunit.client.0.smithi114.stdout:removed './136/20' 2024-05-27T09:31:36.921 INFO:tasks.workunit.client.0.smithi114.stdout:removed './136/12' 2024-05-27T09:31:36.921 INFO:tasks.workunit.client.0.smithi114.stdout:removed './136/39' 2024-05-27T09:31:36.922 INFO:tasks.workunit.client.0.smithi114.stdout:removed './136/10' 2024-05-27T09:31:36.922 INFO:tasks.workunit.client.0.smithi114.stdout:removed './136/34' 2024-05-27T09:31:36.922 INFO:tasks.workunit.client.0.smithi114.stdout:removed './136/38' 2024-05-27T09:31:36.922 INFO:tasks.workunit.client.0.smithi114.stdout:removed './136/11' 2024-05-27T09:31:36.922 INFO:tasks.workunit.client.0.smithi114.stdout:removed './136/16' 2024-05-27T09:31:36.922 INFO:tasks.workunit.client.0.smithi114.stdout:removed './136/32' 2024-05-27T09:31:36.922 INFO:tasks.workunit.client.0.smithi114.stdout:removed './136/30' 2024-05-27T09:31:36.922 INFO:tasks.workunit.client.0.smithi114.stdout:removed './136/14' 2024-05-27T09:31:36.922 INFO:tasks.workunit.client.0.smithi114.stdout:removed './136/18' 2024-05-27T09:31:36.922 INFO:tasks.workunit.client.0.smithi114.stdout:removed './136/13' 2024-05-27T09:31:36.922 INFO:tasks.workunit.client.0.smithi114.stdout:removed './136/33' 2024-05-27T09:31:36.922 INFO:tasks.workunit.client.0.smithi114.stdout:removed './136/17' 2024-05-27T09:31:36.922 INFO:tasks.workunit.client.0.smithi114.stdout:removed './136/35' 2024-05-27T09:31:36.922 INFO:tasks.workunit.client.0.smithi114.stdout:removed './136/37' 2024-05-27T09:31:36.923 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './136' 2024-05-27T09:31:36.923 INFO:tasks.workunit.client.0.smithi114.stdout:removed './137/19' 2024-05-27T09:31:36.923 INFO:tasks.workunit.client.0.smithi114.stdout:removed './137/15' 2024-05-27T09:31:36.923 INFO:tasks.workunit.client.0.smithi114.stdout:removed './137/31' 2024-05-27T09:31:36.923 INFO:tasks.workunit.client.0.smithi114.stdout:removed './137/36' 2024-05-27T09:31:36.923 INFO:tasks.workunit.client.0.smithi114.stdout:removed './137/20' 2024-05-27T09:31:36.923 INFO:tasks.workunit.client.0.smithi114.stdout:removed './137/12' 2024-05-27T09:31:36.923 INFO:tasks.workunit.client.0.smithi114.stdout:removed './137/39' 2024-05-27T09:31:36.923 INFO:tasks.workunit.client.0.smithi114.stdout:removed './137/10' 2024-05-27T09:31:36.923 INFO:tasks.workunit.client.0.smithi114.stdout:removed './137/34' 2024-05-27T09:31:36.923 INFO:tasks.workunit.client.0.smithi114.stdout:removed './137/38' 2024-05-27T09:31:36.923 INFO:tasks.workunit.client.0.smithi114.stdout:removed './137/11' 2024-05-27T09:31:36.923 INFO:tasks.workunit.client.0.smithi114.stdout:removed './137/16' 2024-05-27T09:31:36.923 INFO:tasks.workunit.client.0.smithi114.stdout:removed './137/32' 2024-05-27T09:31:36.924 INFO:tasks.workunit.client.0.smithi114.stdout:removed './137/30' 2024-05-27T09:31:36.924 INFO:tasks.workunit.client.0.smithi114.stdout:removed './137/14' 2024-05-27T09:31:36.924 INFO:tasks.workunit.client.0.smithi114.stdout:removed './137/18' 2024-05-27T09:31:36.924 INFO:tasks.workunit.client.0.smithi114.stdout:removed './137/13' 2024-05-27T09:31:36.924 INFO:tasks.workunit.client.0.smithi114.stdout:removed './137/33' 2024-05-27T09:31:36.924 INFO:tasks.workunit.client.0.smithi114.stdout:removed './137/17' 2024-05-27T09:31:36.924 INFO:tasks.workunit.client.0.smithi114.stdout:removed './137/35' 2024-05-27T09:31:36.924 INFO:tasks.workunit.client.0.smithi114.stdout:removed './137/37' 2024-05-27T09:31:36.924 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './137' 2024-05-27T09:31:36.924 INFO:tasks.workunit.client.0.smithi114.stdout:removed './138/19' 2024-05-27T09:31:36.924 INFO:tasks.workunit.client.0.smithi114.stdout:removed './138/15' 2024-05-27T09:31:36.924 INFO:tasks.workunit.client.0.smithi114.stdout:removed './138/31' 2024-05-27T09:31:36.924 INFO:tasks.workunit.client.0.smithi114.stdout:removed './138/36' 2024-05-27T09:31:36.924 INFO:tasks.workunit.client.0.smithi114.stdout:removed './138/20' 2024-05-27T09:31:36.924 INFO:tasks.workunit.client.0.smithi114.stdout:removed './138/12' 2024-05-27T09:31:36.925 INFO:tasks.workunit.client.0.smithi114.stdout:removed './138/39' 2024-05-27T09:31:36.925 INFO:tasks.workunit.client.0.smithi114.stdout:removed './138/10' 2024-05-27T09:31:36.925 INFO:tasks.workunit.client.0.smithi114.stdout:removed './138/34' 2024-05-27T09:31:36.925 INFO:tasks.workunit.client.0.smithi114.stdout:removed './138/38' 2024-05-27T09:31:36.925 INFO:tasks.workunit.client.0.smithi114.stdout:removed './138/11' 2024-05-27T09:31:36.925 INFO:tasks.workunit.client.0.smithi114.stdout:removed './138/16' 2024-05-27T09:31:37.621 INFO:tasks.workunit.client.0.smithi114.stdout:removed './138/32' 2024-05-27T09:31:37.621 INFO:tasks.workunit.client.0.smithi114.stdout:removed './138/30' 2024-05-27T09:31:37.621 INFO:tasks.workunit.client.0.smithi114.stdout:removed './138/14' 2024-05-27T09:31:37.621 INFO:tasks.workunit.client.0.smithi114.stdout:removed './138/18' 2024-05-27T09:31:37.621 INFO:tasks.workunit.client.0.smithi114.stdout:removed './138/13' 2024-05-27T09:31:37.621 INFO:tasks.workunit.client.0.smithi114.stdout:removed './138/33' 2024-05-27T09:31:37.621 INFO:tasks.workunit.client.0.smithi114.stdout:removed './138/17' 2024-05-27T09:31:37.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './138/35' 2024-05-27T09:31:37.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './138/37' 2024-05-27T09:31:37.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './138' 2024-05-27T09:31:37.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './139/19' 2024-05-27T09:31:37.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './139/15' 2024-05-27T09:31:37.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './139/31' 2024-05-27T09:31:37.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './139/36' 2024-05-27T09:31:37.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './139/20' 2024-05-27T09:31:37.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './139/12' 2024-05-27T09:31:37.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './139/39' 2024-05-27T09:31:37.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './139/10' 2024-05-27T09:31:37.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './139/34' 2024-05-27T09:31:37.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './139/38' 2024-05-27T09:31:37.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './139/11' 2024-05-27T09:31:37.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './139/16' 2024-05-27T09:31:37.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './139/32' 2024-05-27T09:31:37.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './139/30' 2024-05-27T09:31:37.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './139/14' 2024-05-27T09:31:37.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './139/18' 2024-05-27T09:31:37.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './139/13' 2024-05-27T09:31:37.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './139/33' 2024-05-27T09:31:37.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './139/17' 2024-05-27T09:31:37.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './139/35' 2024-05-27T09:31:37.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './139/37' 2024-05-27T09:31:37.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './139' 2024-05-27T09:31:37.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './140/19' 2024-05-27T09:31:37.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './140/15' 2024-05-27T09:31:37.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './140/31' 2024-05-27T09:31:37.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './140/36' 2024-05-27T09:31:37.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './140/20' 2024-05-27T09:31:37.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './140/12' 2024-05-27T09:31:37.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './140/39' 2024-05-27T09:31:37.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './140/10' 2024-05-27T09:31:37.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './140/34' 2024-05-27T09:31:37.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './140/38' 2024-05-27T09:31:37.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './140/11' 2024-05-27T09:31:37.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './140/16' 2024-05-27T09:31:37.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './140/32' 2024-05-27T09:31:37.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './140/30' 2024-05-27T09:31:37.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './140/14' 2024-05-27T09:31:37.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './140/18' 2024-05-27T09:31:37.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './140/13' 2024-05-27T09:31:37.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './140/33' 2024-05-27T09:31:37.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './140/17' 2024-05-27T09:31:37.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './140/35' 2024-05-27T09:31:37.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './140/37' 2024-05-27T09:31:37.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './140' 2024-05-27T09:31:37.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './141/19' 2024-05-27T09:31:37.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './141/15' 2024-05-27T09:31:37.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './141/31' 2024-05-27T09:31:37.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './141/36' 2024-05-27T09:31:37.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './141/20' 2024-05-27T09:31:37.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './141/12' 2024-05-27T09:31:37.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './141/39' 2024-05-27T09:31:37.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './141/10' 2024-05-27T09:31:37.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './141/34' 2024-05-27T09:31:37.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './141/38' 2024-05-27T09:31:37.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './141/11' 2024-05-27T09:31:37.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './141/16' 2024-05-27T09:31:37.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './141/32' 2024-05-27T09:31:37.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './141/30' 2024-05-27T09:31:37.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './141/14' 2024-05-27T09:31:37.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './141/18' 2024-05-27T09:31:37.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './141/13' 2024-05-27T09:31:37.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './141/33' 2024-05-27T09:31:37.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './141/17' 2024-05-27T09:31:37.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './141/35' 2024-05-27T09:31:37.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './141/37' 2024-05-27T09:31:37.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './141' 2024-05-27T09:31:37.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './142/19' 2024-05-27T09:31:37.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './142/15' 2024-05-27T09:31:37.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './142/31' 2024-05-27T09:31:37.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './142/36' 2024-05-27T09:31:37.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './142/20' 2024-05-27T09:31:37.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './142/12' 2024-05-27T09:31:37.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './142/39' 2024-05-27T09:31:37.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './142/10' 2024-05-27T09:31:37.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './142/34' 2024-05-27T09:31:37.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './142/38' 2024-05-27T09:31:37.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './142/11' 2024-05-27T09:31:37.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './142/16' 2024-05-27T09:31:37.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './142/32' 2024-05-27T09:31:37.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './142/30' 2024-05-27T09:31:37.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './142/14' 2024-05-27T09:31:37.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './142/18' 2024-05-27T09:31:37.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './142/13' 2024-05-27T09:31:37.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './142/33' 2024-05-27T09:31:37.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './142/17' 2024-05-27T09:31:37.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './142/35' 2024-05-27T09:31:37.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './142/37' 2024-05-27T09:31:37.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './142' 2024-05-27T09:31:37.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './143/19' 2024-05-27T09:31:37.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './143/15' 2024-05-27T09:31:37.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './143/31' 2024-05-27T09:31:37.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './143/36' 2024-05-27T09:31:37.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './143/20' 2024-05-27T09:31:37.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './143/12' 2024-05-27T09:31:37.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './143/39' 2024-05-27T09:31:37.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './143/10' 2024-05-27T09:31:37.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './143/34' 2024-05-27T09:31:37.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './143/38' 2024-05-27T09:31:37.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './143/11' 2024-05-27T09:31:37.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './143/16' 2024-05-27T09:31:37.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './143/32' 2024-05-27T09:31:37.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './143/30' 2024-05-27T09:31:37.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './143/14' 2024-05-27T09:31:37.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './143/18' 2024-05-27T09:31:37.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './143/13' 2024-05-27T09:31:37.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './143/33' 2024-05-27T09:31:37.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './143/17' 2024-05-27T09:31:37.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './143/35' 2024-05-27T09:31:37.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './143/37' 2024-05-27T09:31:37.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './143' 2024-05-27T09:31:37.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './144/19' 2024-05-27T09:31:37.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './144/15' 2024-05-27T09:31:37.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './144/31' 2024-05-27T09:31:37.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './144/36' 2024-05-27T09:31:37.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './144/20' 2024-05-27T09:31:37.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './144/12' 2024-05-27T09:31:37.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './144/39' 2024-05-27T09:31:37.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './144/10' 2024-05-27T09:31:37.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './144/34' 2024-05-27T09:31:37.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './144/38' 2024-05-27T09:31:37.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './144/11' 2024-05-27T09:31:37.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './144/16' 2024-05-27T09:31:37.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './144/32' 2024-05-27T09:31:37.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './144/30' 2024-05-27T09:31:37.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './144/14' 2024-05-27T09:31:37.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './144/18' 2024-05-27T09:31:37.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './144/13' 2024-05-27T09:31:37.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './144/33' 2024-05-27T09:31:37.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './144/17' 2024-05-27T09:31:37.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './144/35' 2024-05-27T09:31:37.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './144/37' 2024-05-27T09:31:37.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './144' 2024-05-27T09:31:37.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './145/19' 2024-05-27T09:31:37.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './145/15' 2024-05-27T09:31:37.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './145/31' 2024-05-27T09:31:37.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './145/36' 2024-05-27T09:31:37.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './145/20' 2024-05-27T09:31:37.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './145/12' 2024-05-27T09:31:37.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './145/39' 2024-05-27T09:31:37.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './145/10' 2024-05-27T09:31:37.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './145/34' 2024-05-27T09:31:37.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './145/38' 2024-05-27T09:31:37.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './145/11' 2024-05-27T09:31:37.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './145/16' 2024-05-27T09:31:37.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './145/32' 2024-05-27T09:31:37.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './145/30' 2024-05-27T09:31:37.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './145/14' 2024-05-27T09:31:37.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './145/18' 2024-05-27T09:31:37.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './145/13' 2024-05-27T09:31:37.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './145/33' 2024-05-27T09:31:37.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './145/17' 2024-05-27T09:31:37.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './145/35' 2024-05-27T09:31:37.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './145/37' 2024-05-27T09:31:37.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './145' 2024-05-27T09:31:37.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './146/19' 2024-05-27T09:31:37.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './146/15' 2024-05-27T09:31:37.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './146/31' 2024-05-27T09:31:37.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './146/36' 2024-05-27T09:31:37.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './146/20' 2024-05-27T09:31:37.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './146/12' 2024-05-27T09:31:37.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './146/39' 2024-05-27T09:31:37.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './146/10' 2024-05-27T09:31:37.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './146/34' 2024-05-27T09:31:37.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './146/38' 2024-05-27T09:31:37.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './146/11' 2024-05-27T09:31:37.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './146/16' 2024-05-27T09:31:37.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './146/32' 2024-05-27T09:31:37.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './146/30' 2024-05-27T09:31:37.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './146/14' 2024-05-27T09:31:37.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './146/18' 2024-05-27T09:31:37.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './146/13' 2024-05-27T09:31:37.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './146/33' 2024-05-27T09:31:37.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './146/17' 2024-05-27T09:31:37.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './146/35' 2024-05-27T09:31:37.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './146/37' 2024-05-27T09:31:37.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './146' 2024-05-27T09:31:37.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './147/19' 2024-05-27T09:31:37.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './147/15' 2024-05-27T09:31:37.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './147/31' 2024-05-27T09:31:37.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './147/36' 2024-05-27T09:31:37.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './147/20' 2024-05-27T09:31:37.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './147/12' 2024-05-27T09:31:37.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './147/39' 2024-05-27T09:31:37.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './147/10' 2024-05-27T09:31:37.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './147/34' 2024-05-27T09:31:37.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './147/38' 2024-05-27T09:31:37.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './147/11' 2024-05-27T09:31:37.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './147/16' 2024-05-27T09:31:37.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './147/32' 2024-05-27T09:31:37.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './147/30' 2024-05-27T09:31:37.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './147/14' 2024-05-27T09:31:37.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './147/18' 2024-05-27T09:31:37.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './147/13' 2024-05-27T09:31:37.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './147/33' 2024-05-27T09:31:37.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './147/17' 2024-05-27T09:31:37.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './147/35' 2024-05-27T09:31:37.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './147/37' 2024-05-27T09:31:37.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './147' 2024-05-27T09:31:37.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './148/19' 2024-05-27T09:31:37.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './148/15' 2024-05-27T09:31:37.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './148/31' 2024-05-27T09:31:37.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './148/36' 2024-05-27T09:31:38.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './148/20' 2024-05-27T09:31:38.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './148/12' 2024-05-27T09:31:38.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './148/39' 2024-05-27T09:31:38.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './148/10' 2024-05-27T09:31:38.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './148/34' 2024-05-27T09:31:38.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './148/38' 2024-05-27T09:31:38.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './148/11' 2024-05-27T09:31:38.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './148/16' 2024-05-27T09:31:38.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './148/32' 2024-05-27T09:31:38.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './148/30' 2024-05-27T09:31:38.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './148/14' 2024-05-27T09:31:38.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './148/18' 2024-05-27T09:31:38.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './148/13' 2024-05-27T09:31:38.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './148/33' 2024-05-27T09:31:38.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './148/17' 2024-05-27T09:31:38.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './148/35' 2024-05-27T09:31:38.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './148/37' 2024-05-27T09:31:38.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './148' 2024-05-27T09:31:38.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './149/19' 2024-05-27T09:31:38.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './149/15' 2024-05-27T09:31:38.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './149/31' 2024-05-27T09:31:38.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './149/36' 2024-05-27T09:31:38.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './149/20' 2024-05-27T09:31:38.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './149/12' 2024-05-27T09:31:38.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './149/39' 2024-05-27T09:31:38.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './149/10' 2024-05-27T09:31:38.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './149/34' 2024-05-27T09:31:38.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './149/38' 2024-05-27T09:31:38.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './149/11' 2024-05-27T09:31:38.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './149/16' 2024-05-27T09:31:38.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './149/32' 2024-05-27T09:31:38.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './149/30' 2024-05-27T09:31:38.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './149/14' 2024-05-27T09:31:38.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './149/18' 2024-05-27T09:31:38.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './149/13' 2024-05-27T09:31:38.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './149/33' 2024-05-27T09:31:38.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './149/17' 2024-05-27T09:31:38.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './149/35' 2024-05-27T09:31:38.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './149/37' 2024-05-27T09:31:38.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './149' 2024-05-27T09:31:38.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './150/19' 2024-05-27T09:31:38.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './150/15' 2024-05-27T09:31:38.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './150/31' 2024-05-27T09:31:38.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './150/36' 2024-05-27T09:31:38.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './150/20' 2024-05-27T09:31:38.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './150/12' 2024-05-27T09:31:38.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './150/39' 2024-05-27T09:31:38.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './150/10' 2024-05-27T09:31:38.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './150/34' 2024-05-27T09:31:38.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './150/38' 2024-05-27T09:31:38.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './150/11' 2024-05-27T09:31:38.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './150/16' 2024-05-27T09:31:38.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './150/32' 2024-05-27T09:31:38.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './150/30' 2024-05-27T09:31:38.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './150/14' 2024-05-27T09:31:38.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './150/18' 2024-05-27T09:31:38.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './150/13' 2024-05-27T09:31:38.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './150/33' 2024-05-27T09:31:38.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './150/17' 2024-05-27T09:31:38.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './150/35' 2024-05-27T09:31:38.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './150/37' 2024-05-27T09:31:38.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './150' 2024-05-27T09:31:38.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './151/19' 2024-05-27T09:31:38.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './151/15' 2024-05-27T09:31:38.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './151/31' 2024-05-27T09:31:38.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './151/36' 2024-05-27T09:31:38.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './151/20' 2024-05-27T09:31:38.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './151/12' 2024-05-27T09:31:38.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './151/39' 2024-05-27T09:31:38.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './151/10' 2024-05-27T09:31:38.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './151/34' 2024-05-27T09:31:38.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './151/38' 2024-05-27T09:31:38.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './151/11' 2024-05-27T09:31:38.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './151/16' 2024-05-27T09:31:38.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './151/32' 2024-05-27T09:31:38.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './151/30' 2024-05-27T09:31:38.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './151/14' 2024-05-27T09:31:38.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './151/18' 2024-05-27T09:31:38.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './151/13' 2024-05-27T09:31:38.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './151/33' 2024-05-27T09:31:38.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './151/17' 2024-05-27T09:31:38.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './151/35' 2024-05-27T09:31:38.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './151/37' 2024-05-27T09:31:38.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './151' 2024-05-27T09:31:38.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './152/19' 2024-05-27T09:31:38.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './152/15' 2024-05-27T09:31:38.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './152/31' 2024-05-27T09:31:38.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './152/36' 2024-05-27T09:31:38.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './152/20' 2024-05-27T09:31:38.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './152/12' 2024-05-27T09:31:38.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './152/39' 2024-05-27T09:31:38.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './152/10' 2024-05-27T09:31:38.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './152/34' 2024-05-27T09:31:38.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './152/38' 2024-05-27T09:31:38.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './152/11' 2024-05-27T09:31:38.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './152/16' 2024-05-27T09:31:38.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './152/32' 2024-05-27T09:31:38.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './152/30' 2024-05-27T09:31:38.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './152/14' 2024-05-27T09:31:38.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './152/18' 2024-05-27T09:31:38.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './152/13' 2024-05-27T09:31:38.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './152/33' 2024-05-27T09:31:38.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './152/17' 2024-05-27T09:31:38.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './152/35' 2024-05-27T09:31:38.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './152/37' 2024-05-27T09:31:38.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './152' 2024-05-27T09:31:38.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './153/19' 2024-05-27T09:31:38.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './153/15' 2024-05-27T09:31:38.611 INFO:tasks.workunit.client.0.smithi114.stdout:removed './153/31' 2024-05-27T09:31:38.611 INFO:tasks.workunit.client.0.smithi114.stdout:removed './153/36' 2024-05-27T09:31:38.611 INFO:tasks.workunit.client.0.smithi114.stdout:removed './153/20' 2024-05-27T09:31:38.611 INFO:tasks.workunit.client.0.smithi114.stdout:removed './153/12' 2024-05-27T09:31:38.611 INFO:tasks.workunit.client.0.smithi114.stdout:removed './153/39' 2024-05-27T09:31:38.611 INFO:tasks.workunit.client.0.smithi114.stdout:removed './153/10' 2024-05-27T09:31:38.611 INFO:tasks.workunit.client.0.smithi114.stdout:removed './153/34' 2024-05-27T09:31:38.611 INFO:tasks.workunit.client.0.smithi114.stdout:removed './153/38' 2024-05-27T09:31:38.611 INFO:tasks.workunit.client.0.smithi114.stdout:removed './153/11' 2024-05-27T09:31:38.611 INFO:tasks.workunit.client.0.smithi114.stdout:removed './153/16' 2024-05-27T09:31:38.611 INFO:tasks.workunit.client.0.smithi114.stdout:removed './153/32' 2024-05-27T09:31:38.611 INFO:tasks.workunit.client.0.smithi114.stdout:removed './153/30' 2024-05-27T09:31:38.611 INFO:tasks.workunit.client.0.smithi114.stdout:removed './153/14' 2024-05-27T09:31:38.611 INFO:tasks.workunit.client.0.smithi114.stdout:removed './153/18' 2024-05-27T09:31:38.612 INFO:tasks.workunit.client.0.smithi114.stdout:removed './153/13' 2024-05-27T09:31:38.612 INFO:tasks.workunit.client.0.smithi114.stdout:removed './153/33' 2024-05-27T09:31:38.612 INFO:tasks.workunit.client.0.smithi114.stdout:removed './153/17' 2024-05-27T09:31:38.612 INFO:tasks.workunit.client.0.smithi114.stdout:removed './153/35' 2024-05-27T09:31:38.612 INFO:tasks.workunit.client.0.smithi114.stdout:removed './153/37' 2024-05-27T09:31:38.612 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './153' 2024-05-27T09:31:38.612 INFO:tasks.workunit.client.0.smithi114.stdout:removed './154/19' 2024-05-27T09:31:38.612 INFO:tasks.workunit.client.0.smithi114.stdout:removed './154/15' 2024-05-27T09:31:38.612 INFO:tasks.workunit.client.0.smithi114.stdout:removed './154/31' 2024-05-27T09:31:38.612 INFO:tasks.workunit.client.0.smithi114.stdout:removed './154/36' 2024-05-27T09:31:38.612 INFO:tasks.workunit.client.0.smithi114.stdout:removed './154/20' 2024-05-27T09:31:38.612 INFO:tasks.workunit.client.0.smithi114.stdout:removed './154/12' 2024-05-27T09:31:38.612 INFO:tasks.workunit.client.0.smithi114.stdout:removed './154/39' 2024-05-27T09:31:38.612 INFO:tasks.workunit.client.0.smithi114.stdout:removed './154/10' 2024-05-27T09:31:38.612 INFO:tasks.workunit.client.0.smithi114.stdout:removed './154/34' 2024-05-27T09:31:38.613 INFO:tasks.workunit.client.0.smithi114.stdout:removed './154/38' 2024-05-27T09:31:38.613 INFO:tasks.workunit.client.0.smithi114.stdout:removed './154/11' 2024-05-27T09:31:38.613 INFO:tasks.workunit.client.0.smithi114.stdout:removed './154/16' 2024-05-27T09:31:38.613 INFO:tasks.workunit.client.0.smithi114.stdout:removed './154/32' 2024-05-27T09:31:38.613 INFO:tasks.workunit.client.0.smithi114.stdout:removed './154/30' 2024-05-27T09:31:38.613 INFO:tasks.workunit.client.0.smithi114.stdout:removed './154/14' 2024-05-27T09:31:38.613 INFO:tasks.workunit.client.0.smithi114.stdout:removed './154/18' 2024-05-27T09:31:38.613 INFO:tasks.workunit.client.0.smithi114.stdout:removed './154/13' 2024-05-27T09:31:38.613 INFO:tasks.workunit.client.0.smithi114.stdout:removed './154/33' 2024-05-27T09:31:38.613 INFO:tasks.workunit.client.0.smithi114.stdout:removed './154/17' 2024-05-27T09:31:38.613 INFO:tasks.workunit.client.0.smithi114.stdout:removed './154/35' 2024-05-27T09:31:38.613 INFO:tasks.workunit.client.0.smithi114.stdout:removed './154/37' 2024-05-27T09:31:38.613 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './154' 2024-05-27T09:31:38.613 INFO:tasks.workunit.client.0.smithi114.stdout:removed './155/19' 2024-05-27T09:31:38.613 INFO:tasks.workunit.client.0.smithi114.stdout:removed './155/15' 2024-05-27T09:31:38.614 INFO:tasks.workunit.client.0.smithi114.stdout:removed './155/31' 2024-05-27T09:31:38.614 INFO:tasks.workunit.client.0.smithi114.stdout:removed './155/36' 2024-05-27T09:31:38.614 INFO:tasks.workunit.client.0.smithi114.stdout:removed './155/20' 2024-05-27T09:31:38.614 INFO:tasks.workunit.client.0.smithi114.stdout:removed './155/12' 2024-05-27T09:31:38.614 INFO:tasks.workunit.client.0.smithi114.stdout:removed './155/39' 2024-05-27T09:31:38.614 INFO:tasks.workunit.client.0.smithi114.stdout:removed './155/10' 2024-05-27T09:31:38.614 INFO:tasks.workunit.client.0.smithi114.stdout:removed './155/34' 2024-05-27T09:31:38.614 INFO:tasks.workunit.client.0.smithi114.stdout:removed './155/38' 2024-05-27T09:31:38.614 INFO:tasks.workunit.client.0.smithi114.stdout:removed './155/11' 2024-05-27T09:31:38.614 INFO:tasks.workunit.client.0.smithi114.stdout:removed './155/16' 2024-05-27T09:31:38.614 INFO:tasks.workunit.client.0.smithi114.stdout:removed './155/32' 2024-05-27T09:31:38.614 INFO:tasks.workunit.client.0.smithi114.stdout:removed './155/30' 2024-05-27T09:31:38.614 INFO:tasks.workunit.client.0.smithi114.stdout:removed './155/14' 2024-05-27T09:31:38.615 INFO:tasks.workunit.client.0.smithi114.stdout:removed './155/18' 2024-05-27T09:31:38.615 INFO:tasks.workunit.client.0.smithi114.stdout:removed './155/13' 2024-05-27T09:31:38.615 INFO:tasks.workunit.client.0.smithi114.stdout:removed './155/33' 2024-05-27T09:31:38.615 INFO:tasks.workunit.client.0.smithi114.stdout:removed './155/17' 2024-05-27T09:31:38.615 INFO:tasks.workunit.client.0.smithi114.stdout:removed './155/35' 2024-05-27T09:31:38.615 INFO:tasks.workunit.client.0.smithi114.stdout:removed './155/37' 2024-05-27T09:31:38.615 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './155' 2024-05-27T09:31:38.615 INFO:tasks.workunit.client.0.smithi114.stdout:removed './156/19' 2024-05-27T09:31:38.615 INFO:tasks.workunit.client.0.smithi114.stdout:removed './156/15' 2024-05-27T09:31:38.615 INFO:tasks.workunit.client.0.smithi114.stdout:removed './156/31' 2024-05-27T09:31:38.615 INFO:tasks.workunit.client.0.smithi114.stdout:removed './156/36' 2024-05-27T09:31:38.615 INFO:tasks.workunit.client.0.smithi114.stdout:removed './156/20' 2024-05-27T09:31:38.615 INFO:tasks.workunit.client.0.smithi114.stdout:removed './156/12' 2024-05-27T09:31:38.615 INFO:tasks.workunit.client.0.smithi114.stdout:removed './156/39' 2024-05-27T09:31:38.616 INFO:tasks.workunit.client.0.smithi114.stdout:removed './156/10' 2024-05-27T09:31:38.616 INFO:tasks.workunit.client.0.smithi114.stdout:removed './156/34' 2024-05-27T09:31:38.616 INFO:tasks.workunit.client.0.smithi114.stdout:removed './156/38' 2024-05-27T09:31:38.616 INFO:tasks.workunit.client.0.smithi114.stdout:removed './156/11' 2024-05-27T09:31:38.616 INFO:tasks.workunit.client.0.smithi114.stdout:removed './156/16' 2024-05-27T09:31:38.616 INFO:tasks.workunit.client.0.smithi114.stdout:removed './156/32' 2024-05-27T09:31:38.616 INFO:tasks.workunit.client.0.smithi114.stdout:removed './156/30' 2024-05-27T09:31:38.616 INFO:tasks.workunit.client.0.smithi114.stdout:removed './156/14' 2024-05-27T09:31:38.616 INFO:tasks.workunit.client.0.smithi114.stdout:removed './156/18' 2024-05-27T09:31:38.616 INFO:tasks.workunit.client.0.smithi114.stdout:removed './156/13' 2024-05-27T09:31:38.616 INFO:tasks.workunit.client.0.smithi114.stdout:removed './156/33' 2024-05-27T09:31:38.616 INFO:tasks.workunit.client.0.smithi114.stdout:removed './156/17' 2024-05-27T09:31:38.616 INFO:tasks.workunit.client.0.smithi114.stdout:removed './156/35' 2024-05-27T09:31:38.616 INFO:tasks.workunit.client.0.smithi114.stdout:removed './156/37' 2024-05-27T09:31:38.617 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './156' 2024-05-27T09:31:38.617 INFO:tasks.workunit.client.0.smithi114.stdout:removed './157/19' 2024-05-27T09:31:38.617 INFO:tasks.workunit.client.0.smithi114.stdout:removed './157/15' 2024-05-27T09:31:38.617 INFO:tasks.workunit.client.0.smithi114.stdout:removed './157/31' 2024-05-27T09:31:38.617 INFO:tasks.workunit.client.0.smithi114.stdout:removed './157/36' 2024-05-27T09:31:38.617 INFO:tasks.workunit.client.0.smithi114.stdout:removed './157/20' 2024-05-27T09:31:38.617 INFO:tasks.workunit.client.0.smithi114.stdout:removed './157/12' 2024-05-27T09:31:38.617 INFO:tasks.workunit.client.0.smithi114.stdout:removed './157/39' 2024-05-27T09:31:38.617 INFO:tasks.workunit.client.0.smithi114.stdout:removed './157/10' 2024-05-27T09:31:38.617 INFO:tasks.workunit.client.0.smithi114.stdout:removed './157/34' 2024-05-27T09:31:38.617 INFO:tasks.workunit.client.0.smithi114.stdout:removed './157/38' 2024-05-27T09:31:38.617 INFO:tasks.workunit.client.0.smithi114.stdout:removed './157/11' 2024-05-27T09:31:38.617 INFO:tasks.workunit.client.0.smithi114.stdout:removed './157/16' 2024-05-27T09:31:38.617 INFO:tasks.workunit.client.0.smithi114.stdout:removed './157/32' 2024-05-27T09:31:38.618 INFO:tasks.workunit.client.0.smithi114.stdout:removed './157/30' 2024-05-27T09:31:38.618 INFO:tasks.workunit.client.0.smithi114.stdout:removed './157/14' 2024-05-27T09:31:38.618 INFO:tasks.workunit.client.0.smithi114.stdout:removed './157/18' 2024-05-27T09:31:38.618 INFO:tasks.workunit.client.0.smithi114.stdout:removed './157/13' 2024-05-27T09:31:38.618 INFO:tasks.workunit.client.0.smithi114.stdout:removed './157/33' 2024-05-27T09:31:39.708 INFO:tasks.workunit.client.0.smithi114.stdout:removed './157/17' 2024-05-27T09:31:39.708 INFO:tasks.workunit.client.0.smithi114.stdout:removed './157/35' 2024-05-27T09:31:39.708 INFO:tasks.workunit.client.0.smithi114.stdout:removed './157/37' 2024-05-27T09:31:39.708 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './157' 2024-05-27T09:31:39.708 INFO:tasks.workunit.client.0.smithi114.stdout:removed './158/19' 2024-05-27T09:31:39.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './158/15' 2024-05-27T09:31:39.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './158/31' 2024-05-27T09:31:39.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './158/36' 2024-05-27T09:31:39.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './158/20' 2024-05-27T09:31:39.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './158/12' 2024-05-27T09:31:39.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './158/39' 2024-05-27T09:31:39.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './158/10' 2024-05-27T09:31:39.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './158/34' 2024-05-27T09:31:39.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './158/38' 2024-05-27T09:31:39.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './158/11' 2024-05-27T09:31:39.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './158/16' 2024-05-27T09:31:39.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './158/32' 2024-05-27T09:31:39.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './158/30' 2024-05-27T09:31:39.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './158/14' 2024-05-27T09:31:39.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './158/18' 2024-05-27T09:31:39.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './158/13' 2024-05-27T09:31:39.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './158/33' 2024-05-27T09:31:39.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './158/17' 2024-05-27T09:31:39.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './158/35' 2024-05-27T09:31:39.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './158/37' 2024-05-27T09:31:39.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './158' 2024-05-27T09:31:39.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './159/19' 2024-05-27T09:31:39.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './159/15' 2024-05-27T09:31:39.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './159/31' 2024-05-27T09:31:39.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './159/36' 2024-05-27T09:31:39.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './159/20' 2024-05-27T09:31:39.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './159/12' 2024-05-27T09:31:39.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './159/39' 2024-05-27T09:31:39.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './159/10' 2024-05-27T09:31:39.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './159/34' 2024-05-27T09:31:39.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './159/38' 2024-05-27T09:31:39.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './159/11' 2024-05-27T09:31:39.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './159/16' 2024-05-27T09:31:39.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './159/32' 2024-05-27T09:31:39.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './159/30' 2024-05-27T09:31:39.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './159/14' 2024-05-27T09:31:39.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './159/18' 2024-05-27T09:31:39.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './159/13' 2024-05-27T09:31:39.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './159/33' 2024-05-27T09:31:39.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './159/17' 2024-05-27T09:31:39.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './159/35' 2024-05-27T09:31:39.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './159/37' 2024-05-27T09:31:39.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './159' 2024-05-27T09:31:39.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './160/19' 2024-05-27T09:31:39.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './160/15' 2024-05-27T09:31:39.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './160/31' 2024-05-27T09:31:39.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './160/36' 2024-05-27T09:31:39.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './160/20' 2024-05-27T09:31:39.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './160/12' 2024-05-27T09:31:39.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './160/39' 2024-05-27T09:31:39.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './160/10' 2024-05-27T09:31:39.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './160/34' 2024-05-27T09:31:39.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './160/38' 2024-05-27T09:31:39.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './160/11' 2024-05-27T09:31:39.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './160/16' 2024-05-27T09:31:39.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './160/32' 2024-05-27T09:31:39.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './160/30' 2024-05-27T09:31:39.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './160/14' 2024-05-27T09:31:39.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './160/18' 2024-05-27T09:31:39.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './160/13' 2024-05-27T09:31:39.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './160/33' 2024-05-27T09:31:39.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './160/17' 2024-05-27T09:31:39.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './160/35' 2024-05-27T09:31:39.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './160/37' 2024-05-27T09:31:39.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './160' 2024-05-27T09:31:39.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './161/19' 2024-05-27T09:31:39.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './161/15' 2024-05-27T09:31:39.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './161/31' 2024-05-27T09:31:39.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './161/36' 2024-05-27T09:31:39.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './161/20' 2024-05-27T09:31:39.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './161/12' 2024-05-27T09:31:39.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './161/39' 2024-05-27T09:31:39.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './161/10' 2024-05-27T09:31:39.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './161/34' 2024-05-27T09:31:39.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './161/38' 2024-05-27T09:31:39.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './161/11' 2024-05-27T09:31:39.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './161/16' 2024-05-27T09:31:39.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './161/32' 2024-05-27T09:31:39.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './161/30' 2024-05-27T09:31:39.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './161/14' 2024-05-27T09:31:39.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './161/18' 2024-05-27T09:31:39.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './161/13' 2024-05-27T09:31:39.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './161/33' 2024-05-27T09:31:39.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './161/17' 2024-05-27T09:31:39.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './161/35' 2024-05-27T09:31:39.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './161/37' 2024-05-27T09:31:39.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './161' 2024-05-27T09:31:39.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './162/19' 2024-05-27T09:31:39.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './162/15' 2024-05-27T09:31:39.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './162/31' 2024-05-27T09:31:39.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './162/36' 2024-05-27T09:31:39.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './162/20' 2024-05-27T09:31:39.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './162/12' 2024-05-27T09:31:39.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './162/39' 2024-05-27T09:31:39.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './162/10' 2024-05-27T09:31:39.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './162/34' 2024-05-27T09:31:39.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './162/38' 2024-05-27T09:31:39.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './162/11' 2024-05-27T09:31:39.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './162/16' 2024-05-27T09:31:39.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './162/32' 2024-05-27T09:31:39.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './162/30' 2024-05-27T09:31:39.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './162/14' 2024-05-27T09:31:39.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './162/18' 2024-05-27T09:31:39.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './162/13' 2024-05-27T09:31:39.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './162/33' 2024-05-27T09:31:39.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './162/17' 2024-05-27T09:31:39.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './162/35' 2024-05-27T09:31:39.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './162/37' 2024-05-27T09:31:39.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './162' 2024-05-27T09:31:39.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './163/19' 2024-05-27T09:31:39.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './163/15' 2024-05-27T09:31:39.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './163/31' 2024-05-27T09:31:39.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './163/36' 2024-05-27T09:31:39.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './163/20' 2024-05-27T09:31:39.717 INFO:tasks.workunit.client.0.smithi114.stdout:removed './163/12' 2024-05-27T09:31:39.717 INFO:tasks.workunit.client.0.smithi114.stdout:removed './163/39' 2024-05-27T09:31:39.717 INFO:tasks.workunit.client.0.smithi114.stdout:removed './163/10' 2024-05-27T09:31:39.717 INFO:tasks.workunit.client.0.smithi114.stdout:removed './163/34' 2024-05-27T09:31:39.717 INFO:tasks.workunit.client.0.smithi114.stdout:removed './163/38' 2024-05-27T09:31:39.717 INFO:tasks.workunit.client.0.smithi114.stdout:removed './163/11' 2024-05-27T09:31:39.717 INFO:tasks.workunit.client.0.smithi114.stdout:removed './163/16' 2024-05-27T09:31:39.717 INFO:tasks.workunit.client.0.smithi114.stdout:removed './163/32' 2024-05-27T09:31:39.717 INFO:tasks.workunit.client.0.smithi114.stdout:removed './163/30' 2024-05-27T09:31:39.717 INFO:tasks.workunit.client.0.smithi114.stdout:removed './163/14' 2024-05-27T09:31:39.717 INFO:tasks.workunit.client.0.smithi114.stdout:removed './163/18' 2024-05-27T09:31:39.717 INFO:tasks.workunit.client.0.smithi114.stdout:removed './163/13' 2024-05-27T09:31:39.717 INFO:tasks.workunit.client.0.smithi114.stdout:removed './163/33' 2024-05-27T09:31:39.717 INFO:tasks.workunit.client.0.smithi114.stdout:removed './163/17' 2024-05-27T09:31:39.718 INFO:tasks.workunit.client.0.smithi114.stdout:removed './163/35' 2024-05-27T09:31:39.718 INFO:tasks.workunit.client.0.smithi114.stdout:removed './163/37' 2024-05-27T09:31:39.718 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './163' 2024-05-27T09:31:39.718 INFO:tasks.workunit.client.0.smithi114.stdout:removed './164/19' 2024-05-27T09:31:39.718 INFO:tasks.workunit.client.0.smithi114.stdout:removed './164/15' 2024-05-27T09:31:39.718 INFO:tasks.workunit.client.0.smithi114.stdout:removed './164/31' 2024-05-27T09:31:39.718 INFO:tasks.workunit.client.0.smithi114.stdout:removed './164/36' 2024-05-27T09:31:39.718 INFO:tasks.workunit.client.0.smithi114.stdout:removed './164/20' 2024-05-27T09:31:39.718 INFO:tasks.workunit.client.0.smithi114.stdout:removed './164/12' 2024-05-27T09:31:39.718 INFO:tasks.workunit.client.0.smithi114.stdout:removed './164/39' 2024-05-27T09:31:39.718 INFO:tasks.workunit.client.0.smithi114.stdout:removed './164/10' 2024-05-27T09:31:39.718 INFO:tasks.workunit.client.0.smithi114.stdout:removed './164/34' 2024-05-27T09:31:39.718 INFO:tasks.workunit.client.0.smithi114.stdout:removed './164/38' 2024-05-27T09:31:39.719 INFO:tasks.workunit.client.0.smithi114.stdout:removed './164/11' 2024-05-27T09:31:39.719 INFO:tasks.workunit.client.0.smithi114.stdout:removed './164/16' 2024-05-27T09:31:39.719 INFO:tasks.workunit.client.0.smithi114.stdout:removed './164/32' 2024-05-27T09:31:39.719 INFO:tasks.workunit.client.0.smithi114.stdout:removed './164/30' 2024-05-27T09:31:39.719 INFO:tasks.workunit.client.0.smithi114.stdout:removed './164/14' 2024-05-27T09:31:39.719 INFO:tasks.workunit.client.0.smithi114.stdout:removed './164/18' 2024-05-27T09:31:39.719 INFO:tasks.workunit.client.0.smithi114.stdout:removed './164/13' 2024-05-27T09:31:39.719 INFO:tasks.workunit.client.0.smithi114.stdout:removed './164/33' 2024-05-27T09:31:39.719 INFO:tasks.workunit.client.0.smithi114.stdout:removed './164/17' 2024-05-27T09:31:39.719 INFO:tasks.workunit.client.0.smithi114.stdout:removed './164/35' 2024-05-27T09:31:39.719 INFO:tasks.workunit.client.0.smithi114.stdout:removed './164/37' 2024-05-27T09:31:39.719 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './164' 2024-05-27T09:31:39.719 INFO:tasks.workunit.client.0.smithi114.stdout:removed './165/19' 2024-05-27T09:31:39.720 INFO:tasks.workunit.client.0.smithi114.stdout:removed './165/15' 2024-05-27T09:31:39.720 INFO:tasks.workunit.client.0.smithi114.stdout:removed './165/31' 2024-05-27T09:31:39.720 INFO:tasks.workunit.client.0.smithi114.stdout:removed './165/36' 2024-05-27T09:31:39.720 INFO:tasks.workunit.client.0.smithi114.stdout:removed './165/20' 2024-05-27T09:31:39.720 INFO:tasks.workunit.client.0.smithi114.stdout:removed './165/12' 2024-05-27T09:31:39.720 INFO:tasks.workunit.client.0.smithi114.stdout:removed './165/39' 2024-05-27T09:31:39.720 INFO:tasks.workunit.client.0.smithi114.stdout:removed './165/10' 2024-05-27T09:31:39.720 INFO:tasks.workunit.client.0.smithi114.stdout:removed './165/34' 2024-05-27T09:31:39.720 INFO:tasks.workunit.client.0.smithi114.stdout:removed './165/38' 2024-05-27T09:31:39.720 INFO:tasks.workunit.client.0.smithi114.stdout:removed './165/11' 2024-05-27T09:31:39.720 INFO:tasks.workunit.client.0.smithi114.stdout:removed './165/16' 2024-05-27T09:31:39.720 INFO:tasks.workunit.client.0.smithi114.stdout:removed './165/32' 2024-05-27T09:31:39.720 INFO:tasks.workunit.client.0.smithi114.stdout:removed './165/30' 2024-05-27T09:31:39.720 INFO:tasks.workunit.client.0.smithi114.stdout:removed './165/14' 2024-05-27T09:31:39.720 INFO:tasks.workunit.client.0.smithi114.stdout:removed './165/18' 2024-05-27T09:31:39.720 INFO:tasks.workunit.client.0.smithi114.stdout:removed './165/13' 2024-05-27T09:31:39.721 INFO:tasks.workunit.client.0.smithi114.stdout:removed './165/33' 2024-05-27T09:31:39.721 INFO:tasks.workunit.client.0.smithi114.stdout:removed './165/17' 2024-05-27T09:31:39.721 INFO:tasks.workunit.client.0.smithi114.stdout:removed './165/35' 2024-05-27T09:31:39.721 INFO:tasks.workunit.client.0.smithi114.stdout:removed './165/37' 2024-05-27T09:31:39.721 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './165' 2024-05-27T09:31:39.721 INFO:tasks.workunit.client.0.smithi114.stdout:removed './166/19' 2024-05-27T09:31:39.721 INFO:tasks.workunit.client.0.smithi114.stdout:removed './166/15' 2024-05-27T09:31:39.721 INFO:tasks.workunit.client.0.smithi114.stdout:removed './166/31' 2024-05-27T09:31:39.721 INFO:tasks.workunit.client.0.smithi114.stdout:removed './166/36' 2024-05-27T09:31:39.721 INFO:tasks.workunit.client.0.smithi114.stdout:removed './166/20' 2024-05-27T09:31:39.721 INFO:tasks.workunit.client.0.smithi114.stdout:removed './166/12' 2024-05-27T09:31:39.721 INFO:tasks.workunit.client.0.smithi114.stdout:removed './166/39' 2024-05-27T09:31:39.721 INFO:tasks.workunit.client.0.smithi114.stdout:removed './166/10' 2024-05-27T09:31:39.721 INFO:tasks.workunit.client.0.smithi114.stdout:removed './166/34' 2024-05-27T09:31:39.721 INFO:tasks.workunit.client.0.smithi114.stdout:removed './166/38' 2024-05-27T09:31:39.722 INFO:tasks.workunit.client.0.smithi114.stdout:removed './166/11' 2024-05-27T09:31:39.722 INFO:tasks.workunit.client.0.smithi114.stdout:removed './166/16' 2024-05-27T09:31:39.722 INFO:tasks.workunit.client.0.smithi114.stdout:removed './166/32' 2024-05-27T09:31:39.722 INFO:tasks.workunit.client.0.smithi114.stdout:removed './166/30' 2024-05-27T09:31:39.722 INFO:tasks.workunit.client.0.smithi114.stdout:removed './166/14' 2024-05-27T09:31:39.722 INFO:tasks.workunit.client.0.smithi114.stdout:removed './166/18' 2024-05-27T09:31:39.722 INFO:tasks.workunit.client.0.smithi114.stdout:removed './166/13' 2024-05-27T09:31:39.722 INFO:tasks.workunit.client.0.smithi114.stdout:removed './166/33' 2024-05-27T09:31:39.722 INFO:tasks.workunit.client.0.smithi114.stdout:removed './166/17' 2024-05-27T09:31:39.722 INFO:tasks.workunit.client.0.smithi114.stdout:removed './166/35' 2024-05-27T09:31:39.722 INFO:tasks.workunit.client.0.smithi114.stdout:removed './166/37' 2024-05-27T09:31:39.722 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './166' 2024-05-27T09:31:39.722 INFO:tasks.workunit.client.0.smithi114.stdout:removed './167/19' 2024-05-27T09:31:39.722 INFO:tasks.workunit.client.0.smithi114.stdout:removed './167/15' 2024-05-27T09:31:39.722 INFO:tasks.workunit.client.0.smithi114.stdout:removed './167/31' 2024-05-27T09:31:39.723 INFO:tasks.workunit.client.0.smithi114.stdout:removed './167/36' 2024-05-27T09:31:39.723 INFO:tasks.workunit.client.0.smithi114.stdout:removed './167/20' 2024-05-27T09:31:39.723 INFO:tasks.workunit.client.0.smithi114.stdout:removed './167/12' 2024-05-27T09:31:39.723 INFO:tasks.workunit.client.0.smithi114.stdout:removed './167/39' 2024-05-27T09:31:39.723 INFO:tasks.workunit.client.0.smithi114.stdout:removed './167/10' 2024-05-27T09:31:39.723 INFO:tasks.workunit.client.0.smithi114.stdout:removed './167/34' 2024-05-27T09:31:39.723 INFO:tasks.workunit.client.0.smithi114.stdout:removed './167/38' 2024-05-27T09:31:40.451 INFO:tasks.workunit.client.0.smithi114.stdout:removed './167/11' 2024-05-27T09:31:40.451 INFO:tasks.workunit.client.0.smithi114.stdout:removed './167/16' 2024-05-27T09:31:40.452 INFO:tasks.workunit.client.0.smithi114.stdout:removed './167/32' 2024-05-27T09:31:40.452 INFO:tasks.workunit.client.0.smithi114.stdout:removed './167/30' 2024-05-27T09:31:40.452 INFO:tasks.workunit.client.0.smithi114.stdout:removed './167/14' 2024-05-27T09:31:40.452 INFO:tasks.workunit.client.0.smithi114.stdout:removed './167/18' 2024-05-27T09:31:40.452 INFO:tasks.workunit.client.0.smithi114.stdout:removed './167/13' 2024-05-27T09:31:40.452 INFO:tasks.workunit.client.0.smithi114.stdout:removed './167/33' 2024-05-27T09:31:40.453 INFO:tasks.workunit.client.0.smithi114.stdout:removed './167/17' 2024-05-27T09:31:40.453 INFO:tasks.workunit.client.0.smithi114.stdout:removed './167/35' 2024-05-27T09:31:40.453 INFO:tasks.workunit.client.0.smithi114.stdout:removed './167/37' 2024-05-27T09:31:40.453 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './167' 2024-05-27T09:31:40.453 INFO:tasks.workunit.client.0.smithi114.stdout:removed './168/19' 2024-05-27T09:31:40.453 INFO:tasks.workunit.client.0.smithi114.stdout:removed './168/15' 2024-05-27T09:31:40.453 INFO:tasks.workunit.client.0.smithi114.stdout:removed './168/31' 2024-05-27T09:31:40.454 INFO:tasks.workunit.client.0.smithi114.stdout:removed './168/36' 2024-05-27T09:31:40.454 INFO:tasks.workunit.client.0.smithi114.stdout:removed './168/20' 2024-05-27T09:31:40.454 INFO:tasks.workunit.client.0.smithi114.stdout:removed './168/12' 2024-05-27T09:31:40.454 INFO:tasks.workunit.client.0.smithi114.stdout:removed './168/39' 2024-05-27T09:31:40.454 INFO:tasks.workunit.client.0.smithi114.stdout:removed './168/10' 2024-05-27T09:31:40.454 INFO:tasks.workunit.client.0.smithi114.stdout:removed './168/34' 2024-05-27T09:31:40.455 INFO:tasks.workunit.client.0.smithi114.stdout:removed './168/38' 2024-05-27T09:31:40.455 INFO:tasks.workunit.client.0.smithi114.stdout:removed './168/11' 2024-05-27T09:31:40.455 INFO:tasks.workunit.client.0.smithi114.stdout:removed './168/16' 2024-05-27T09:31:40.455 INFO:tasks.workunit.client.0.smithi114.stdout:removed './168/32' 2024-05-27T09:31:40.455 INFO:tasks.workunit.client.0.smithi114.stdout:removed './168/30' 2024-05-27T09:31:40.455 INFO:tasks.workunit.client.0.smithi114.stdout:removed './168/14' 2024-05-27T09:31:40.455 INFO:tasks.workunit.client.0.smithi114.stdout:removed './168/18' 2024-05-27T09:31:40.455 INFO:tasks.workunit.client.0.smithi114.stdout:removed './168/13' 2024-05-27T09:31:40.456 INFO:tasks.workunit.client.0.smithi114.stdout:removed './168/33' 2024-05-27T09:31:40.456 INFO:tasks.workunit.client.0.smithi114.stdout:removed './168/17' 2024-05-27T09:31:40.456 INFO:tasks.workunit.client.0.smithi114.stdout:removed './168/35' 2024-05-27T09:31:40.456 INFO:tasks.workunit.client.0.smithi114.stdout:removed './168/37' 2024-05-27T09:31:40.456 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './168' 2024-05-27T09:31:40.456 INFO:tasks.workunit.client.0.smithi114.stdout:removed './169/19' 2024-05-27T09:31:40.456 INFO:tasks.workunit.client.0.smithi114.stdout:removed './169/15' 2024-05-27T09:31:40.457 INFO:tasks.workunit.client.0.smithi114.stdout:removed './169/31' 2024-05-27T09:31:40.457 INFO:tasks.workunit.client.0.smithi114.stdout:removed './169/36' 2024-05-27T09:31:40.457 INFO:tasks.workunit.client.0.smithi114.stdout:removed './169/20' 2024-05-27T09:31:40.457 INFO:tasks.workunit.client.0.smithi114.stdout:removed './169/12' 2024-05-27T09:31:40.457 INFO:tasks.workunit.client.0.smithi114.stdout:removed './169/39' 2024-05-27T09:31:40.457 INFO:tasks.workunit.client.0.smithi114.stdout:removed './169/10' 2024-05-27T09:31:40.457 INFO:tasks.workunit.client.0.smithi114.stdout:removed './169/34' 2024-05-27T09:31:40.458 INFO:tasks.workunit.client.0.smithi114.stdout:removed './169/38' 2024-05-27T09:31:40.458 INFO:tasks.workunit.client.0.smithi114.stdout:removed './169/11' 2024-05-27T09:31:40.458 INFO:tasks.workunit.client.0.smithi114.stdout:removed './169/16' 2024-05-27T09:31:40.458 INFO:tasks.workunit.client.0.smithi114.stdout:removed './169/32' 2024-05-27T09:31:40.458 INFO:tasks.workunit.client.0.smithi114.stdout:removed './169/30' 2024-05-27T09:31:40.458 INFO:tasks.workunit.client.0.smithi114.stdout:removed './169/14' 2024-05-27T09:31:40.459 INFO:tasks.workunit.client.0.smithi114.stdout:removed './169/18' 2024-05-27T09:31:40.459 INFO:tasks.workunit.client.0.smithi114.stdout:removed './169/13' 2024-05-27T09:31:40.459 INFO:tasks.workunit.client.0.smithi114.stdout:removed './169/33' 2024-05-27T09:31:40.459 INFO:tasks.workunit.client.0.smithi114.stdout:removed './169/17' 2024-05-27T09:31:40.459 INFO:tasks.workunit.client.0.smithi114.stdout:removed './169/35' 2024-05-27T09:31:40.459 INFO:tasks.workunit.client.0.smithi114.stdout:removed './169/37' 2024-05-27T09:31:40.459 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './169' 2024-05-27T09:31:40.460 INFO:tasks.workunit.client.0.smithi114.stdout:removed './170/19' 2024-05-27T09:31:40.460 INFO:tasks.workunit.client.0.smithi114.stdout:removed './170/15' 2024-05-27T09:31:40.460 INFO:tasks.workunit.client.0.smithi114.stdout:removed './170/31' 2024-05-27T09:31:40.460 INFO:tasks.workunit.client.0.smithi114.stdout:removed './170/36' 2024-05-27T09:31:40.460 INFO:tasks.workunit.client.0.smithi114.stdout:removed './170/20' 2024-05-27T09:31:40.460 INFO:tasks.workunit.client.0.smithi114.stdout:removed './170/12' 2024-05-27T09:31:40.460 INFO:tasks.workunit.client.0.smithi114.stdout:removed './170/39' 2024-05-27T09:31:40.461 INFO:tasks.workunit.client.0.smithi114.stdout:removed './170/10' 2024-05-27T09:31:40.461 INFO:tasks.workunit.client.0.smithi114.stdout:removed './170/34' 2024-05-27T09:31:40.461 INFO:tasks.workunit.client.0.smithi114.stdout:removed './170/38' 2024-05-27T09:31:40.461 INFO:tasks.workunit.client.0.smithi114.stdout:removed './170/11' 2024-05-27T09:31:40.461 INFO:tasks.workunit.client.0.smithi114.stdout:removed './170/16' 2024-05-27T09:31:40.461 INFO:tasks.workunit.client.0.smithi114.stdout:removed './170/32' 2024-05-27T09:31:40.461 INFO:tasks.workunit.client.0.smithi114.stdout:removed './170/30' 2024-05-27T09:31:40.462 INFO:tasks.workunit.client.0.smithi114.stdout:removed './170/14' 2024-05-27T09:31:40.462 INFO:tasks.workunit.client.0.smithi114.stdout:removed './170/18' 2024-05-27T09:31:40.462 INFO:tasks.workunit.client.0.smithi114.stdout:removed './170/13' 2024-05-27T09:31:40.462 INFO:tasks.workunit.client.0.smithi114.stdout:removed './170/33' 2024-05-27T09:31:40.462 INFO:tasks.workunit.client.0.smithi114.stdout:removed './170/17' 2024-05-27T09:31:40.462 INFO:tasks.workunit.client.0.smithi114.stdout:removed './170/35' 2024-05-27T09:31:40.463 INFO:tasks.workunit.client.0.smithi114.stdout:removed './170/37' 2024-05-27T09:31:40.463 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './170' 2024-05-27T09:31:40.463 INFO:tasks.workunit.client.0.smithi114.stdout:removed './171/19' 2024-05-27T09:31:40.463 INFO:tasks.workunit.client.0.smithi114.stdout:removed './171/15' 2024-05-27T09:31:40.463 INFO:tasks.workunit.client.0.smithi114.stdout:removed './171/31' 2024-05-27T09:31:40.463 INFO:tasks.workunit.client.0.smithi114.stdout:removed './171/36' 2024-05-27T09:31:40.463 INFO:tasks.workunit.client.0.smithi114.stdout:removed './171/20' 2024-05-27T09:31:40.463 INFO:tasks.workunit.client.0.smithi114.stdout:removed './171/12' 2024-05-27T09:31:40.464 INFO:tasks.workunit.client.0.smithi114.stdout:removed './171/39' 2024-05-27T09:31:40.464 INFO:tasks.workunit.client.0.smithi114.stdout:removed './171/10' 2024-05-27T09:31:40.464 INFO:tasks.workunit.client.0.smithi114.stdout:removed './171/34' 2024-05-27T09:31:40.464 INFO:tasks.workunit.client.0.smithi114.stdout:removed './171/38' 2024-05-27T09:31:40.464 INFO:tasks.workunit.client.0.smithi114.stdout:removed './171/11' 2024-05-27T09:31:40.464 INFO:tasks.workunit.client.0.smithi114.stdout:removed './171/16' 2024-05-27T09:31:40.465 INFO:tasks.workunit.client.0.smithi114.stdout:removed './171/32' 2024-05-27T09:31:40.465 INFO:tasks.workunit.client.0.smithi114.stdout:removed './171/30' 2024-05-27T09:31:40.465 INFO:tasks.workunit.client.0.smithi114.stdout:removed './171/14' 2024-05-27T09:31:40.465 INFO:tasks.workunit.client.0.smithi114.stdout:removed './171/18' 2024-05-27T09:31:40.465 INFO:tasks.workunit.client.0.smithi114.stdout:removed './171/13' 2024-05-27T09:31:40.465 INFO:tasks.workunit.client.0.smithi114.stdout:removed './171/33' 2024-05-27T09:31:40.465 INFO:tasks.workunit.client.0.smithi114.stdout:removed './171/17' 2024-05-27T09:31:40.465 INFO:tasks.workunit.client.0.smithi114.stdout:removed './171/35' 2024-05-27T09:31:40.466 INFO:tasks.workunit.client.0.smithi114.stdout:removed './171/37' 2024-05-27T09:31:40.466 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './171' 2024-05-27T09:31:40.466 INFO:tasks.workunit.client.0.smithi114.stdout:removed './172/19' 2024-05-27T09:31:40.466 INFO:tasks.workunit.client.0.smithi114.stdout:removed './172/15' 2024-05-27T09:31:40.466 INFO:tasks.workunit.client.0.smithi114.stdout:removed './172/31' 2024-05-27T09:31:40.466 INFO:tasks.workunit.client.0.smithi114.stdout:removed './172/36' 2024-05-27T09:31:40.467 INFO:tasks.workunit.client.0.smithi114.stdout:removed './172/20' 2024-05-27T09:31:40.467 INFO:tasks.workunit.client.0.smithi114.stdout:removed './172/12' 2024-05-27T09:31:40.467 INFO:tasks.workunit.client.0.smithi114.stdout:removed './172/39' 2024-05-27T09:31:40.467 INFO:tasks.workunit.client.0.smithi114.stdout:removed './172/10' 2024-05-27T09:31:40.467 INFO:tasks.workunit.client.0.smithi114.stdout:removed './172/34' 2024-05-27T09:31:40.467 INFO:tasks.workunit.client.0.smithi114.stdout:removed './172/38' 2024-05-27T09:31:40.467 INFO:tasks.workunit.client.0.smithi114.stdout:removed './172/11' 2024-05-27T09:31:40.468 INFO:tasks.workunit.client.0.smithi114.stdout:removed './172/16' 2024-05-27T09:31:40.468 INFO:tasks.workunit.client.0.smithi114.stdout:removed './172/32' 2024-05-27T09:31:40.468 INFO:tasks.workunit.client.0.smithi114.stdout:removed './172/30' 2024-05-27T09:31:40.468 INFO:tasks.workunit.client.0.smithi114.stdout:removed './172/14' 2024-05-27T09:31:40.468 INFO:tasks.workunit.client.0.smithi114.stdout:removed './172/18' 2024-05-27T09:31:40.468 INFO:tasks.workunit.client.0.smithi114.stdout:removed './172/13' 2024-05-27T09:31:40.468 INFO:tasks.workunit.client.0.smithi114.stdout:removed './172/33' 2024-05-27T09:31:40.469 INFO:tasks.workunit.client.0.smithi114.stdout:removed './172/17' 2024-05-27T09:31:40.469 INFO:tasks.workunit.client.0.smithi114.stdout:removed './172/35' 2024-05-27T09:31:40.469 INFO:tasks.workunit.client.0.smithi114.stdout:removed './172/37' 2024-05-27T09:31:40.469 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './172' 2024-05-27T09:31:40.469 INFO:tasks.workunit.client.0.smithi114.stdout:removed './173/19' 2024-05-27T09:31:40.469 INFO:tasks.workunit.client.0.smithi114.stdout:removed './173/15' 2024-05-27T09:31:40.470 INFO:tasks.workunit.client.0.smithi114.stdout:removed './173/31' 2024-05-27T09:31:40.470 INFO:tasks.workunit.client.0.smithi114.stdout:removed './173/36' 2024-05-27T09:31:40.470 INFO:tasks.workunit.client.0.smithi114.stdout:removed './173/20' 2024-05-27T09:31:40.470 INFO:tasks.workunit.client.0.smithi114.stdout:removed './173/12' 2024-05-27T09:31:40.470 INFO:tasks.workunit.client.0.smithi114.stdout:removed './173/39' 2024-05-27T09:31:40.470 INFO:tasks.workunit.client.0.smithi114.stdout:removed './173/10' 2024-05-27T09:31:40.470 INFO:tasks.workunit.client.0.smithi114.stdout:removed './173/34' 2024-05-27T09:31:40.471 INFO:tasks.workunit.client.0.smithi114.stdout:removed './173/38' 2024-05-27T09:31:40.471 INFO:tasks.workunit.client.0.smithi114.stdout:removed './173/11' 2024-05-27T09:31:40.471 INFO:tasks.workunit.client.0.smithi114.stdout:removed './173/16' 2024-05-27T09:31:40.471 INFO:tasks.workunit.client.0.smithi114.stdout:removed './173/32' 2024-05-27T09:31:40.471 INFO:tasks.workunit.client.0.smithi114.stdout:removed './173/30' 2024-05-27T09:31:40.471 INFO:tasks.workunit.client.0.smithi114.stdout:removed './173/14' 2024-05-27T09:31:40.471 INFO:tasks.workunit.client.0.smithi114.stdout:removed './173/18' 2024-05-27T09:31:40.471 INFO:tasks.workunit.client.0.smithi114.stdout:removed './173/13' 2024-05-27T09:31:40.472 INFO:tasks.workunit.client.0.smithi114.stdout:removed './173/33' 2024-05-27T09:31:40.472 INFO:tasks.workunit.client.0.smithi114.stdout:removed './173/17' 2024-05-27T09:31:40.472 INFO:tasks.workunit.client.0.smithi114.stdout:removed './173/35' 2024-05-27T09:31:40.472 INFO:tasks.workunit.client.0.smithi114.stdout:removed './173/37' 2024-05-27T09:31:40.472 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './173' 2024-05-27T09:31:40.472 INFO:tasks.workunit.client.0.smithi114.stdout:removed './174/19' 2024-05-27T09:31:40.473 INFO:tasks.workunit.client.0.smithi114.stdout:removed './174/15' 2024-05-27T09:31:40.473 INFO:tasks.workunit.client.0.smithi114.stdout:removed './174/31' 2024-05-27T09:31:40.473 INFO:tasks.workunit.client.0.smithi114.stdout:removed './174/36' 2024-05-27T09:31:40.473 INFO:tasks.workunit.client.0.smithi114.stdout:removed './174/20' 2024-05-27T09:31:40.473 INFO:tasks.workunit.client.0.smithi114.stdout:removed './174/12' 2024-05-27T09:31:40.473 INFO:tasks.workunit.client.0.smithi114.stdout:removed './174/39' 2024-05-27T09:31:40.473 INFO:tasks.workunit.client.0.smithi114.stdout:removed './174/10' 2024-05-27T09:31:40.474 INFO:tasks.workunit.client.0.smithi114.stdout:removed './174/34' 2024-05-27T09:31:40.474 INFO:tasks.workunit.client.0.smithi114.stdout:removed './174/38' 2024-05-27T09:31:40.474 INFO:tasks.workunit.client.0.smithi114.stdout:removed './174/11' 2024-05-27T09:31:40.474 INFO:tasks.workunit.client.0.smithi114.stdout:removed './174/16' 2024-05-27T09:31:40.474 INFO:tasks.workunit.client.0.smithi114.stdout:removed './174/32' 2024-05-27T09:31:40.474 INFO:tasks.workunit.client.0.smithi114.stdout:removed './174/30' 2024-05-27T09:31:40.474 INFO:tasks.workunit.client.0.smithi114.stdout:removed './174/14' 2024-05-27T09:31:40.475 INFO:tasks.workunit.client.0.smithi114.stdout:removed './174/18' 2024-05-27T09:31:40.475 INFO:tasks.workunit.client.0.smithi114.stdout:removed './174/13' 2024-05-27T09:31:40.475 INFO:tasks.workunit.client.0.smithi114.stdout:removed './174/33' 2024-05-27T09:31:40.475 INFO:tasks.workunit.client.0.smithi114.stdout:removed './174/17' 2024-05-27T09:31:40.475 INFO:tasks.workunit.client.0.smithi114.stdout:removed './174/35' 2024-05-27T09:31:40.475 INFO:tasks.workunit.client.0.smithi114.stdout:removed './174/37' 2024-05-27T09:31:40.475 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './174' 2024-05-27T09:31:40.476 INFO:tasks.workunit.client.0.smithi114.stdout:removed './175/19' 2024-05-27T09:31:40.476 INFO:tasks.workunit.client.0.smithi114.stdout:removed './175/15' 2024-05-27T09:31:40.476 INFO:tasks.workunit.client.0.smithi114.stdout:removed './175/31' 2024-05-27T09:31:40.476 INFO:tasks.workunit.client.0.smithi114.stdout:removed './175/36' 2024-05-27T09:31:40.476 INFO:tasks.workunit.client.0.smithi114.stdout:removed './175/20' 2024-05-27T09:31:40.476 INFO:tasks.workunit.client.0.smithi114.stdout:removed './175/12' 2024-05-27T09:31:40.476 INFO:tasks.workunit.client.0.smithi114.stdout:removed './175/39' 2024-05-27T09:31:40.477 INFO:tasks.workunit.client.0.smithi114.stdout:removed './175/10' 2024-05-27T09:31:40.477 INFO:tasks.workunit.client.0.smithi114.stdout:removed './175/34' 2024-05-27T09:31:40.477 INFO:tasks.workunit.client.0.smithi114.stdout:removed './175/38' 2024-05-27T09:31:40.477 INFO:tasks.workunit.client.0.smithi114.stdout:removed './175/11' 2024-05-27T09:31:40.477 INFO:tasks.workunit.client.0.smithi114.stdout:removed './175/16' 2024-05-27T09:31:40.477 INFO:tasks.workunit.client.0.smithi114.stdout:removed './175/32' 2024-05-27T09:31:40.477 INFO:tasks.workunit.client.0.smithi114.stdout:removed './175/30' 2024-05-27T09:31:40.478 INFO:tasks.workunit.client.0.smithi114.stdout:removed './175/14' 2024-05-27T09:31:40.478 INFO:tasks.workunit.client.0.smithi114.stdout:removed './175/18' 2024-05-27T09:31:40.478 INFO:tasks.workunit.client.0.smithi114.stdout:removed './175/13' 2024-05-27T09:31:40.478 INFO:tasks.workunit.client.0.smithi114.stdout:removed './175/33' 2024-05-27T09:31:40.478 INFO:tasks.workunit.client.0.smithi114.stdout:removed './175/17' 2024-05-27T09:31:40.478 INFO:tasks.workunit.client.0.smithi114.stdout:removed './175/35' 2024-05-27T09:31:40.478 INFO:tasks.workunit.client.0.smithi114.stdout:removed './175/37' 2024-05-27T09:31:40.479 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './175' 2024-05-27T09:31:40.479 INFO:tasks.workunit.client.0.smithi114.stdout:removed './176/19' 2024-05-27T09:31:40.479 INFO:tasks.workunit.client.0.smithi114.stdout:removed './176/15' 2024-05-27T09:31:40.479 INFO:tasks.workunit.client.0.smithi114.stdout:removed './176/31' 2024-05-27T09:31:40.479 INFO:tasks.workunit.client.0.smithi114.stdout:removed './176/36' 2024-05-27T09:31:40.479 INFO:tasks.workunit.client.0.smithi114.stdout:removed './176/20' 2024-05-27T09:31:40.480 INFO:tasks.workunit.client.0.smithi114.stdout:removed './176/12' 2024-05-27T09:31:40.480 INFO:tasks.workunit.client.0.smithi114.stdout:removed './176/39' 2024-05-27T09:31:40.480 INFO:tasks.workunit.client.0.smithi114.stdout:removed './176/10' 2024-05-27T09:31:40.480 INFO:tasks.workunit.client.0.smithi114.stdout:removed './176/34' 2024-05-27T09:31:40.480 INFO:tasks.workunit.client.0.smithi114.stdout:removed './176/38' 2024-05-27T09:31:40.480 INFO:tasks.workunit.client.0.smithi114.stdout:removed './176/11' 2024-05-27T09:31:40.480 INFO:tasks.workunit.client.0.smithi114.stdout:removed './176/16' 2024-05-27T09:31:40.481 INFO:tasks.workunit.client.0.smithi114.stdout:removed './176/32' 2024-05-27T09:31:40.481 INFO:tasks.workunit.client.0.smithi114.stdout:removed './176/30' 2024-05-27T09:31:40.481 INFO:tasks.workunit.client.0.smithi114.stdout:removed './176/14' 2024-05-27T09:31:40.481 INFO:tasks.workunit.client.0.smithi114.stdout:removed './176/18' 2024-05-27T09:31:40.481 INFO:tasks.workunit.client.0.smithi114.stdout:removed './176/13' 2024-05-27T09:31:40.481 INFO:tasks.workunit.client.0.smithi114.stdout:removed './176/33' 2024-05-27T09:31:40.481 INFO:tasks.workunit.client.0.smithi114.stdout:removed './176/17' 2024-05-27T09:31:40.482 INFO:tasks.workunit.client.0.smithi114.stdout:removed './176/35' 2024-05-27T09:31:40.482 INFO:tasks.workunit.client.0.smithi114.stdout:removed './176/37' 2024-05-27T09:31:40.482 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './176' 2024-05-27T09:31:40.482 INFO:tasks.workunit.client.0.smithi114.stdout:removed './177/19' 2024-05-27T09:31:40.482 INFO:tasks.workunit.client.0.smithi114.stdout:removed './177/15' 2024-05-27T09:31:41.520 INFO:tasks.workunit.client.0.smithi114.stdout:removed './177/31' 2024-05-27T09:31:41.520 INFO:tasks.workunit.client.0.smithi114.stdout:removed './177/36' 2024-05-27T09:31:41.520 INFO:tasks.workunit.client.0.smithi114.stdout:removed './177/20' 2024-05-27T09:31:41.520 INFO:tasks.workunit.client.0.smithi114.stdout:removed './177/12' 2024-05-27T09:31:41.520 INFO:tasks.workunit.client.0.smithi114.stdout:removed './177/39' 2024-05-27T09:31:41.521 INFO:tasks.workunit.client.0.smithi114.stdout:removed './177/10' 2024-05-27T09:31:41.521 INFO:tasks.workunit.client.0.smithi114.stdout:removed './177/34' 2024-05-27T09:31:41.521 INFO:tasks.workunit.client.0.smithi114.stdout:removed './177/38' 2024-05-27T09:31:41.521 INFO:tasks.workunit.client.0.smithi114.stdout:removed './177/11' 2024-05-27T09:31:41.521 INFO:tasks.workunit.client.0.smithi114.stdout:removed './177/16' 2024-05-27T09:31:41.521 INFO:tasks.workunit.client.0.smithi114.stdout:removed './177/32' 2024-05-27T09:31:41.521 INFO:tasks.workunit.client.0.smithi114.stdout:removed './177/30' 2024-05-27T09:31:41.521 INFO:tasks.workunit.client.0.smithi114.stdout:removed './177/14' 2024-05-27T09:31:41.521 INFO:tasks.workunit.client.0.smithi114.stdout:removed './177/18' 2024-05-27T09:31:41.521 INFO:tasks.workunit.client.0.smithi114.stdout:removed './177/13' 2024-05-27T09:31:41.521 INFO:tasks.workunit.client.0.smithi114.stdout:removed './177/33' 2024-05-27T09:31:41.521 INFO:tasks.workunit.client.0.smithi114.stdout:removed './177/17' 2024-05-27T09:31:41.521 INFO:tasks.workunit.client.0.smithi114.stdout:removed './177/35' 2024-05-27T09:31:41.521 INFO:tasks.workunit.client.0.smithi114.stdout:removed './177/37' 2024-05-27T09:31:41.521 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './177' 2024-05-27T09:31:41.522 INFO:tasks.workunit.client.0.smithi114.stdout:removed './178/19' 2024-05-27T09:31:41.522 INFO:tasks.workunit.client.0.smithi114.stdout:removed './178/15' 2024-05-27T09:31:41.522 INFO:tasks.workunit.client.0.smithi114.stdout:removed './178/31' 2024-05-27T09:31:41.522 INFO:tasks.workunit.client.0.smithi114.stdout:removed './178/36' 2024-05-27T09:31:41.522 INFO:tasks.workunit.client.0.smithi114.stdout:removed './178/20' 2024-05-27T09:31:41.522 INFO:tasks.workunit.client.0.smithi114.stdout:removed './178/12' 2024-05-27T09:31:41.522 INFO:tasks.workunit.client.0.smithi114.stdout:removed './178/39' 2024-05-27T09:31:41.522 INFO:tasks.workunit.client.0.smithi114.stdout:removed './178/10' 2024-05-27T09:31:41.522 INFO:tasks.workunit.client.0.smithi114.stdout:removed './178/34' 2024-05-27T09:31:41.522 INFO:tasks.workunit.client.0.smithi114.stdout:removed './178/38' 2024-05-27T09:31:41.522 INFO:tasks.workunit.client.0.smithi114.stdout:removed './178/11' 2024-05-27T09:31:41.522 INFO:tasks.workunit.client.0.smithi114.stdout:removed './178/16' 2024-05-27T09:31:41.522 INFO:tasks.workunit.client.0.smithi114.stdout:removed './178/32' 2024-05-27T09:31:41.522 INFO:tasks.workunit.client.0.smithi114.stdout:removed './178/30' 2024-05-27T09:31:41.523 INFO:tasks.workunit.client.0.smithi114.stdout:removed './178/14' 2024-05-27T09:31:41.523 INFO:tasks.workunit.client.0.smithi114.stdout:removed './178/18' 2024-05-27T09:31:41.523 INFO:tasks.workunit.client.0.smithi114.stdout:removed './178/13' 2024-05-27T09:31:41.523 INFO:tasks.workunit.client.0.smithi114.stdout:removed './178/33' 2024-05-27T09:31:41.523 INFO:tasks.workunit.client.0.smithi114.stdout:removed './178/17' 2024-05-27T09:31:41.523 INFO:tasks.workunit.client.0.smithi114.stdout:removed './178/35' 2024-05-27T09:31:41.523 INFO:tasks.workunit.client.0.smithi114.stdout:removed './178/37' 2024-05-27T09:31:41.523 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './178' 2024-05-27T09:31:41.523 INFO:tasks.workunit.client.0.smithi114.stdout:removed './179/19' 2024-05-27T09:31:41.523 INFO:tasks.workunit.client.0.smithi114.stdout:removed './179/15' 2024-05-27T09:31:41.523 INFO:tasks.workunit.client.0.smithi114.stdout:removed './179/31' 2024-05-27T09:31:41.523 INFO:tasks.workunit.client.0.smithi114.stdout:removed './179/36' 2024-05-27T09:31:41.523 INFO:tasks.workunit.client.0.smithi114.stdout:removed './179/20' 2024-05-27T09:31:41.523 INFO:tasks.workunit.client.0.smithi114.stdout:removed './179/12' 2024-05-27T09:31:41.524 INFO:tasks.workunit.client.0.smithi114.stdout:removed './179/39' 2024-05-27T09:31:41.524 INFO:tasks.workunit.client.0.smithi114.stdout:removed './179/10' 2024-05-27T09:31:41.524 INFO:tasks.workunit.client.0.smithi114.stdout:removed './179/34' 2024-05-27T09:31:41.524 INFO:tasks.workunit.client.0.smithi114.stdout:removed './179/38' 2024-05-27T09:31:41.524 INFO:tasks.workunit.client.0.smithi114.stdout:removed './179/11' 2024-05-27T09:31:41.524 INFO:tasks.workunit.client.0.smithi114.stdout:removed './179/16' 2024-05-27T09:31:41.524 INFO:tasks.workunit.client.0.smithi114.stdout:removed './179/32' 2024-05-27T09:31:41.524 INFO:tasks.workunit.client.0.smithi114.stdout:removed './179/30' 2024-05-27T09:31:41.524 INFO:tasks.workunit.client.0.smithi114.stdout:removed './179/14' 2024-05-27T09:31:41.524 INFO:tasks.workunit.client.0.smithi114.stdout:removed './179/18' 2024-05-27T09:31:41.524 INFO:tasks.workunit.client.0.smithi114.stdout:removed './179/13' 2024-05-27T09:31:41.524 INFO:tasks.workunit.client.0.smithi114.stdout:removed './179/33' 2024-05-27T09:31:41.524 INFO:tasks.workunit.client.0.smithi114.stdout:removed './179/17' 2024-05-27T09:31:41.524 INFO:tasks.workunit.client.0.smithi114.stdout:removed './179/35' 2024-05-27T09:31:41.524 INFO:tasks.workunit.client.0.smithi114.stdout:removed './179/37' 2024-05-27T09:31:41.525 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './179' 2024-05-27T09:31:41.525 INFO:tasks.workunit.client.0.smithi114.stdout:removed './180/19' 2024-05-27T09:31:41.525 INFO:tasks.workunit.client.0.smithi114.stdout:removed './180/15' 2024-05-27T09:31:41.525 INFO:tasks.workunit.client.0.smithi114.stdout:removed './180/31' 2024-05-27T09:31:41.525 INFO:tasks.workunit.client.0.smithi114.stdout:removed './180/36' 2024-05-27T09:31:41.525 INFO:tasks.workunit.client.0.smithi114.stdout:removed './180/20' 2024-05-27T09:31:41.525 INFO:tasks.workunit.client.0.smithi114.stdout:removed './180/12' 2024-05-27T09:31:41.525 INFO:tasks.workunit.client.0.smithi114.stdout:removed './180/39' 2024-05-27T09:31:41.525 INFO:tasks.workunit.client.0.smithi114.stdout:removed './180/10' 2024-05-27T09:31:41.525 INFO:tasks.workunit.client.0.smithi114.stdout:removed './180/34' 2024-05-27T09:31:41.525 INFO:tasks.workunit.client.0.smithi114.stdout:removed './180/38' 2024-05-27T09:31:41.525 INFO:tasks.workunit.client.0.smithi114.stdout:removed './180/11' 2024-05-27T09:31:41.525 INFO:tasks.workunit.client.0.smithi114.stdout:removed './180/16' 2024-05-27T09:31:41.525 INFO:tasks.workunit.client.0.smithi114.stdout:removed './180/32' 2024-05-27T09:31:41.525 INFO:tasks.workunit.client.0.smithi114.stdout:removed './180/30' 2024-05-27T09:31:41.526 INFO:tasks.workunit.client.0.smithi114.stdout:removed './180/14' 2024-05-27T09:31:41.526 INFO:tasks.workunit.client.0.smithi114.stdout:removed './180/18' 2024-05-27T09:31:41.526 INFO:tasks.workunit.client.0.smithi114.stdout:removed './180/13' 2024-05-27T09:31:41.526 INFO:tasks.workunit.client.0.smithi114.stdout:removed './180/33' 2024-05-27T09:31:41.526 INFO:tasks.workunit.client.0.smithi114.stdout:removed './180/17' 2024-05-27T09:31:41.526 INFO:tasks.workunit.client.0.smithi114.stdout:removed './180/35' 2024-05-27T09:31:41.526 INFO:tasks.workunit.client.0.smithi114.stdout:removed './180/37' 2024-05-27T09:31:41.526 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './180' 2024-05-27T09:31:41.526 INFO:tasks.workunit.client.0.smithi114.stdout:removed './181/19' 2024-05-27T09:31:41.526 INFO:tasks.workunit.client.0.smithi114.stdout:removed './181/15' 2024-05-27T09:31:41.526 INFO:tasks.workunit.client.0.smithi114.stdout:removed './181/31' 2024-05-27T09:31:41.526 INFO:tasks.workunit.client.0.smithi114.stdout:removed './181/36' 2024-05-27T09:31:41.526 INFO:tasks.workunit.client.0.smithi114.stdout:removed './181/20' 2024-05-27T09:31:41.526 INFO:tasks.workunit.client.0.smithi114.stdout:removed './181/12' 2024-05-27T09:31:41.527 INFO:tasks.workunit.client.0.smithi114.stdout:removed './181/39' 2024-05-27T09:31:41.527 INFO:tasks.workunit.client.0.smithi114.stdout:removed './181/10' 2024-05-27T09:31:41.527 INFO:tasks.workunit.client.0.smithi114.stdout:removed './181/34' 2024-05-27T09:31:41.527 INFO:tasks.workunit.client.0.smithi114.stdout:removed './181/38' 2024-05-27T09:31:41.527 INFO:tasks.workunit.client.0.smithi114.stdout:removed './181/11' 2024-05-27T09:31:41.527 INFO:tasks.workunit.client.0.smithi114.stdout:removed './181/16' 2024-05-27T09:31:41.527 INFO:tasks.workunit.client.0.smithi114.stdout:removed './181/32' 2024-05-27T09:31:41.527 INFO:tasks.workunit.client.0.smithi114.stdout:removed './181/30' 2024-05-27T09:31:41.527 INFO:tasks.workunit.client.0.smithi114.stdout:removed './181/14' 2024-05-27T09:31:41.527 INFO:tasks.workunit.client.0.smithi114.stdout:removed './181/18' 2024-05-27T09:31:41.527 INFO:tasks.workunit.client.0.smithi114.stdout:removed './181/13' 2024-05-27T09:31:41.527 INFO:tasks.workunit.client.0.smithi114.stdout:removed './181/33' 2024-05-27T09:31:41.527 INFO:tasks.workunit.client.0.smithi114.stdout:removed './181/17' 2024-05-27T09:31:41.527 INFO:tasks.workunit.client.0.smithi114.stdout:removed './181/35' 2024-05-27T09:31:41.528 INFO:tasks.workunit.client.0.smithi114.stdout:removed './181/37' 2024-05-27T09:31:41.528 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './181' 2024-05-27T09:31:41.528 INFO:tasks.workunit.client.0.smithi114.stdout:removed './182/19' 2024-05-27T09:31:41.528 INFO:tasks.workunit.client.0.smithi114.stdout:removed './182/15' 2024-05-27T09:31:41.528 INFO:tasks.workunit.client.0.smithi114.stdout:removed './182/31' 2024-05-27T09:31:41.528 INFO:tasks.workunit.client.0.smithi114.stdout:removed './182/36' 2024-05-27T09:31:41.528 INFO:tasks.workunit.client.0.smithi114.stdout:removed './182/20' 2024-05-27T09:31:41.528 INFO:tasks.workunit.client.0.smithi114.stdout:removed './182/12' 2024-05-27T09:31:41.528 INFO:tasks.workunit.client.0.smithi114.stdout:removed './182/39' 2024-05-27T09:31:41.528 INFO:tasks.workunit.client.0.smithi114.stdout:removed './182/10' 2024-05-27T09:31:41.528 INFO:tasks.workunit.client.0.smithi114.stdout:removed './182/34' 2024-05-27T09:31:41.528 INFO:tasks.workunit.client.0.smithi114.stdout:removed './182/38' 2024-05-27T09:31:41.528 INFO:tasks.workunit.client.0.smithi114.stdout:removed './182/11' 2024-05-27T09:31:41.528 INFO:tasks.workunit.client.0.smithi114.stdout:removed './182/16' 2024-05-27T09:31:41.528 INFO:tasks.workunit.client.0.smithi114.stdout:removed './182/32' 2024-05-27T09:31:41.529 INFO:tasks.workunit.client.0.smithi114.stdout:removed './182/30' 2024-05-27T09:31:41.529 INFO:tasks.workunit.client.0.smithi114.stdout:removed './182/14' 2024-05-27T09:31:41.529 INFO:tasks.workunit.client.0.smithi114.stdout:removed './182/18' 2024-05-27T09:31:41.529 INFO:tasks.workunit.client.0.smithi114.stdout:removed './182/13' 2024-05-27T09:31:41.529 INFO:tasks.workunit.client.0.smithi114.stdout:removed './182/33' 2024-05-27T09:31:41.529 INFO:tasks.workunit.client.0.smithi114.stdout:removed './182/17' 2024-05-27T09:31:41.529 INFO:tasks.workunit.client.0.smithi114.stdout:removed './182/35' 2024-05-27T09:31:41.529 INFO:tasks.workunit.client.0.smithi114.stdout:removed './182/37' 2024-05-27T09:31:41.529 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './182' 2024-05-27T09:31:41.529 INFO:tasks.workunit.client.0.smithi114.stdout:removed './183/19' 2024-05-27T09:31:41.529 INFO:tasks.workunit.client.0.smithi114.stdout:removed './183/15' 2024-05-27T09:31:41.529 INFO:tasks.workunit.client.0.smithi114.stdout:removed './183/31' 2024-05-27T09:31:41.529 INFO:tasks.workunit.client.0.smithi114.stdout:removed './183/36' 2024-05-27T09:31:41.529 INFO:tasks.workunit.client.0.smithi114.stdout:removed './183/20' 2024-05-27T09:31:41.529 INFO:tasks.workunit.client.0.smithi114.stdout:removed './183/12' 2024-05-27T09:31:41.530 INFO:tasks.workunit.client.0.smithi114.stdout:removed './183/39' 2024-05-27T09:31:41.530 INFO:tasks.workunit.client.0.smithi114.stdout:removed './183/10' 2024-05-27T09:31:41.530 INFO:tasks.workunit.client.0.smithi114.stdout:removed './183/34' 2024-05-27T09:31:41.530 INFO:tasks.workunit.client.0.smithi114.stdout:removed './183/38' 2024-05-27T09:31:41.530 INFO:tasks.workunit.client.0.smithi114.stdout:removed './183/11' 2024-05-27T09:31:41.530 INFO:tasks.workunit.client.0.smithi114.stdout:removed './183/16' 2024-05-27T09:31:41.530 INFO:tasks.workunit.client.0.smithi114.stdout:removed './183/32' 2024-05-27T09:31:41.530 INFO:tasks.workunit.client.0.smithi114.stdout:removed './183/30' 2024-05-27T09:31:41.530 INFO:tasks.workunit.client.0.smithi114.stdout:removed './183/14' 2024-05-27T09:31:41.530 INFO:tasks.workunit.client.0.smithi114.stdout:removed './183/18' 2024-05-27T09:31:41.530 INFO:tasks.workunit.client.0.smithi114.stdout:removed './183/13' 2024-05-27T09:31:41.530 INFO:tasks.workunit.client.0.smithi114.stdout:removed './183/33' 2024-05-27T09:31:41.530 INFO:tasks.workunit.client.0.smithi114.stdout:removed './183/17' 2024-05-27T09:31:41.530 INFO:tasks.workunit.client.0.smithi114.stdout:removed './183/35' 2024-05-27T09:31:41.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed './183/37' 2024-05-27T09:31:41.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './183' 2024-05-27T09:31:41.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed './184/19' 2024-05-27T09:31:41.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed './184/15' 2024-05-27T09:31:41.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed './184/31' 2024-05-27T09:31:41.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed './184/36' 2024-05-27T09:31:41.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed './184/20' 2024-05-27T09:31:41.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed './184/12' 2024-05-27T09:31:41.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed './184/39' 2024-05-27T09:31:41.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed './184/10' 2024-05-27T09:31:41.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed './184/34' 2024-05-27T09:31:41.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed './184/38' 2024-05-27T09:31:41.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed './184/11' 2024-05-27T09:31:41.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed './184/16' 2024-05-27T09:31:41.531 INFO:tasks.workunit.client.0.smithi114.stdout:removed './184/32' 2024-05-27T09:31:41.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './184/30' 2024-05-27T09:31:41.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './184/14' 2024-05-27T09:31:41.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './184/18' 2024-05-27T09:31:41.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './184/13' 2024-05-27T09:31:41.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './184/33' 2024-05-27T09:31:41.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './184/17' 2024-05-27T09:31:41.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './184/35' 2024-05-27T09:31:41.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './184/37' 2024-05-27T09:31:41.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './184' 2024-05-27T09:31:41.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './185/19' 2024-05-27T09:31:41.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './185/15' 2024-05-27T09:31:41.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './185/31' 2024-05-27T09:31:41.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './185/36' 2024-05-27T09:31:41.532 INFO:tasks.workunit.client.0.smithi114.stdout:removed './185/20' 2024-05-27T09:31:41.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './185/12' 2024-05-27T09:31:41.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './185/39' 2024-05-27T09:31:41.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './185/10' 2024-05-27T09:31:41.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './185/34' 2024-05-27T09:31:41.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './185/38' 2024-05-27T09:31:41.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './185/11' 2024-05-27T09:31:41.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './185/16' 2024-05-27T09:31:41.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './185/32' 2024-05-27T09:31:41.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './185/30' 2024-05-27T09:31:41.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './185/14' 2024-05-27T09:31:41.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './185/18' 2024-05-27T09:31:41.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './185/13' 2024-05-27T09:31:41.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './185/33' 2024-05-27T09:31:41.533 INFO:tasks.workunit.client.0.smithi114.stdout:removed './185/17' 2024-05-27T09:31:41.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './185/35' 2024-05-27T09:31:41.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './185/37' 2024-05-27T09:31:41.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './185' 2024-05-27T09:31:41.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './186/19' 2024-05-27T09:31:41.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './186/15' 2024-05-27T09:31:41.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './186/31' 2024-05-27T09:31:41.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './186/36' 2024-05-27T09:31:41.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './186/20' 2024-05-27T09:31:41.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './186/12' 2024-05-27T09:31:41.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './186/39' 2024-05-27T09:31:41.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './186/10' 2024-05-27T09:31:41.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './186/34' 2024-05-27T09:31:41.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './186/38' 2024-05-27T09:31:41.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './186/11' 2024-05-27T09:31:41.534 INFO:tasks.workunit.client.0.smithi114.stdout:removed './186/16' 2024-05-27T09:31:41.535 INFO:tasks.workunit.client.0.smithi114.stdout:removed './186/32' 2024-05-27T09:31:41.535 INFO:tasks.workunit.client.0.smithi114.stdout:removed './186/30' 2024-05-27T09:31:41.535 INFO:tasks.workunit.client.0.smithi114.stdout:removed './186/14' 2024-05-27T09:31:41.535 INFO:tasks.workunit.client.0.smithi114.stdout:removed './186/18' 2024-05-27T09:31:42.404 INFO:tasks.workunit.client.0.smithi114.stdout:removed './186/13' 2024-05-27T09:31:42.404 INFO:tasks.workunit.client.0.smithi114.stdout:removed './186/33' 2024-05-27T09:31:42.404 INFO:tasks.workunit.client.0.smithi114.stdout:removed './186/17' 2024-05-27T09:31:42.404 INFO:tasks.workunit.client.0.smithi114.stdout:removed './186/35' 2024-05-27T09:31:42.404 INFO:tasks.workunit.client.0.smithi114.stdout:removed './186/37' 2024-05-27T09:31:42.404 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './186' 2024-05-27T09:31:42.404 INFO:tasks.workunit.client.0.smithi114.stdout:removed './187/19' 2024-05-27T09:31:42.404 INFO:tasks.workunit.client.0.smithi114.stdout:removed './187/15' 2024-05-27T09:31:42.404 INFO:tasks.workunit.client.0.smithi114.stdout:removed './187/31' 2024-05-27T09:31:42.404 INFO:tasks.workunit.client.0.smithi114.stdout:removed './187/36' 2024-05-27T09:31:42.405 INFO:tasks.workunit.client.0.smithi114.stdout:removed './187/20' 2024-05-27T09:31:42.405 INFO:tasks.workunit.client.0.smithi114.stdout:removed './187/12' 2024-05-27T09:31:42.405 INFO:tasks.workunit.client.0.smithi114.stdout:removed './187/39' 2024-05-27T09:31:42.405 INFO:tasks.workunit.client.0.smithi114.stdout:removed './187/10' 2024-05-27T09:31:42.405 INFO:tasks.workunit.client.0.smithi114.stdout:removed './187/34' 2024-05-27T09:31:42.405 INFO:tasks.workunit.client.0.smithi114.stdout:removed './187/38' 2024-05-27T09:31:42.405 INFO:tasks.workunit.client.0.smithi114.stdout:removed './187/11' 2024-05-27T09:31:42.405 INFO:tasks.workunit.client.0.smithi114.stdout:removed './187/16' 2024-05-27T09:31:42.405 INFO:tasks.workunit.client.0.smithi114.stdout:removed './187/32' 2024-05-27T09:31:42.405 INFO:tasks.workunit.client.0.smithi114.stdout:removed './187/30' 2024-05-27T09:31:42.405 INFO:tasks.workunit.client.0.smithi114.stdout:removed './187/14' 2024-05-27T09:31:42.405 INFO:tasks.workunit.client.0.smithi114.stdout:removed './187/18' 2024-05-27T09:31:42.405 INFO:tasks.workunit.client.0.smithi114.stdout:removed './187/13' 2024-05-27T09:31:42.406 INFO:tasks.workunit.client.0.smithi114.stdout:removed './187/33' 2024-05-27T09:31:42.406 INFO:tasks.workunit.client.0.smithi114.stdout:removed './187/17' 2024-05-27T09:31:42.406 INFO:tasks.workunit.client.0.smithi114.stdout:removed './187/35' 2024-05-27T09:31:42.406 INFO:tasks.workunit.client.0.smithi114.stdout:removed './187/37' 2024-05-27T09:31:42.406 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './187' 2024-05-27T09:31:42.406 INFO:tasks.workunit.client.0.smithi114.stdout:removed './188/19' 2024-05-27T09:31:42.406 INFO:tasks.workunit.client.0.smithi114.stdout:removed './188/15' 2024-05-27T09:31:42.406 INFO:tasks.workunit.client.0.smithi114.stdout:removed './188/31' 2024-05-27T09:31:42.407 INFO:tasks.workunit.client.0.smithi114.stdout:removed './188/36' 2024-05-27T09:31:42.407 INFO:tasks.workunit.client.0.smithi114.stdout:removed './188/20' 2024-05-27T09:31:42.407 INFO:tasks.workunit.client.0.smithi114.stdout:removed './188/12' 2024-05-27T09:31:42.407 INFO:tasks.workunit.client.0.smithi114.stdout:removed './188/39' 2024-05-27T09:31:42.407 INFO:tasks.workunit.client.0.smithi114.stdout:removed './188/10' 2024-05-27T09:31:42.407 INFO:tasks.workunit.client.0.smithi114.stdout:removed './188/34' 2024-05-27T09:31:42.407 INFO:tasks.workunit.client.0.smithi114.stdout:removed './188/38' 2024-05-27T09:31:42.407 INFO:tasks.workunit.client.0.smithi114.stdout:removed './188/11' 2024-05-27T09:31:42.407 INFO:tasks.workunit.client.0.smithi114.stdout:removed './188/16' 2024-05-27T09:31:42.407 INFO:tasks.workunit.client.0.smithi114.stdout:removed './188/32' 2024-05-27T09:31:42.407 INFO:tasks.workunit.client.0.smithi114.stdout:removed './188/30' 2024-05-27T09:31:42.407 INFO:tasks.workunit.client.0.smithi114.stdout:removed './188/14' 2024-05-27T09:31:42.408 INFO:tasks.workunit.client.0.smithi114.stdout:removed './188/18' 2024-05-27T09:31:42.408 INFO:tasks.workunit.client.0.smithi114.stdout:removed './188/13' 2024-05-27T09:31:42.408 INFO:tasks.workunit.client.0.smithi114.stdout:removed './188/33' 2024-05-27T09:31:42.408 INFO:tasks.workunit.client.0.smithi114.stdout:removed './188/17' 2024-05-27T09:31:42.408 INFO:tasks.workunit.client.0.smithi114.stdout:removed './188/35' 2024-05-27T09:31:42.408 INFO:tasks.workunit.client.0.smithi114.stdout:removed './188/37' 2024-05-27T09:31:42.408 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './188' 2024-05-27T09:31:42.408 INFO:tasks.workunit.client.0.smithi114.stdout:removed './189/19' 2024-05-27T09:31:42.408 INFO:tasks.workunit.client.0.smithi114.stdout:removed './189/15' 2024-05-27T09:31:42.408 INFO:tasks.workunit.client.0.smithi114.stdout:removed './189/31' 2024-05-27T09:31:42.408 INFO:tasks.workunit.client.0.smithi114.stdout:removed './189/36' 2024-05-27T09:31:42.409 INFO:tasks.workunit.client.0.smithi114.stdout:removed './189/20' 2024-05-27T09:31:42.409 INFO:tasks.workunit.client.0.smithi114.stdout:removed './189/12' 2024-05-27T09:31:42.409 INFO:tasks.workunit.client.0.smithi114.stdout:removed './189/39' 2024-05-27T09:31:42.409 INFO:tasks.workunit.client.0.smithi114.stdout:removed './189/10' 2024-05-27T09:31:42.409 INFO:tasks.workunit.client.0.smithi114.stdout:removed './189/34' 2024-05-27T09:31:42.409 INFO:tasks.workunit.client.0.smithi114.stdout:removed './189/38' 2024-05-27T09:31:42.409 INFO:tasks.workunit.client.0.smithi114.stdout:removed './189/11' 2024-05-27T09:31:42.409 INFO:tasks.workunit.client.0.smithi114.stdout:removed './189/16' 2024-05-27T09:31:42.409 INFO:tasks.workunit.client.0.smithi114.stdout:removed './189/32' 2024-05-27T09:31:42.409 INFO:tasks.workunit.client.0.smithi114.stdout:removed './189/30' 2024-05-27T09:31:42.410 INFO:tasks.workunit.client.0.smithi114.stdout:removed './189/14' 2024-05-27T09:31:42.410 INFO:tasks.workunit.client.0.smithi114.stdout:removed './189/18' 2024-05-27T09:31:42.410 INFO:tasks.workunit.client.0.smithi114.stdout:removed './189/13' 2024-05-27T09:31:42.410 INFO:tasks.workunit.client.0.smithi114.stdout:removed './189/33' 2024-05-27T09:31:42.410 INFO:tasks.workunit.client.0.smithi114.stdout:removed './189/17' 2024-05-27T09:31:42.410 INFO:tasks.workunit.client.0.smithi114.stdout:removed './189/35' 2024-05-27T09:31:42.410 INFO:tasks.workunit.client.0.smithi114.stdout:removed './189/37' 2024-05-27T09:31:42.410 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './189' 2024-05-27T09:31:42.410 INFO:tasks.workunit.client.0.smithi114.stdout:removed './190/19' 2024-05-27T09:31:42.410 INFO:tasks.workunit.client.0.smithi114.stdout:removed './190/15' 2024-05-27T09:31:42.410 INFO:tasks.workunit.client.0.smithi114.stdout:removed './190/31' 2024-05-27T09:31:42.411 INFO:tasks.workunit.client.0.smithi114.stdout:removed './190/36' 2024-05-27T09:31:42.411 INFO:tasks.workunit.client.0.smithi114.stdout:removed './190/20' 2024-05-27T09:31:42.411 INFO:tasks.workunit.client.0.smithi114.stdout:removed './190/12' 2024-05-27T09:31:42.411 INFO:tasks.workunit.client.0.smithi114.stdout:removed './190/39' 2024-05-27T09:31:42.411 INFO:tasks.workunit.client.0.smithi114.stdout:removed './190/10' 2024-05-27T09:31:42.411 INFO:tasks.workunit.client.0.smithi114.stdout:removed './190/34' 2024-05-27T09:31:42.411 INFO:tasks.workunit.client.0.smithi114.stdout:removed './190/38' 2024-05-27T09:31:42.411 INFO:tasks.workunit.client.0.smithi114.stdout:removed './190/11' 2024-05-27T09:31:42.411 INFO:tasks.workunit.client.0.smithi114.stdout:removed './190/16' 2024-05-27T09:31:42.411 INFO:tasks.workunit.client.0.smithi114.stdout:removed './190/32' 2024-05-27T09:31:42.411 INFO:tasks.workunit.client.0.smithi114.stdout:removed './190/30' 2024-05-27T09:31:42.411 INFO:tasks.workunit.client.0.smithi114.stdout:removed './190/14' 2024-05-27T09:31:42.411 INFO:tasks.workunit.client.0.smithi114.stdout:removed './190/18' 2024-05-27T09:31:42.411 INFO:tasks.workunit.client.0.smithi114.stdout:removed './190/13' 2024-05-27T09:31:42.412 INFO:tasks.workunit.client.0.smithi114.stdout:removed './190/33' 2024-05-27T09:31:42.412 INFO:tasks.workunit.client.0.smithi114.stdout:removed './190/17' 2024-05-27T09:31:42.412 INFO:tasks.workunit.client.0.smithi114.stdout:removed './190/35' 2024-05-27T09:31:42.412 INFO:tasks.workunit.client.0.smithi114.stdout:removed './190/37' 2024-05-27T09:31:42.412 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './190' 2024-05-27T09:31:42.412 INFO:tasks.workunit.client.0.smithi114.stdout:removed './191/19' 2024-05-27T09:31:42.412 INFO:tasks.workunit.client.0.smithi114.stdout:removed './191/15' 2024-05-27T09:31:42.412 INFO:tasks.workunit.client.0.smithi114.stdout:removed './191/31' 2024-05-27T09:31:42.412 INFO:tasks.workunit.client.0.smithi114.stdout:removed './191/36' 2024-05-27T09:31:42.412 INFO:tasks.workunit.client.0.smithi114.stdout:removed './191/20' 2024-05-27T09:31:42.412 INFO:tasks.workunit.client.0.smithi114.stdout:removed './191/12' 2024-05-27T09:31:42.413 INFO:tasks.workunit.client.0.smithi114.stdout:removed './191/39' 2024-05-27T09:31:42.413 INFO:tasks.workunit.client.0.smithi114.stdout:removed './191/10' 2024-05-27T09:31:42.413 INFO:tasks.workunit.client.0.smithi114.stdout:removed './191/34' 2024-05-27T09:31:42.413 INFO:tasks.workunit.client.0.smithi114.stdout:removed './191/38' 2024-05-27T09:31:42.413 INFO:tasks.workunit.client.0.smithi114.stdout:removed './191/11' 2024-05-27T09:31:42.413 INFO:tasks.workunit.client.0.smithi114.stdout:removed './191/16' 2024-05-27T09:31:42.413 INFO:tasks.workunit.client.0.smithi114.stdout:removed './191/32' 2024-05-27T09:31:42.413 INFO:tasks.workunit.client.0.smithi114.stdout:removed './191/30' 2024-05-27T09:31:42.413 INFO:tasks.workunit.client.0.smithi114.stdout:removed './191/14' 2024-05-27T09:31:42.413 INFO:tasks.workunit.client.0.smithi114.stdout:removed './191/18' 2024-05-27T09:31:42.413 INFO:tasks.workunit.client.0.smithi114.stdout:removed './191/13' 2024-05-27T09:31:42.413 INFO:tasks.workunit.client.0.smithi114.stdout:removed './191/33' 2024-05-27T09:31:42.413 INFO:tasks.workunit.client.0.smithi114.stdout:removed './191/17' 2024-05-27T09:31:42.413 INFO:tasks.workunit.client.0.smithi114.stdout:removed './191/35' 2024-05-27T09:31:42.414 INFO:tasks.workunit.client.0.smithi114.stdout:removed './191/37' 2024-05-27T09:31:42.414 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './191' 2024-05-27T09:31:42.414 INFO:tasks.workunit.client.0.smithi114.stdout:removed './192/19' 2024-05-27T09:31:42.414 INFO:tasks.workunit.client.0.smithi114.stdout:removed './192/15' 2024-05-27T09:31:42.414 INFO:tasks.workunit.client.0.smithi114.stdout:removed './192/31' 2024-05-27T09:31:42.414 INFO:tasks.workunit.client.0.smithi114.stdout:removed './192/36' 2024-05-27T09:31:42.414 INFO:tasks.workunit.client.0.smithi114.stdout:removed './192/20' 2024-05-27T09:31:42.414 INFO:tasks.workunit.client.0.smithi114.stdout:removed './192/12' 2024-05-27T09:31:42.414 INFO:tasks.workunit.client.0.smithi114.stdout:removed './192/39' 2024-05-27T09:31:42.414 INFO:tasks.workunit.client.0.smithi114.stdout:removed './192/10' 2024-05-27T09:31:42.414 INFO:tasks.workunit.client.0.smithi114.stdout:removed './192/34' 2024-05-27T09:31:42.414 INFO:tasks.workunit.client.0.smithi114.stdout:removed './192/38' 2024-05-27T09:31:42.414 INFO:tasks.workunit.client.0.smithi114.stdout:removed './192/11' 2024-05-27T09:31:42.415 INFO:tasks.workunit.client.0.smithi114.stdout:removed './192/16' 2024-05-27T09:31:42.415 INFO:tasks.workunit.client.0.smithi114.stdout:removed './192/32' 2024-05-27T09:31:42.415 INFO:tasks.workunit.client.0.smithi114.stdout:removed './192/30' 2024-05-27T09:31:42.415 INFO:tasks.workunit.client.0.smithi114.stdout:removed './192/14' 2024-05-27T09:31:42.415 INFO:tasks.workunit.client.0.smithi114.stdout:removed './192/18' 2024-05-27T09:31:42.415 INFO:tasks.workunit.client.0.smithi114.stdout:removed './192/13' 2024-05-27T09:31:42.415 INFO:tasks.workunit.client.0.smithi114.stdout:removed './192/33' 2024-05-27T09:31:42.415 INFO:tasks.workunit.client.0.smithi114.stdout:removed './192/17' 2024-05-27T09:31:42.415 INFO:tasks.workunit.client.0.smithi114.stdout:removed './192/35' 2024-05-27T09:31:42.415 INFO:tasks.workunit.client.0.smithi114.stdout:removed './192/37' 2024-05-27T09:31:42.415 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './192' 2024-05-27T09:31:42.415 INFO:tasks.workunit.client.0.smithi114.stdout:removed './193/19' 2024-05-27T09:31:42.415 INFO:tasks.workunit.client.0.smithi114.stdout:removed './193/15' 2024-05-27T09:31:42.415 INFO:tasks.workunit.client.0.smithi114.stdout:removed './193/31' 2024-05-27T09:31:42.416 INFO:tasks.workunit.client.0.smithi114.stdout:removed './193/36' 2024-05-27T09:31:42.416 INFO:tasks.workunit.client.0.smithi114.stdout:removed './193/20' 2024-05-27T09:31:42.416 INFO:tasks.workunit.client.0.smithi114.stdout:removed './193/12' 2024-05-27T09:31:42.416 INFO:tasks.workunit.client.0.smithi114.stdout:removed './193/39' 2024-05-27T09:31:42.416 INFO:tasks.workunit.client.0.smithi114.stdout:removed './193/10' 2024-05-27T09:31:42.416 INFO:tasks.workunit.client.0.smithi114.stdout:removed './193/34' 2024-05-27T09:31:42.416 INFO:tasks.workunit.client.0.smithi114.stdout:removed './193/38' 2024-05-27T09:31:42.416 INFO:tasks.workunit.client.0.smithi114.stdout:removed './193/11' 2024-05-27T09:31:42.416 INFO:tasks.workunit.client.0.smithi114.stdout:removed './193/16' 2024-05-27T09:31:42.416 INFO:tasks.workunit.client.0.smithi114.stdout:removed './193/32' 2024-05-27T09:31:42.416 INFO:tasks.workunit.client.0.smithi114.stdout:removed './193/30' 2024-05-27T09:31:42.416 INFO:tasks.workunit.client.0.smithi114.stdout:removed './193/14' 2024-05-27T09:31:42.416 INFO:tasks.workunit.client.0.smithi114.stdout:removed './193/18' 2024-05-27T09:31:42.416 INFO:tasks.workunit.client.0.smithi114.stdout:removed './193/13' 2024-05-27T09:31:42.417 INFO:tasks.workunit.client.0.smithi114.stdout:removed './193/33' 2024-05-27T09:31:42.417 INFO:tasks.workunit.client.0.smithi114.stdout:removed './193/17' 2024-05-27T09:31:42.417 INFO:tasks.workunit.client.0.smithi114.stdout:removed './193/35' 2024-05-27T09:31:42.417 INFO:tasks.workunit.client.0.smithi114.stdout:removed './193/37' 2024-05-27T09:31:42.417 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './193' 2024-05-27T09:31:42.417 INFO:tasks.workunit.client.0.smithi114.stdout:removed './194/19' 2024-05-27T09:31:42.417 INFO:tasks.workunit.client.0.smithi114.stdout:removed './194/15' 2024-05-27T09:31:42.417 INFO:tasks.workunit.client.0.smithi114.stdout:removed './194/31' 2024-05-27T09:31:42.417 INFO:tasks.workunit.client.0.smithi114.stdout:removed './194/36' 2024-05-27T09:31:42.417 INFO:tasks.workunit.client.0.smithi114.stdout:removed './194/20' 2024-05-27T09:31:42.417 INFO:tasks.workunit.client.0.smithi114.stdout:removed './194/12' 2024-05-27T09:31:42.417 INFO:tasks.workunit.client.0.smithi114.stdout:removed './194/39' 2024-05-27T09:31:42.417 INFO:tasks.workunit.client.0.smithi114.stdout:removed './194/10' 2024-05-27T09:31:42.417 INFO:tasks.workunit.client.0.smithi114.stdout:removed './194/34' 2024-05-27T09:31:42.418 INFO:tasks.workunit.client.0.smithi114.stdout:removed './194/38' 2024-05-27T09:31:42.418 INFO:tasks.workunit.client.0.smithi114.stdout:removed './194/11' 2024-05-27T09:31:42.418 INFO:tasks.workunit.client.0.smithi114.stdout:removed './194/16' 2024-05-27T09:31:42.418 INFO:tasks.workunit.client.0.smithi114.stdout:removed './194/32' 2024-05-27T09:31:42.418 INFO:tasks.workunit.client.0.smithi114.stdout:removed './194/30' 2024-05-27T09:31:42.418 INFO:tasks.workunit.client.0.smithi114.stdout:removed './194/14' 2024-05-27T09:31:42.418 INFO:tasks.workunit.client.0.smithi114.stdout:removed './194/18' 2024-05-27T09:31:42.418 INFO:tasks.workunit.client.0.smithi114.stdout:removed './194/13' 2024-05-27T09:31:42.418 INFO:tasks.workunit.client.0.smithi114.stdout:removed './194/33' 2024-05-27T09:31:42.418 INFO:tasks.workunit.client.0.smithi114.stdout:removed './194/17' 2024-05-27T09:31:42.418 INFO:tasks.workunit.client.0.smithi114.stdout:removed './194/35' 2024-05-27T09:31:42.418 INFO:tasks.workunit.client.0.smithi114.stdout:removed './194/37' 2024-05-27T09:31:42.418 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './194' 2024-05-27T09:31:42.418 INFO:tasks.workunit.client.0.smithi114.stdout:removed './195/19' 2024-05-27T09:31:42.418 INFO:tasks.workunit.client.0.smithi114.stdout:removed './195/15' 2024-05-27T09:31:42.419 INFO:tasks.workunit.client.0.smithi114.stdout:removed './195/31' 2024-05-27T09:31:42.419 INFO:tasks.workunit.client.0.smithi114.stdout:removed './195/36' 2024-05-27T09:31:42.419 INFO:tasks.workunit.client.0.smithi114.stdout:removed './195/20' 2024-05-27T09:31:42.419 INFO:tasks.workunit.client.0.smithi114.stdout:removed './195/12' 2024-05-27T09:31:42.419 INFO:tasks.workunit.client.0.smithi114.stdout:removed './195/39' 2024-05-27T09:31:42.419 INFO:tasks.workunit.client.0.smithi114.stdout:removed './195/10' 2024-05-27T09:31:42.419 INFO:tasks.workunit.client.0.smithi114.stdout:removed './195/34' 2024-05-27T09:31:42.419 INFO:tasks.workunit.client.0.smithi114.stdout:removed './195/38' 2024-05-27T09:31:42.419 INFO:tasks.workunit.client.0.smithi114.stdout:removed './195/11' 2024-05-27T09:31:42.419 INFO:tasks.workunit.client.0.smithi114.stdout:removed './195/16' 2024-05-27T09:31:42.419 INFO:tasks.workunit.client.0.smithi114.stdout:removed './195/32' 2024-05-27T09:31:42.419 INFO:tasks.workunit.client.0.smithi114.stdout:removed './195/30' 2024-05-27T09:31:42.419 INFO:tasks.workunit.client.0.smithi114.stdout:removed './195/14' 2024-05-27T09:31:42.419 INFO:tasks.workunit.client.0.smithi114.stdout:removed './195/18' 2024-05-27T09:31:42.419 INFO:tasks.workunit.client.0.smithi114.stdout:removed './195/13' 2024-05-27T09:31:42.420 INFO:tasks.workunit.client.0.smithi114.stdout:removed './195/33' 2024-05-27T09:31:42.420 INFO:tasks.workunit.client.0.smithi114.stdout:removed './195/17' 2024-05-27T09:31:42.420 INFO:tasks.workunit.client.0.smithi114.stdout:removed './195/35' 2024-05-27T09:31:42.420 INFO:tasks.workunit.client.0.smithi114.stdout:removed './195/37' 2024-05-27T09:31:42.420 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './195' 2024-05-27T09:31:42.420 INFO:tasks.workunit.client.0.smithi114.stdout:removed './196/19' 2024-05-27T09:31:42.420 INFO:tasks.workunit.client.0.smithi114.stdout:removed './196/15' 2024-05-27T09:31:42.420 INFO:tasks.workunit.client.0.smithi114.stdout:removed './196/31' 2024-05-27T09:31:42.420 INFO:tasks.workunit.client.0.smithi114.stdout:removed './196/36' 2024-05-27T09:31:42.420 INFO:tasks.workunit.client.0.smithi114.stdout:removed './196/20' 2024-05-27T09:31:42.420 INFO:tasks.workunit.client.0.smithi114.stdout:removed './196/12' 2024-05-27T09:31:42.420 INFO:tasks.workunit.client.0.smithi114.stdout:removed './196/39' 2024-05-27T09:31:42.420 INFO:tasks.workunit.client.0.smithi114.stdout:removed './196/10' 2024-05-27T09:31:43.347 INFO:tasks.workunit.client.0.smithi114.stdout:removed './196/34' 2024-05-27T09:31:43.347 INFO:tasks.workunit.client.0.smithi114.stdout:removed './196/38' 2024-05-27T09:31:43.348 INFO:tasks.workunit.client.0.smithi114.stdout:removed './196/11' 2024-05-27T09:31:43.348 INFO:tasks.workunit.client.0.smithi114.stdout:removed './196/16' 2024-05-27T09:31:43.348 INFO:tasks.workunit.client.0.smithi114.stdout:removed './196/32' 2024-05-27T09:31:43.348 INFO:tasks.workunit.client.0.smithi114.stdout:removed './196/30' 2024-05-27T09:31:43.348 INFO:tasks.workunit.client.0.smithi114.stdout:removed './196/14' 2024-05-27T09:31:43.348 INFO:tasks.workunit.client.0.smithi114.stdout:removed './196/18' 2024-05-27T09:31:43.348 INFO:tasks.workunit.client.0.smithi114.stdout:removed './196/13' 2024-05-27T09:31:43.348 INFO:tasks.workunit.client.0.smithi114.stdout:removed './196/33' 2024-05-27T09:31:43.348 INFO:tasks.workunit.client.0.smithi114.stdout:removed './196/17' 2024-05-27T09:31:43.348 INFO:tasks.workunit.client.0.smithi114.stdout:removed './196/35' 2024-05-27T09:31:43.348 INFO:tasks.workunit.client.0.smithi114.stdout:removed './196/37' 2024-05-27T09:31:43.348 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './196' 2024-05-27T09:31:43.349 INFO:tasks.workunit.client.0.smithi114.stdout:removed './197/19' 2024-05-27T09:31:43.349 INFO:tasks.workunit.client.0.smithi114.stdout:removed './197/15' 2024-05-27T09:31:43.349 INFO:tasks.workunit.client.0.smithi114.stdout:removed './197/31' 2024-05-27T09:31:43.349 INFO:tasks.workunit.client.0.smithi114.stdout:removed './197/36' 2024-05-27T09:31:43.349 INFO:tasks.workunit.client.0.smithi114.stdout:removed './197/20' 2024-05-27T09:31:43.349 INFO:tasks.workunit.client.0.smithi114.stdout:removed './197/12' 2024-05-27T09:31:43.349 INFO:tasks.workunit.client.0.smithi114.stdout:removed './197/39' 2024-05-27T09:31:43.349 INFO:tasks.workunit.client.0.smithi114.stdout:removed './197/10' 2024-05-27T09:31:43.349 INFO:tasks.workunit.client.0.smithi114.stdout:removed './197/34' 2024-05-27T09:31:43.349 INFO:tasks.workunit.client.0.smithi114.stdout:removed './197/38' 2024-05-27T09:31:43.349 INFO:tasks.workunit.client.0.smithi114.stdout:removed './197/11' 2024-05-27T09:31:43.349 INFO:tasks.workunit.client.0.smithi114.stdout:removed './197/16' 2024-05-27T09:31:43.350 INFO:tasks.workunit.client.0.smithi114.stdout:removed './197/32' 2024-05-27T09:31:43.350 INFO:tasks.workunit.client.0.smithi114.stdout:removed './197/30' 2024-05-27T09:31:43.350 INFO:tasks.workunit.client.0.smithi114.stdout:removed './197/14' 2024-05-27T09:31:43.350 INFO:tasks.workunit.client.0.smithi114.stdout:removed './197/18' 2024-05-27T09:31:43.350 INFO:tasks.workunit.client.0.smithi114.stdout:removed './197/13' 2024-05-27T09:31:43.350 INFO:tasks.workunit.client.0.smithi114.stdout:removed './197/33' 2024-05-27T09:31:43.350 INFO:tasks.workunit.client.0.smithi114.stdout:removed './197/17' 2024-05-27T09:31:43.350 INFO:tasks.workunit.client.0.smithi114.stdout:removed './197/35' 2024-05-27T09:31:43.350 INFO:tasks.workunit.client.0.smithi114.stdout:removed './197/37' 2024-05-27T09:31:43.350 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './197' 2024-05-27T09:31:43.350 INFO:tasks.workunit.client.0.smithi114.stdout:removed './198/19' 2024-05-27T09:31:43.350 INFO:tasks.workunit.client.0.smithi114.stdout:removed './198/15' 2024-05-27T09:31:43.350 INFO:tasks.workunit.client.0.smithi114.stdout:removed './198/31' 2024-05-27T09:31:43.350 INFO:tasks.workunit.client.0.smithi114.stdout:removed './198/36' 2024-05-27T09:31:43.351 INFO:tasks.workunit.client.0.smithi114.stdout:removed './198/20' 2024-05-27T09:31:43.351 INFO:tasks.workunit.client.0.smithi114.stdout:removed './198/12' 2024-05-27T09:31:43.351 INFO:tasks.workunit.client.0.smithi114.stdout:removed './198/39' 2024-05-27T09:31:43.351 INFO:tasks.workunit.client.0.smithi114.stdout:removed './198/10' 2024-05-27T09:31:43.351 INFO:tasks.workunit.client.0.smithi114.stdout:removed './198/34' 2024-05-27T09:31:43.351 INFO:tasks.workunit.client.0.smithi114.stdout:removed './198/38' 2024-05-27T09:31:43.351 INFO:tasks.workunit.client.0.smithi114.stdout:removed './198/11' 2024-05-27T09:31:43.351 INFO:tasks.workunit.client.0.smithi114.stdout:removed './198/16' 2024-05-27T09:31:43.351 INFO:tasks.workunit.client.0.smithi114.stdout:removed './198/32' 2024-05-27T09:31:43.351 INFO:tasks.workunit.client.0.smithi114.stdout:removed './198/30' 2024-05-27T09:31:43.351 INFO:tasks.workunit.client.0.smithi114.stdout:removed './198/14' 2024-05-27T09:31:43.351 INFO:tasks.workunit.client.0.smithi114.stdout:removed './198/18' 2024-05-27T09:31:43.351 INFO:tasks.workunit.client.0.smithi114.stdout:removed './198/13' 2024-05-27T09:31:43.352 INFO:tasks.workunit.client.0.smithi114.stdout:removed './198/33' 2024-05-27T09:31:43.352 INFO:tasks.workunit.client.0.smithi114.stdout:removed './198/17' 2024-05-27T09:31:43.352 INFO:tasks.workunit.client.0.smithi114.stdout:removed './198/35' 2024-05-27T09:31:43.352 INFO:tasks.workunit.client.0.smithi114.stdout:removed './198/37' 2024-05-27T09:31:43.352 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './198' 2024-05-27T09:31:43.352 INFO:tasks.workunit.client.0.smithi114.stdout:removed './199/19' 2024-05-27T09:31:43.352 INFO:tasks.workunit.client.0.smithi114.stdout:removed './199/15' 2024-05-27T09:31:43.352 INFO:tasks.workunit.client.0.smithi114.stdout:removed './199/31' 2024-05-27T09:31:43.352 INFO:tasks.workunit.client.0.smithi114.stdout:removed './199/36' 2024-05-27T09:31:43.352 INFO:tasks.workunit.client.0.smithi114.stdout:removed './199/20' 2024-05-27T09:31:43.352 INFO:tasks.workunit.client.0.smithi114.stdout:removed './199/12' 2024-05-27T09:31:43.352 INFO:tasks.workunit.client.0.smithi114.stdout:removed './199/39' 2024-05-27T09:31:43.352 INFO:tasks.workunit.client.0.smithi114.stdout:removed './199/10' 2024-05-27T09:31:43.352 INFO:tasks.workunit.client.0.smithi114.stdout:removed './199/34' 2024-05-27T09:31:43.353 INFO:tasks.workunit.client.0.smithi114.stdout:removed './199/38' 2024-05-27T09:31:43.353 INFO:tasks.workunit.client.0.smithi114.stdout:removed './199/11' 2024-05-27T09:31:43.353 INFO:tasks.workunit.client.0.smithi114.stdout:removed './199/16' 2024-05-27T09:31:43.353 INFO:tasks.workunit.client.0.smithi114.stdout:removed './199/32' 2024-05-27T09:31:43.353 INFO:tasks.workunit.client.0.smithi114.stdout:removed './199/30' 2024-05-27T09:31:43.353 INFO:tasks.workunit.client.0.smithi114.stdout:removed './199/14' 2024-05-27T09:31:43.353 INFO:tasks.workunit.client.0.smithi114.stdout:removed './199/18' 2024-05-27T09:31:43.353 INFO:tasks.workunit.client.0.smithi114.stdout:removed './199/13' 2024-05-27T09:31:43.353 INFO:tasks.workunit.client.0.smithi114.stdout:removed './199/33' 2024-05-27T09:31:43.353 INFO:tasks.workunit.client.0.smithi114.stdout:removed './199/17' 2024-05-27T09:31:43.353 INFO:tasks.workunit.client.0.smithi114.stdout:removed './199/35' 2024-05-27T09:31:43.353 INFO:tasks.workunit.client.0.smithi114.stdout:removed './199/37' 2024-05-27T09:31:43.353 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './199' 2024-05-27T09:31:43.353 INFO:tasks.workunit.client.0.smithi114.stdout:removed './200/21' 2024-05-27T09:31:43.353 INFO:tasks.workunit.client.0.smithi114.stdout:removed './200/27' 2024-05-27T09:31:43.354 INFO:tasks.workunit.client.0.smithi114.stdout:removed './200/31' 2024-05-27T09:31:43.354 INFO:tasks.workunit.client.0.smithi114.stdout:removed './200/36' 2024-05-27T09:31:43.354 INFO:tasks.workunit.client.0.smithi114.stdout:removed './200/20' 2024-05-27T09:31:43.354 INFO:tasks.workunit.client.0.smithi114.stdout:removed './200/39' 2024-05-27T09:31:43.354 INFO:tasks.workunit.client.0.smithi114.stdout:removed './200/34' 2024-05-27T09:31:43.354 INFO:tasks.workunit.client.0.smithi114.stdout:removed './200/38' 2024-05-27T09:31:43.354 INFO:tasks.workunit.client.0.smithi114.stdout:removed './200/28' 2024-05-27T09:31:43.354 INFO:tasks.workunit.client.0.smithi114.stdout:removed './200/24' 2024-05-27T09:31:43.354 INFO:tasks.workunit.client.0.smithi114.stdout:removed './200/29' 2024-05-27T09:31:43.354 INFO:tasks.workunit.client.0.smithi114.stdout:removed './200/32' 2024-05-27T09:31:43.354 INFO:tasks.workunit.client.0.smithi114.stdout:removed './200/22' 2024-05-27T09:31:43.354 INFO:tasks.workunit.client.0.smithi114.stdout:removed './200/25' 2024-05-27T09:31:43.354 INFO:tasks.workunit.client.0.smithi114.stdout:removed './200/30' 2024-05-27T09:31:43.355 INFO:tasks.workunit.client.0.smithi114.stdout:removed './200/33' 2024-05-27T09:31:43.355 INFO:tasks.workunit.client.0.smithi114.stdout:removed './200/23' 2024-05-27T09:31:43.355 INFO:tasks.workunit.client.0.smithi114.stdout:removed './200/35' 2024-05-27T09:31:43.355 INFO:tasks.workunit.client.0.smithi114.stdout:removed './200/26' 2024-05-27T09:31:43.355 INFO:tasks.workunit.client.0.smithi114.stdout:removed './200/37' 2024-05-27T09:31:43.355 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './200' 2024-05-27T09:31:43.355 INFO:tasks.workunit.client.0.smithi114.stdout:removed './201/21' 2024-05-27T09:31:43.355 INFO:tasks.workunit.client.0.smithi114.stdout:removed './201/27' 2024-05-27T09:31:43.355 INFO:tasks.workunit.client.0.smithi114.stdout:removed './201/31' 2024-05-27T09:31:43.355 INFO:tasks.workunit.client.0.smithi114.stdout:removed './201/36' 2024-05-27T09:31:43.355 INFO:tasks.workunit.client.0.smithi114.stdout:removed './201/20' 2024-05-27T09:31:43.355 INFO:tasks.workunit.client.0.smithi114.stdout:removed './201/39' 2024-05-27T09:31:43.355 INFO:tasks.workunit.client.0.smithi114.stdout:removed './201/34' 2024-05-27T09:31:43.355 INFO:tasks.workunit.client.0.smithi114.stdout:removed './201/38' 2024-05-27T09:31:43.356 INFO:tasks.workunit.client.0.smithi114.stdout:removed './201/28' 2024-05-27T09:31:43.356 INFO:tasks.workunit.client.0.smithi114.stdout:removed './201/24' 2024-05-27T09:31:43.356 INFO:tasks.workunit.client.0.smithi114.stdout:removed './201/29' 2024-05-27T09:31:43.356 INFO:tasks.workunit.client.0.smithi114.stdout:removed './201/32' 2024-05-27T09:31:43.356 INFO:tasks.workunit.client.0.smithi114.stdout:removed './201/22' 2024-05-27T09:31:43.356 INFO:tasks.workunit.client.0.smithi114.stdout:removed './201/25' 2024-05-27T09:31:43.356 INFO:tasks.workunit.client.0.smithi114.stdout:removed './201/30' 2024-05-27T09:31:43.356 INFO:tasks.workunit.client.0.smithi114.stdout:removed './201/33' 2024-05-27T09:31:43.356 INFO:tasks.workunit.client.0.smithi114.stdout:removed './201/23' 2024-05-27T09:31:43.356 INFO:tasks.workunit.client.0.smithi114.stdout:removed './201/35' 2024-05-27T09:31:43.356 INFO:tasks.workunit.client.0.smithi114.stdout:removed './201/26' 2024-05-27T09:31:43.356 INFO:tasks.workunit.client.0.smithi114.stdout:removed './201/37' 2024-05-27T09:31:43.356 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './201' 2024-05-27T09:31:43.356 INFO:tasks.workunit.client.0.smithi114.stdout:removed './202/21' 2024-05-27T09:31:43.357 INFO:tasks.workunit.client.0.smithi114.stdout:removed './202/27' 2024-05-27T09:31:43.357 INFO:tasks.workunit.client.0.smithi114.stdout:removed './202/31' 2024-05-27T09:31:43.357 INFO:tasks.workunit.client.0.smithi114.stdout:removed './202/36' 2024-05-27T09:31:43.357 INFO:tasks.workunit.client.0.smithi114.stdout:removed './202/20' 2024-05-27T09:31:43.357 INFO:tasks.workunit.client.0.smithi114.stdout:removed './202/39' 2024-05-27T09:31:43.357 INFO:tasks.workunit.client.0.smithi114.stdout:removed './202/34' 2024-05-27T09:31:43.357 INFO:tasks.workunit.client.0.smithi114.stdout:removed './202/38' 2024-05-27T09:31:43.357 INFO:tasks.workunit.client.0.smithi114.stdout:removed './202/28' 2024-05-27T09:31:43.357 INFO:tasks.workunit.client.0.smithi114.stdout:removed './202/24' 2024-05-27T09:31:43.357 INFO:tasks.workunit.client.0.smithi114.stdout:removed './202/29' 2024-05-27T09:31:43.357 INFO:tasks.workunit.client.0.smithi114.stdout:removed './202/32' 2024-05-27T09:31:43.357 INFO:tasks.workunit.client.0.smithi114.stdout:removed './202/22' 2024-05-27T09:31:43.357 INFO:tasks.workunit.client.0.smithi114.stdout:removed './202/25' 2024-05-27T09:31:43.357 INFO:tasks.workunit.client.0.smithi114.stdout:removed './202/30' 2024-05-27T09:31:43.358 INFO:tasks.workunit.client.0.smithi114.stdout:removed './202/33' 2024-05-27T09:31:43.358 INFO:tasks.workunit.client.0.smithi114.stdout:removed './202/23' 2024-05-27T09:31:43.358 INFO:tasks.workunit.client.0.smithi114.stdout:removed './202/35' 2024-05-27T09:31:43.358 INFO:tasks.workunit.client.0.smithi114.stdout:removed './202/26' 2024-05-27T09:31:43.358 INFO:tasks.workunit.client.0.smithi114.stdout:removed './202/37' 2024-05-27T09:31:43.358 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './202' 2024-05-27T09:31:43.358 INFO:tasks.workunit.client.0.smithi114.stdout:removed './203/21' 2024-05-27T09:31:43.358 INFO:tasks.workunit.client.0.smithi114.stdout:removed './203/27' 2024-05-27T09:31:43.358 INFO:tasks.workunit.client.0.smithi114.stdout:removed './203/31' 2024-05-27T09:31:43.358 INFO:tasks.workunit.client.0.smithi114.stdout:removed './203/36' 2024-05-27T09:31:43.358 INFO:tasks.workunit.client.0.smithi114.stdout:removed './203/20' 2024-05-27T09:31:43.358 INFO:tasks.workunit.client.0.smithi114.stdout:removed './203/39' 2024-05-27T09:31:43.358 INFO:tasks.workunit.client.0.smithi114.stdout:removed './203/34' 2024-05-27T09:31:43.359 INFO:tasks.workunit.client.0.smithi114.stdout:removed './203/38' 2024-05-27T09:31:43.359 INFO:tasks.workunit.client.0.smithi114.stdout:removed './203/28' 2024-05-27T09:31:43.359 INFO:tasks.workunit.client.0.smithi114.stdout:removed './203/24' 2024-05-27T09:31:43.359 INFO:tasks.workunit.client.0.smithi114.stdout:removed './203/29' 2024-05-27T09:31:43.359 INFO:tasks.workunit.client.0.smithi114.stdout:removed './203/32' 2024-05-27T09:31:43.359 INFO:tasks.workunit.client.0.smithi114.stdout:removed './203/22' 2024-05-27T09:31:43.359 INFO:tasks.workunit.client.0.smithi114.stdout:removed './203/25' 2024-05-27T09:31:43.359 INFO:tasks.workunit.client.0.smithi114.stdout:removed './203/30' 2024-05-27T09:31:43.359 INFO:tasks.workunit.client.0.smithi114.stdout:removed './203/33' 2024-05-27T09:31:43.359 INFO:tasks.workunit.client.0.smithi114.stdout:removed './203/23' 2024-05-27T09:31:43.359 INFO:tasks.workunit.client.0.smithi114.stdout:removed './203/35' 2024-05-27T09:31:43.359 INFO:tasks.workunit.client.0.smithi114.stdout:removed './203/26' 2024-05-27T09:31:43.359 INFO:tasks.workunit.client.0.smithi114.stdout:removed './203/37' 2024-05-27T09:31:43.359 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './203' 2024-05-27T09:31:43.360 INFO:tasks.workunit.client.0.smithi114.stdout:removed './204/21' 2024-05-27T09:31:43.360 INFO:tasks.workunit.client.0.smithi114.stdout:removed './204/27' 2024-05-27T09:31:43.360 INFO:tasks.workunit.client.0.smithi114.stdout:removed './204/31' 2024-05-27T09:31:43.360 INFO:tasks.workunit.client.0.smithi114.stdout:removed './204/36' 2024-05-27T09:31:43.360 INFO:tasks.workunit.client.0.smithi114.stdout:removed './204/20' 2024-05-27T09:31:43.360 INFO:tasks.workunit.client.0.smithi114.stdout:removed './204/39' 2024-05-27T09:31:43.360 INFO:tasks.workunit.client.0.smithi114.stdout:removed './204/34' 2024-05-27T09:31:43.360 INFO:tasks.workunit.client.0.smithi114.stdout:removed './204/38' 2024-05-27T09:31:43.360 INFO:tasks.workunit.client.0.smithi114.stdout:removed './204/28' 2024-05-27T09:31:43.360 INFO:tasks.workunit.client.0.smithi114.stdout:removed './204/24' 2024-05-27T09:31:43.360 INFO:tasks.workunit.client.0.smithi114.stdout:removed './204/29' 2024-05-27T09:31:43.360 INFO:tasks.workunit.client.0.smithi114.stdout:removed './204/32' 2024-05-27T09:31:43.360 INFO:tasks.workunit.client.0.smithi114.stdout:removed './204/22' 2024-05-27T09:31:43.360 INFO:tasks.workunit.client.0.smithi114.stdout:removed './204/25' 2024-05-27T09:31:43.361 INFO:tasks.workunit.client.0.smithi114.stdout:removed './204/30' 2024-05-27T09:31:43.361 INFO:tasks.workunit.client.0.smithi114.stdout:removed './204/33' 2024-05-27T09:31:43.361 INFO:tasks.workunit.client.0.smithi114.stdout:removed './204/23' 2024-05-27T09:31:43.361 INFO:tasks.workunit.client.0.smithi114.stdout:removed './204/35' 2024-05-27T09:31:43.361 INFO:tasks.workunit.client.0.smithi114.stdout:removed './204/26' 2024-05-27T09:31:43.361 INFO:tasks.workunit.client.0.smithi114.stdout:removed './204/37' 2024-05-27T09:31:43.361 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './204' 2024-05-27T09:31:43.361 INFO:tasks.workunit.client.0.smithi114.stdout:removed './205/21' 2024-05-27T09:31:43.361 INFO:tasks.workunit.client.0.smithi114.stdout:removed './205/27' 2024-05-27T09:31:43.361 INFO:tasks.workunit.client.0.smithi114.stdout:removed './205/31' 2024-05-27T09:31:43.361 INFO:tasks.workunit.client.0.smithi114.stdout:removed './205/36' 2024-05-27T09:31:43.362 INFO:tasks.workunit.client.0.smithi114.stdout:removed './205/20' 2024-05-27T09:31:43.362 INFO:tasks.workunit.client.0.smithi114.stdout:removed './205/39' 2024-05-27T09:31:43.362 INFO:tasks.workunit.client.0.smithi114.stdout:removed './205/34' 2024-05-27T09:31:43.362 INFO:tasks.workunit.client.0.smithi114.stdout:removed './205/38' 2024-05-27T09:31:43.362 INFO:tasks.workunit.client.0.smithi114.stdout:removed './205/28' 2024-05-27T09:31:43.362 INFO:tasks.workunit.client.0.smithi114.stdout:removed './205/24' 2024-05-27T09:31:43.362 INFO:tasks.workunit.client.0.smithi114.stdout:removed './205/29' 2024-05-27T09:31:43.362 INFO:tasks.workunit.client.0.smithi114.stdout:removed './205/32' 2024-05-27T09:31:43.362 INFO:tasks.workunit.client.0.smithi114.stdout:removed './205/22' 2024-05-27T09:31:43.362 INFO:tasks.workunit.client.0.smithi114.stdout:removed './205/25' 2024-05-27T09:31:43.362 INFO:tasks.workunit.client.0.smithi114.stdout:removed './205/30' 2024-05-27T09:31:43.362 INFO:tasks.workunit.client.0.smithi114.stdout:removed './205/33' 2024-05-27T09:31:43.362 INFO:tasks.workunit.client.0.smithi114.stdout:removed './205/23' 2024-05-27T09:31:43.362 INFO:tasks.workunit.client.0.smithi114.stdout:removed './205/35' 2024-05-27T09:31:43.363 INFO:tasks.workunit.client.0.smithi114.stdout:removed './205/26' 2024-05-27T09:31:43.363 INFO:tasks.workunit.client.0.smithi114.stdout:removed './205/37' 2024-05-27T09:31:43.363 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './205' 2024-05-27T09:31:43.363 INFO:tasks.workunit.client.0.smithi114.stdout:removed './206/21' 2024-05-27T09:31:43.363 INFO:tasks.workunit.client.0.smithi114.stdout:removed './206/27' 2024-05-27T09:31:43.363 INFO:tasks.workunit.client.0.smithi114.stdout:removed './206/31' 2024-05-27T09:31:43.363 INFO:tasks.workunit.client.0.smithi114.stdout:removed './206/36' 2024-05-27T09:31:43.363 INFO:tasks.workunit.client.0.smithi114.stdout:removed './206/20' 2024-05-27T09:31:43.363 INFO:tasks.workunit.client.0.smithi114.stdout:removed './206/39' 2024-05-27T09:31:44.550 INFO:tasks.workunit.client.0.smithi114.stdout:removed './206/34' 2024-05-27T09:31:44.550 INFO:tasks.workunit.client.0.smithi114.stdout:removed './206/38' 2024-05-27T09:31:44.550 INFO:tasks.workunit.client.0.smithi114.stdout:removed './206/28' 2024-05-27T09:31:44.550 INFO:tasks.workunit.client.0.smithi114.stdout:removed './206/24' 2024-05-27T09:31:44.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './206/29' 2024-05-27T09:31:44.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './206/32' 2024-05-27T09:31:44.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './206/22' 2024-05-27T09:31:44.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './206/25' 2024-05-27T09:31:44.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './206/30' 2024-05-27T09:31:44.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './206/33' 2024-05-27T09:31:44.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './206/23' 2024-05-27T09:31:44.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './206/35' 2024-05-27T09:31:44.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './206/26' 2024-05-27T09:31:44.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './206/37' 2024-05-27T09:31:44.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './206' 2024-05-27T09:31:44.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './207/21' 2024-05-27T09:31:44.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './207/27' 2024-05-27T09:31:44.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './207/31' 2024-05-27T09:31:44.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './207/36' 2024-05-27T09:31:44.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './207/20' 2024-05-27T09:31:44.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './207/39' 2024-05-27T09:31:44.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './207/34' 2024-05-27T09:31:44.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './207/38' 2024-05-27T09:31:44.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './207/28' 2024-05-27T09:31:44.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './207/24' 2024-05-27T09:31:44.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './207/29' 2024-05-27T09:31:44.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './207/32' 2024-05-27T09:31:44.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './207/22' 2024-05-27T09:31:44.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './207/25' 2024-05-27T09:31:44.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './207/30' 2024-05-27T09:31:44.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './207/33' 2024-05-27T09:31:44.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './207/23' 2024-05-27T09:31:44.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './207/35' 2024-05-27T09:31:44.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './207/26' 2024-05-27T09:31:44.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './207/37' 2024-05-27T09:31:44.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './207' 2024-05-27T09:31:44.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './208/21' 2024-05-27T09:31:44.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './208/27' 2024-05-27T09:31:44.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './208/31' 2024-05-27T09:31:44.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './208/36' 2024-05-27T09:31:44.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './208/20' 2024-05-27T09:31:44.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './208/39' 2024-05-27T09:31:44.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './208/34' 2024-05-27T09:31:44.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './208/38' 2024-05-27T09:31:44.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './208/28' 2024-05-27T09:31:44.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './208/24' 2024-05-27T09:31:44.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './208/29' 2024-05-27T09:31:44.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './208/32' 2024-05-27T09:31:44.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './208/22' 2024-05-27T09:31:44.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './208/25' 2024-05-27T09:31:44.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './208/30' 2024-05-27T09:31:44.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './208/33' 2024-05-27T09:31:44.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './208/23' 2024-05-27T09:31:44.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './208/35' 2024-05-27T09:31:44.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './208/26' 2024-05-27T09:31:44.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './208/37' 2024-05-27T09:31:44.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './208' 2024-05-27T09:31:44.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './209/21' 2024-05-27T09:31:44.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './209/27' 2024-05-27T09:31:44.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './209/31' 2024-05-27T09:31:44.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './209/36' 2024-05-27T09:31:44.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './209/20' 2024-05-27T09:31:44.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './209/39' 2024-05-27T09:31:44.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './209/34' 2024-05-27T09:31:44.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './209/38' 2024-05-27T09:31:44.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './209/28' 2024-05-27T09:31:44.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './209/24' 2024-05-27T09:31:44.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './209/29' 2024-05-27T09:31:44.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './209/32' 2024-05-27T09:31:44.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './209/22' 2024-05-27T09:31:44.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './209/25' 2024-05-27T09:31:44.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './209/30' 2024-05-27T09:31:44.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './209/33' 2024-05-27T09:31:44.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './209/23' 2024-05-27T09:31:44.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './209/35' 2024-05-27T09:31:44.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './209/26' 2024-05-27T09:31:44.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './209/37' 2024-05-27T09:31:44.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './209' 2024-05-27T09:31:44.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './210/21' 2024-05-27T09:31:44.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './210/27' 2024-05-27T09:31:44.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './210/31' 2024-05-27T09:31:44.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './210/36' 2024-05-27T09:31:44.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './210/20' 2024-05-27T09:31:44.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './210/39' 2024-05-27T09:31:44.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './210/34' 2024-05-27T09:31:44.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './210/38' 2024-05-27T09:31:44.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './210/28' 2024-05-27T09:31:44.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './210/24' 2024-05-27T09:31:44.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './210/29' 2024-05-27T09:31:44.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './210/32' 2024-05-27T09:31:44.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './210/22' 2024-05-27T09:31:44.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './210/25' 2024-05-27T09:31:44.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './210/30' 2024-05-27T09:31:44.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './210/33' 2024-05-27T09:31:44.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './210/23' 2024-05-27T09:31:44.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './210/35' 2024-05-27T09:31:44.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './210/26' 2024-05-27T09:31:44.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './210/37' 2024-05-27T09:31:44.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './210' 2024-05-27T09:31:44.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './211/21' 2024-05-27T09:31:44.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './211/27' 2024-05-27T09:31:44.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './211/31' 2024-05-27T09:31:44.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './211/36' 2024-05-27T09:31:44.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './211/20' 2024-05-27T09:31:44.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './211/39' 2024-05-27T09:31:44.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './211/34' 2024-05-27T09:31:44.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './211/38' 2024-05-27T09:31:44.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './211/28' 2024-05-27T09:31:44.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './211/24' 2024-05-27T09:31:44.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './211/29' 2024-05-27T09:31:44.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './211/32' 2024-05-27T09:31:44.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './211/22' 2024-05-27T09:31:44.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './211/25' 2024-05-27T09:31:44.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './211/30' 2024-05-27T09:31:44.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './211/33' 2024-05-27T09:31:44.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './211/23' 2024-05-27T09:31:44.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './211/35' 2024-05-27T09:31:44.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './211/26' 2024-05-27T09:31:44.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './211/37' 2024-05-27T09:31:44.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './211' 2024-05-27T09:31:44.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './212/21' 2024-05-27T09:31:44.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './212/27' 2024-05-27T09:31:44.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './212/31' 2024-05-27T09:31:44.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './212/36' 2024-05-27T09:31:44.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './212/20' 2024-05-27T09:31:44.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './212/39' 2024-05-27T09:31:44.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './212/34' 2024-05-27T09:31:44.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './212/38' 2024-05-27T09:31:44.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './212/28' 2024-05-27T09:31:44.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './212/24' 2024-05-27T09:31:44.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './212/29' 2024-05-27T09:31:44.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './212/32' 2024-05-27T09:31:44.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './212/22' 2024-05-27T09:31:44.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './212/25' 2024-05-27T09:31:44.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './212/30' 2024-05-27T09:31:44.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './212/33' 2024-05-27T09:31:44.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './212/23' 2024-05-27T09:31:44.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './212/35' 2024-05-27T09:31:44.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './212/26' 2024-05-27T09:31:44.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './212/37' 2024-05-27T09:31:44.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './212' 2024-05-27T09:31:44.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed './213/21' 2024-05-27T09:31:44.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed './213/27' 2024-05-27T09:31:44.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed './213/31' 2024-05-27T09:31:44.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed './213/36' 2024-05-27T09:31:44.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed './213/20' 2024-05-27T09:31:44.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed './213/39' 2024-05-27T09:31:44.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed './213/34' 2024-05-27T09:31:44.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed './213/38' 2024-05-27T09:31:44.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed './213/28' 2024-05-27T09:31:44.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed './213/24' 2024-05-27T09:31:44.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed './213/29' 2024-05-27T09:31:44.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed './213/32' 2024-05-27T09:31:44.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed './213/22' 2024-05-27T09:31:44.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed './213/25' 2024-05-27T09:31:44.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed './213/30' 2024-05-27T09:31:44.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed './213/33' 2024-05-27T09:31:44.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed './213/23' 2024-05-27T09:31:44.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed './213/35' 2024-05-27T09:31:44.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed './213/26' 2024-05-27T09:31:44.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed './213/37' 2024-05-27T09:31:44.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './213' 2024-05-27T09:31:44.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed './214/21' 2024-05-27T09:31:44.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed './214/27' 2024-05-27T09:31:44.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed './214/31' 2024-05-27T09:31:44.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './214/36' 2024-05-27T09:31:44.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './214/20' 2024-05-27T09:31:44.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './214/39' 2024-05-27T09:31:44.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './214/34' 2024-05-27T09:31:44.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './214/38' 2024-05-27T09:31:44.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './214/28' 2024-05-27T09:31:44.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './214/24' 2024-05-27T09:31:44.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './214/29' 2024-05-27T09:31:44.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './214/32' 2024-05-27T09:31:44.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './214/22' 2024-05-27T09:31:44.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './214/25' 2024-05-27T09:31:44.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './214/30' 2024-05-27T09:31:44.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './214/33' 2024-05-27T09:31:44.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './214/23' 2024-05-27T09:31:44.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './214/35' 2024-05-27T09:31:44.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './214/26' 2024-05-27T09:31:44.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './214/37' 2024-05-27T09:31:44.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './214' 2024-05-27T09:31:44.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './215/21' 2024-05-27T09:31:44.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './215/27' 2024-05-27T09:31:44.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './215/31' 2024-05-27T09:31:44.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './215/36' 2024-05-27T09:31:44.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './215/20' 2024-05-27T09:31:44.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './215/39' 2024-05-27T09:31:44.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './215/34' 2024-05-27T09:31:44.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './215/38' 2024-05-27T09:31:44.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './215/28' 2024-05-27T09:31:44.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './215/24' 2024-05-27T09:31:44.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './215/29' 2024-05-27T09:31:44.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './215/32' 2024-05-27T09:31:44.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './215/22' 2024-05-27T09:31:44.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './215/25' 2024-05-27T09:31:44.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './215/30' 2024-05-27T09:31:44.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './215/33' 2024-05-27T09:31:44.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './215/23' 2024-05-27T09:31:44.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './215/35' 2024-05-27T09:31:44.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './215/26' 2024-05-27T09:31:44.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './215/37' 2024-05-27T09:31:44.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './215' 2024-05-27T09:31:44.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './216/21' 2024-05-27T09:31:44.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './216/27' 2024-05-27T09:31:44.566 INFO:tasks.workunit.client.0.smithi114.stdout:removed './216/31' 2024-05-27T09:31:44.566 INFO:tasks.workunit.client.0.smithi114.stdout:removed './216/36' 2024-05-27T09:31:44.566 INFO:tasks.workunit.client.0.smithi114.stdout:removed './216/20' 2024-05-27T09:31:44.566 INFO:tasks.workunit.client.0.smithi114.stdout:removed './216/39' 2024-05-27T09:31:44.566 INFO:tasks.workunit.client.0.smithi114.stdout:removed './216/34' 2024-05-27T09:31:44.566 INFO:tasks.workunit.client.0.smithi114.stdout:removed './216/38' 2024-05-27T09:31:45.548 INFO:tasks.workunit.client.0.smithi114.stdout:removed './216/28' 2024-05-27T09:31:45.549 INFO:tasks.workunit.client.0.smithi114.stdout:removed './216/24' 2024-05-27T09:31:45.549 INFO:tasks.workunit.client.0.smithi114.stdout:removed './216/29' 2024-05-27T09:31:45.549 INFO:tasks.workunit.client.0.smithi114.stdout:removed './216/32' 2024-05-27T09:31:45.549 INFO:tasks.workunit.client.0.smithi114.stdout:removed './216/22' 2024-05-27T09:31:45.549 INFO:tasks.workunit.client.0.smithi114.stdout:removed './216/25' 2024-05-27T09:31:45.549 INFO:tasks.workunit.client.0.smithi114.stdout:removed './216/30' 2024-05-27T09:31:45.549 INFO:tasks.workunit.client.0.smithi114.stdout:removed './216/33' 2024-05-27T09:31:45.549 INFO:tasks.workunit.client.0.smithi114.stdout:removed './216/23' 2024-05-27T09:31:45.549 INFO:tasks.workunit.client.0.smithi114.stdout:removed './216/35' 2024-05-27T09:31:45.549 INFO:tasks.workunit.client.0.smithi114.stdout:removed './216/26' 2024-05-27T09:31:45.549 INFO:tasks.workunit.client.0.smithi114.stdout:removed './216/37' 2024-05-27T09:31:45.549 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './216' 2024-05-27T09:31:45.549 INFO:tasks.workunit.client.0.smithi114.stdout:removed './217/21' 2024-05-27T09:31:45.550 INFO:tasks.workunit.client.0.smithi114.stdout:removed './217/27' 2024-05-27T09:31:45.550 INFO:tasks.workunit.client.0.smithi114.stdout:removed './217/31' 2024-05-27T09:31:45.550 INFO:tasks.workunit.client.0.smithi114.stdout:removed './217/36' 2024-05-27T09:31:45.550 INFO:tasks.workunit.client.0.smithi114.stdout:removed './217/20' 2024-05-27T09:31:45.550 INFO:tasks.workunit.client.0.smithi114.stdout:removed './217/39' 2024-05-27T09:31:45.550 INFO:tasks.workunit.client.0.smithi114.stdout:removed './217/34' 2024-05-27T09:31:45.550 INFO:tasks.workunit.client.0.smithi114.stdout:removed './217/38' 2024-05-27T09:31:45.550 INFO:tasks.workunit.client.0.smithi114.stdout:removed './217/28' 2024-05-27T09:31:45.550 INFO:tasks.workunit.client.0.smithi114.stdout:removed './217/24' 2024-05-27T09:31:45.550 INFO:tasks.workunit.client.0.smithi114.stdout:removed './217/29' 2024-05-27T09:31:45.550 INFO:tasks.workunit.client.0.smithi114.stdout:removed './217/32' 2024-05-27T09:31:45.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './217/22' 2024-05-27T09:31:45.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './217/25' 2024-05-27T09:31:45.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './217/30' 2024-05-27T09:31:45.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './217/33' 2024-05-27T09:31:45.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './217/23' 2024-05-27T09:31:45.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './217/35' 2024-05-27T09:31:45.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './217/26' 2024-05-27T09:31:45.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './217/37' 2024-05-27T09:31:45.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './217' 2024-05-27T09:31:45.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './218/21' 2024-05-27T09:31:45.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './218/27' 2024-05-27T09:31:45.551 INFO:tasks.workunit.client.0.smithi114.stdout:removed './218/31' 2024-05-27T09:31:45.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './218/36' 2024-05-27T09:31:45.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './218/20' 2024-05-27T09:31:45.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './218/39' 2024-05-27T09:31:45.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './218/34' 2024-05-27T09:31:45.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './218/38' 2024-05-27T09:31:45.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './218/28' 2024-05-27T09:31:45.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './218/24' 2024-05-27T09:31:45.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './218/29' 2024-05-27T09:31:45.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './218/32' 2024-05-27T09:31:45.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './218/22' 2024-05-27T09:31:45.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './218/25' 2024-05-27T09:31:45.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './218/30' 2024-05-27T09:31:45.552 INFO:tasks.workunit.client.0.smithi114.stdout:removed './218/33' 2024-05-27T09:31:45.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './218/23' 2024-05-27T09:31:45.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './218/35' 2024-05-27T09:31:45.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './218/26' 2024-05-27T09:31:45.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './218/37' 2024-05-27T09:31:45.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './218' 2024-05-27T09:31:45.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './219/21' 2024-05-27T09:31:45.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './219/27' 2024-05-27T09:31:45.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './219/31' 2024-05-27T09:31:45.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './219/36' 2024-05-27T09:31:45.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './219/20' 2024-05-27T09:31:45.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './219/39' 2024-05-27T09:31:45.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './219/34' 2024-05-27T09:31:45.553 INFO:tasks.workunit.client.0.smithi114.stdout:removed './219/38' 2024-05-27T09:31:45.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './219/28' 2024-05-27T09:31:45.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './219/24' 2024-05-27T09:31:45.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './219/29' 2024-05-27T09:31:45.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './219/32' 2024-05-27T09:31:45.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './219/22' 2024-05-27T09:31:45.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './219/25' 2024-05-27T09:31:45.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './219/30' 2024-05-27T09:31:45.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './219/33' 2024-05-27T09:31:45.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './219/23' 2024-05-27T09:31:45.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './219/35' 2024-05-27T09:31:45.554 INFO:tasks.workunit.client.0.smithi114.stdout:removed './219/26' 2024-05-27T09:31:45.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './219/37' 2024-05-27T09:31:45.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './219' 2024-05-27T09:31:45.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './220/21' 2024-05-27T09:31:45.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './220/27' 2024-05-27T09:31:45.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './220/31' 2024-05-27T09:31:45.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './220/36' 2024-05-27T09:31:45.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './220/20' 2024-05-27T09:31:45.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './220/39' 2024-05-27T09:31:45.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './220/34' 2024-05-27T09:31:45.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './220/38' 2024-05-27T09:31:45.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './220/28' 2024-05-27T09:31:45.555 INFO:tasks.workunit.client.0.smithi114.stdout:removed './220/24' 2024-05-27T09:31:45.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './220/29' 2024-05-27T09:31:45.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './220/32' 2024-05-27T09:31:45.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './220/22' 2024-05-27T09:31:45.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './220/25' 2024-05-27T09:31:45.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './220/30' 2024-05-27T09:31:45.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './220/33' 2024-05-27T09:31:45.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './220/23' 2024-05-27T09:31:45.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './220/35' 2024-05-27T09:31:45.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './220/26' 2024-05-27T09:31:45.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './220/37' 2024-05-27T09:31:45.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './220' 2024-05-27T09:31:45.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './221/21' 2024-05-27T09:31:45.556 INFO:tasks.workunit.client.0.smithi114.stdout:removed './221/27' 2024-05-27T09:31:45.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './221/31' 2024-05-27T09:31:45.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './221/36' 2024-05-27T09:31:45.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './221/20' 2024-05-27T09:31:45.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './221/39' 2024-05-27T09:31:45.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './221/34' 2024-05-27T09:31:45.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './221/38' 2024-05-27T09:31:45.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './221/28' 2024-05-27T09:31:45.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './221/24' 2024-05-27T09:31:45.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './221/29' 2024-05-27T09:31:45.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './221/32' 2024-05-27T09:31:45.557 INFO:tasks.workunit.client.0.smithi114.stdout:removed './221/22' 2024-05-27T09:31:45.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './221/25' 2024-05-27T09:31:45.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './221/30' 2024-05-27T09:31:45.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './221/33' 2024-05-27T09:31:45.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './221/23' 2024-05-27T09:31:45.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './221/35' 2024-05-27T09:31:45.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './221/26' 2024-05-27T09:31:45.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './221/37' 2024-05-27T09:31:45.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './221' 2024-05-27T09:31:45.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './222/21' 2024-05-27T09:31:45.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './222/27' 2024-05-27T09:31:45.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './222/31' 2024-05-27T09:31:45.558 INFO:tasks.workunit.client.0.smithi114.stdout:removed './222/36' 2024-05-27T09:31:45.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './222/20' 2024-05-27T09:31:45.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './222/39' 2024-05-27T09:31:45.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './222/34' 2024-05-27T09:31:45.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './222/38' 2024-05-27T09:31:45.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './222/28' 2024-05-27T09:31:45.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './222/24' 2024-05-27T09:31:45.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './222/29' 2024-05-27T09:31:45.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './222/32' 2024-05-27T09:31:45.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './222/22' 2024-05-27T09:31:45.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './222/25' 2024-05-27T09:31:45.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './222/30' 2024-05-27T09:31:45.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './222/33' 2024-05-27T09:31:45.559 INFO:tasks.workunit.client.0.smithi114.stdout:removed './222/23' 2024-05-27T09:31:45.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './222/35' 2024-05-27T09:31:45.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './222/26' 2024-05-27T09:31:45.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './222/37' 2024-05-27T09:31:45.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './222' 2024-05-27T09:31:45.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './223/21' 2024-05-27T09:31:45.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './223/27' 2024-05-27T09:31:45.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './223/31' 2024-05-27T09:31:45.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './223/36' 2024-05-27T09:31:45.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './223/20' 2024-05-27T09:31:45.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './223/39' 2024-05-27T09:31:45.560 INFO:tasks.workunit.client.0.smithi114.stdout:removed './223/34' 2024-05-27T09:31:45.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed './223/38' 2024-05-27T09:31:45.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed './223/28' 2024-05-27T09:31:45.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed './223/24' 2024-05-27T09:31:45.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed './223/29' 2024-05-27T09:31:45.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed './223/32' 2024-05-27T09:31:45.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed './223/22' 2024-05-27T09:31:45.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed './223/25' 2024-05-27T09:31:45.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed './223/30' 2024-05-27T09:31:45.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed './223/33' 2024-05-27T09:31:45.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed './223/23' 2024-05-27T09:31:45.561 INFO:tasks.workunit.client.0.smithi114.stdout:removed './223/35' 2024-05-27T09:31:45.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed './223/26' 2024-05-27T09:31:45.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed './223/37' 2024-05-27T09:31:45.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './223' 2024-05-27T09:31:45.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed './224/21' 2024-05-27T09:31:45.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed './224/27' 2024-05-27T09:31:45.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed './224/31' 2024-05-27T09:31:45.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed './224/36' 2024-05-27T09:31:45.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed './224/20' 2024-05-27T09:31:45.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed './224/39' 2024-05-27T09:31:45.562 INFO:tasks.workunit.client.0.smithi114.stdout:removed './224/34' 2024-05-27T09:31:45.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './224/38' 2024-05-27T09:31:45.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './224/28' 2024-05-27T09:31:45.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './224/24' 2024-05-27T09:31:45.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './224/29' 2024-05-27T09:31:45.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './224/32' 2024-05-27T09:31:45.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './224/22' 2024-05-27T09:31:45.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './224/25' 2024-05-27T09:31:45.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './224/30' 2024-05-27T09:31:45.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './224/33' 2024-05-27T09:31:45.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './224/23' 2024-05-27T09:31:45.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './224/35' 2024-05-27T09:31:45.563 INFO:tasks.workunit.client.0.smithi114.stdout:removed './224/26' 2024-05-27T09:31:45.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './224/37' 2024-05-27T09:31:45.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './224' 2024-05-27T09:31:45.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './225/21' 2024-05-27T09:31:45.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './225/27' 2024-05-27T09:31:45.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './225/31' 2024-05-27T09:31:45.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './225/36' 2024-05-27T09:31:45.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './225/20' 2024-05-27T09:31:45.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './225/39' 2024-05-27T09:31:45.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './225/34' 2024-05-27T09:31:45.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './225/38' 2024-05-27T09:31:45.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './225/28' 2024-05-27T09:31:45.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './225/24' 2024-05-27T09:31:45.564 INFO:tasks.workunit.client.0.smithi114.stdout:removed './225/29' 2024-05-27T09:31:45.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './225/32' 2024-05-27T09:31:45.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './225/22' 2024-05-27T09:31:45.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './225/25' 2024-05-27T09:31:45.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './225/30' 2024-05-27T09:31:45.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './225/33' 2024-05-27T09:31:45.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './225/23' 2024-05-27T09:31:45.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './225/35' 2024-05-27T09:31:45.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './225/26' 2024-05-27T09:31:45.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './225/37' 2024-05-27T09:31:45.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './225' 2024-05-27T09:31:45.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './226/21' 2024-05-27T09:31:45.565 INFO:tasks.workunit.client.0.smithi114.stdout:removed './226/27' 2024-05-27T09:31:45.566 INFO:tasks.workunit.client.0.smithi114.stdout:removed './226/31' 2024-05-27T09:31:45.566 INFO:tasks.workunit.client.0.smithi114.stdout:removed './226/36' 2024-05-27T09:31:45.566 INFO:tasks.workunit.client.0.smithi114.stdout:removed './226/20' 2024-05-27T09:31:45.566 INFO:tasks.workunit.client.0.smithi114.stdout:removed './226/39' 2024-05-27T09:31:45.566 INFO:tasks.workunit.client.0.smithi114.stdout:removed './226/34' 2024-05-27T09:31:45.566 INFO:tasks.workunit.client.0.smithi114.stdout:removed './226/38' 2024-05-27T09:31:45.566 INFO:tasks.workunit.client.0.smithi114.stdout:removed './226/28' 2024-05-27T09:31:45.566 INFO:tasks.workunit.client.0.smithi114.stdout:removed './226/24' 2024-05-27T09:31:46.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './226/29' 2024-05-27T09:31:46.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './226/32' 2024-05-27T09:31:46.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './226/22' 2024-05-27T09:31:46.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './226/25' 2024-05-27T09:31:46.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './226/30' 2024-05-27T09:31:46.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './226/33' 2024-05-27T09:31:46.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './226/23' 2024-05-27T09:31:46.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './226/35' 2024-05-27T09:31:46.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './226/26' 2024-05-27T09:31:46.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './226/37' 2024-05-27T09:31:46.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './226' 2024-05-27T09:31:46.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './227/21' 2024-05-27T09:31:46.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './227/27' 2024-05-27T09:31:46.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './227/31' 2024-05-27T09:31:46.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './227/36' 2024-05-27T09:31:46.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './227/20' 2024-05-27T09:31:46.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './227/39' 2024-05-27T09:31:46.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './227/34' 2024-05-27T09:31:46.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './227/38' 2024-05-27T09:31:46.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './227/28' 2024-05-27T09:31:46.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './227/24' 2024-05-27T09:31:46.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './227/29' 2024-05-27T09:31:46.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './227/32' 2024-05-27T09:31:46.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './227/22' 2024-05-27T09:31:46.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './227/25' 2024-05-27T09:31:46.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './227/30' 2024-05-27T09:31:46.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './227/33' 2024-05-27T09:31:46.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './227/23' 2024-05-27T09:31:46.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './227/35' 2024-05-27T09:31:46.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './227/26' 2024-05-27T09:31:46.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './227/37' 2024-05-27T09:31:46.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './227' 2024-05-27T09:31:46.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './228/21' 2024-05-27T09:31:46.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './228/27' 2024-05-27T09:31:46.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './228/31' 2024-05-27T09:31:46.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './228/36' 2024-05-27T09:31:46.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './228/20' 2024-05-27T09:31:46.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './228/39' 2024-05-27T09:31:46.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './228/34' 2024-05-27T09:31:46.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './228/38' 2024-05-27T09:31:46.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './228/28' 2024-05-27T09:31:46.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './228/24' 2024-05-27T09:31:46.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './228/29' 2024-05-27T09:31:46.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './228/32' 2024-05-27T09:31:46.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './228/22' 2024-05-27T09:31:46.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './228/25' 2024-05-27T09:31:46.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './228/30' 2024-05-27T09:31:46.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './228/33' 2024-05-27T09:31:46.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './228/23' 2024-05-27T09:31:46.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './228/35' 2024-05-27T09:31:46.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './228/26' 2024-05-27T09:31:46.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './228/37' 2024-05-27T09:31:46.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './228' 2024-05-27T09:31:46.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './229/21' 2024-05-27T09:31:46.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './229/27' 2024-05-27T09:31:46.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './229/31' 2024-05-27T09:31:46.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './229/36' 2024-05-27T09:31:46.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './229/20' 2024-05-27T09:31:46.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './229/39' 2024-05-27T09:31:46.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './229/34' 2024-05-27T09:31:46.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './229/38' 2024-05-27T09:31:46.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './229/28' 2024-05-27T09:31:46.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './229/24' 2024-05-27T09:31:46.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './229/29' 2024-05-27T09:31:46.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './229/32' 2024-05-27T09:31:46.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './229/22' 2024-05-27T09:31:46.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './229/25' 2024-05-27T09:31:46.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './229/30' 2024-05-27T09:31:46.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './229/33' 2024-05-27T09:31:46.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './229/23' 2024-05-27T09:31:46.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './229/35' 2024-05-27T09:31:46.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './229/26' 2024-05-27T09:31:46.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './229/37' 2024-05-27T09:31:46.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './229' 2024-05-27T09:31:46.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './230/21' 2024-05-27T09:31:46.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './230/27' 2024-05-27T09:31:46.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './230/31' 2024-05-27T09:31:46.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './230/36' 2024-05-27T09:31:46.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './230/20' 2024-05-27T09:31:46.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './230/39' 2024-05-27T09:31:46.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './230/34' 2024-05-27T09:31:46.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './230/38' 2024-05-27T09:31:46.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './230/28' 2024-05-27T09:31:46.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './230/24' 2024-05-27T09:31:46.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './230/29' 2024-05-27T09:31:46.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './230/32' 2024-05-27T09:31:46.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './230/22' 2024-05-27T09:31:46.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './230/25' 2024-05-27T09:31:46.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './230/30' 2024-05-27T09:31:46.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './230/33' 2024-05-27T09:31:46.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './230/23' 2024-05-27T09:31:46.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './230/35' 2024-05-27T09:31:46.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './230/26' 2024-05-27T09:31:46.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './230/37' 2024-05-27T09:31:46.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './230' 2024-05-27T09:31:46.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './231/21' 2024-05-27T09:31:46.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './231/27' 2024-05-27T09:31:46.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './231/31' 2024-05-27T09:31:46.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './231/36' 2024-05-27T09:31:46.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './231/20' 2024-05-27T09:31:46.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './231/39' 2024-05-27T09:31:46.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './231/34' 2024-05-27T09:31:46.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './231/38' 2024-05-27T09:31:46.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './231/28' 2024-05-27T09:31:46.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './231/24' 2024-05-27T09:31:46.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './231/29' 2024-05-27T09:31:46.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './231/32' 2024-05-27T09:31:46.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './231/22' 2024-05-27T09:31:46.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './231/25' 2024-05-27T09:31:46.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './231/30' 2024-05-27T09:31:46.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './231/33' 2024-05-27T09:31:46.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './231/23' 2024-05-27T09:31:46.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './231/35' 2024-05-27T09:31:46.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './231/26' 2024-05-27T09:31:46.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './231/37' 2024-05-27T09:31:46.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './231' 2024-05-27T09:31:46.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './232/21' 2024-05-27T09:31:46.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './232/27' 2024-05-27T09:31:46.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './232/31' 2024-05-27T09:31:46.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './232/36' 2024-05-27T09:31:46.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './232/20' 2024-05-27T09:31:46.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './232/39' 2024-05-27T09:31:46.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './232/34' 2024-05-27T09:31:46.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './232/38' 2024-05-27T09:31:46.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './232/28' 2024-05-27T09:31:46.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './232/24' 2024-05-27T09:31:46.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './232/29' 2024-05-27T09:31:46.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './232/32' 2024-05-27T09:31:46.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './232/22' 2024-05-27T09:31:46.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './232/25' 2024-05-27T09:31:46.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './232/30' 2024-05-27T09:31:46.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './232/33' 2024-05-27T09:31:46.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './232/23' 2024-05-27T09:31:46.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './232/35' 2024-05-27T09:31:46.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './232/26' 2024-05-27T09:31:46.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './232/37' 2024-05-27T09:31:46.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './232' 2024-05-27T09:31:46.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './233/21' 2024-05-27T09:31:46.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './233/27' 2024-05-27T09:31:46.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './233/31' 2024-05-27T09:31:46.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './233/36' 2024-05-27T09:31:46.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './233/20' 2024-05-27T09:31:46.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './233/39' 2024-05-27T09:31:46.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './233/34' 2024-05-27T09:31:46.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './233/38' 2024-05-27T09:31:46.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './233/28' 2024-05-27T09:31:46.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './233/24' 2024-05-27T09:31:46.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './233/29' 2024-05-27T09:31:46.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './233/32' 2024-05-27T09:31:46.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './233/22' 2024-05-27T09:31:46.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './233/25' 2024-05-27T09:31:46.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './233/30' 2024-05-27T09:31:46.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './233/33' 2024-05-27T09:31:46.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './233/23' 2024-05-27T09:31:46.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './233/35' 2024-05-27T09:31:46.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './233/26' 2024-05-27T09:31:46.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './233/37' 2024-05-27T09:31:46.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './233' 2024-05-27T09:31:46.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './234/21' 2024-05-27T09:31:46.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './234/27' 2024-05-27T09:31:46.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './234/31' 2024-05-27T09:31:46.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './234/36' 2024-05-27T09:31:46.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './234/20' 2024-05-27T09:31:46.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './234/39' 2024-05-27T09:31:46.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './234/34' 2024-05-27T09:31:46.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './234/38' 2024-05-27T09:31:46.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './234/28' 2024-05-27T09:31:46.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './234/24' 2024-05-27T09:31:46.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './234/29' 2024-05-27T09:31:46.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './234/32' 2024-05-27T09:31:46.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './234/22' 2024-05-27T09:31:46.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './234/25' 2024-05-27T09:31:46.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './234/30' 2024-05-27T09:31:46.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './234/33' 2024-05-27T09:31:46.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './234/23' 2024-05-27T09:31:46.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './234/35' 2024-05-27T09:31:46.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './234/26' 2024-05-27T09:31:46.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './234/37' 2024-05-27T09:31:46.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './234' 2024-05-27T09:31:46.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './235/21' 2024-05-27T09:31:46.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './235/27' 2024-05-27T09:31:46.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './235/31' 2024-05-27T09:31:46.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './235/36' 2024-05-27T09:31:46.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './235/20' 2024-05-27T09:31:46.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './235/39' 2024-05-27T09:31:46.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './235/34' 2024-05-27T09:31:46.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './235/38' 2024-05-27T09:31:46.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './235/28' 2024-05-27T09:31:46.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './235/24' 2024-05-27T09:31:46.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './235/29' 2024-05-27T09:31:46.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './235/32' 2024-05-27T09:31:46.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './235/22' 2024-05-27T09:31:46.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './235/25' 2024-05-27T09:31:46.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './235/30' 2024-05-27T09:31:46.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './235/33' 2024-05-27T09:31:46.609 INFO:tasks.workunit.client.0.smithi114.stdout:removed './235/23' 2024-05-27T09:31:46.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './235/35' 2024-05-27T09:31:46.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './235/26' 2024-05-27T09:31:46.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './235/37' 2024-05-27T09:31:46.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './235' 2024-05-27T09:31:46.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './236/21' 2024-05-27T09:31:46.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './236/27' 2024-05-27T09:31:46.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './236/31' 2024-05-27T09:31:46.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './236/36' 2024-05-27T09:31:46.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './236/20' 2024-05-27T09:31:46.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './236/39' 2024-05-27T09:31:46.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './236/34' 2024-05-27T09:31:46.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './236/38' 2024-05-27T09:31:46.610 INFO:tasks.workunit.client.0.smithi114.stdout:removed './236/28' 2024-05-27T09:31:46.611 INFO:tasks.workunit.client.0.smithi114.stdout:removed './236/24' 2024-05-27T09:31:46.611 INFO:tasks.workunit.client.0.smithi114.stdout:removed './236/29' 2024-05-27T09:31:46.611 INFO:tasks.workunit.client.0.smithi114.stdout:removed './236/32' 2024-05-27T09:31:47.582 INFO:tasks.workunit.client.0.smithi114.stdout:removed './236/22' 2024-05-27T09:31:47.582 INFO:tasks.workunit.client.0.smithi114.stdout:removed './236/25' 2024-05-27T09:31:47.583 INFO:tasks.workunit.client.0.smithi114.stdout:removed './236/30' 2024-05-27T09:31:47.583 INFO:tasks.workunit.client.0.smithi114.stdout:removed './236/33' 2024-05-27T09:31:47.583 INFO:tasks.workunit.client.0.smithi114.stdout:removed './236/23' 2024-05-27T09:31:47.583 INFO:tasks.workunit.client.0.smithi114.stdout:removed './236/35' 2024-05-27T09:31:47.583 INFO:tasks.workunit.client.0.smithi114.stdout:removed './236/26' 2024-05-27T09:31:47.583 INFO:tasks.workunit.client.0.smithi114.stdout:removed './236/37' 2024-05-27T09:31:47.583 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './236' 2024-05-27T09:31:47.583 INFO:tasks.workunit.client.0.smithi114.stdout:removed './237/21' 2024-05-27T09:31:47.583 INFO:tasks.workunit.client.0.smithi114.stdout:removed './237/27' 2024-05-27T09:31:47.583 INFO:tasks.workunit.client.0.smithi114.stdout:removed './237/31' 2024-05-27T09:31:47.583 INFO:tasks.workunit.client.0.smithi114.stdout:removed './237/36' 2024-05-27T09:31:47.583 INFO:tasks.workunit.client.0.smithi114.stdout:removed './237/20' 2024-05-27T09:31:47.584 INFO:tasks.workunit.client.0.smithi114.stdout:removed './237/39' 2024-05-27T09:31:47.584 INFO:tasks.workunit.client.0.smithi114.stdout:removed './237/34' 2024-05-27T09:31:47.584 INFO:tasks.workunit.client.0.smithi114.stdout:removed './237/38' 2024-05-27T09:31:47.584 INFO:tasks.workunit.client.0.smithi114.stdout:removed './237/28' 2024-05-27T09:31:47.584 INFO:tasks.workunit.client.0.smithi114.stdout:removed './237/24' 2024-05-27T09:31:47.584 INFO:tasks.workunit.client.0.smithi114.stdout:removed './237/29' 2024-05-27T09:31:47.584 INFO:tasks.workunit.client.0.smithi114.stdout:removed './237/32' 2024-05-27T09:31:47.584 INFO:tasks.workunit.client.0.smithi114.stdout:removed './237/22' 2024-05-27T09:31:47.584 INFO:tasks.workunit.client.0.smithi114.stdout:removed './237/25' 2024-05-27T09:31:47.584 INFO:tasks.workunit.client.0.smithi114.stdout:removed './237/30' 2024-05-27T09:31:47.584 INFO:tasks.workunit.client.0.smithi114.stdout:removed './237/33' 2024-05-27T09:31:47.584 INFO:tasks.workunit.client.0.smithi114.stdout:removed './237/23' 2024-05-27T09:31:47.584 INFO:tasks.workunit.client.0.smithi114.stdout:removed './237/35' 2024-05-27T09:31:47.584 INFO:tasks.workunit.client.0.smithi114.stdout:removed './237/26' 2024-05-27T09:31:47.585 INFO:tasks.workunit.client.0.smithi114.stdout:removed './237/37' 2024-05-27T09:31:47.585 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './237' 2024-05-27T09:31:47.585 INFO:tasks.workunit.client.0.smithi114.stdout:removed './238/21' 2024-05-27T09:31:47.585 INFO:tasks.workunit.client.0.smithi114.stdout:removed './238/27' 2024-05-27T09:31:47.585 INFO:tasks.workunit.client.0.smithi114.stdout:removed './238/31' 2024-05-27T09:31:47.585 INFO:tasks.workunit.client.0.smithi114.stdout:removed './238/36' 2024-05-27T09:31:47.585 INFO:tasks.workunit.client.0.smithi114.stdout:removed './238/20' 2024-05-27T09:31:47.585 INFO:tasks.workunit.client.0.smithi114.stdout:removed './238/39' 2024-05-27T09:31:47.585 INFO:tasks.workunit.client.0.smithi114.stdout:removed './238/34' 2024-05-27T09:31:47.585 INFO:tasks.workunit.client.0.smithi114.stdout:removed './238/38' 2024-05-27T09:31:47.585 INFO:tasks.workunit.client.0.smithi114.stdout:removed './238/28' 2024-05-27T09:31:47.585 INFO:tasks.workunit.client.0.smithi114.stdout:removed './238/24' 2024-05-27T09:31:47.585 INFO:tasks.workunit.client.0.smithi114.stdout:removed './238/29' 2024-05-27T09:31:47.586 INFO:tasks.workunit.client.0.smithi114.stdout:removed './238/32' 2024-05-27T09:31:47.586 INFO:tasks.workunit.client.0.smithi114.stdout:removed './238/22' 2024-05-27T09:31:47.586 INFO:tasks.workunit.client.0.smithi114.stdout:removed './238/25' 2024-05-27T09:31:47.586 INFO:tasks.workunit.client.0.smithi114.stdout:removed './238/30' 2024-05-27T09:31:47.586 INFO:tasks.workunit.client.0.smithi114.stdout:removed './238/33' 2024-05-27T09:31:47.586 INFO:tasks.workunit.client.0.smithi114.stdout:removed './238/23' 2024-05-27T09:31:47.586 INFO:tasks.workunit.client.0.smithi114.stdout:removed './238/35' 2024-05-27T09:31:47.586 INFO:tasks.workunit.client.0.smithi114.stdout:removed './238/26' 2024-05-27T09:31:47.586 INFO:tasks.workunit.client.0.smithi114.stdout:removed './238/37' 2024-05-27T09:31:47.586 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './238' 2024-05-27T09:31:47.586 INFO:tasks.workunit.client.0.smithi114.stdout:removed './239/21' 2024-05-27T09:31:47.586 INFO:tasks.workunit.client.0.smithi114.stdout:removed './239/27' 2024-05-27T09:31:47.586 INFO:tasks.workunit.client.0.smithi114.stdout:removed './239/31' 2024-05-27T09:31:47.586 INFO:tasks.workunit.client.0.smithi114.stdout:removed './239/36' 2024-05-27T09:31:47.587 INFO:tasks.workunit.client.0.smithi114.stdout:removed './239/20' 2024-05-27T09:31:47.587 INFO:tasks.workunit.client.0.smithi114.stdout:removed './239/39' 2024-05-27T09:31:47.587 INFO:tasks.workunit.client.0.smithi114.stdout:removed './239/34' 2024-05-27T09:31:47.587 INFO:tasks.workunit.client.0.smithi114.stdout:removed './239/38' 2024-05-27T09:31:47.587 INFO:tasks.workunit.client.0.smithi114.stdout:removed './239/28' 2024-05-27T09:31:47.587 INFO:tasks.workunit.client.0.smithi114.stdout:removed './239/24' 2024-05-27T09:31:47.587 INFO:tasks.workunit.client.0.smithi114.stdout:removed './239/29' 2024-05-27T09:31:47.587 INFO:tasks.workunit.client.0.smithi114.stdout:removed './239/32' 2024-05-27T09:31:47.587 INFO:tasks.workunit.client.0.smithi114.stdout:removed './239/22' 2024-05-27T09:31:47.587 INFO:tasks.workunit.client.0.smithi114.stdout:removed './239/25' 2024-05-27T09:31:47.587 INFO:tasks.workunit.client.0.smithi114.stdout:removed './239/30' 2024-05-27T09:31:47.587 INFO:tasks.workunit.client.0.smithi114.stdout:removed './239/33' 2024-05-27T09:31:47.587 INFO:tasks.workunit.client.0.smithi114.stdout:removed './239/23' 2024-05-27T09:31:47.588 INFO:tasks.workunit.client.0.smithi114.stdout:removed './239/35' 2024-05-27T09:31:47.588 INFO:tasks.workunit.client.0.smithi114.stdout:removed './239/26' 2024-05-27T09:31:47.588 INFO:tasks.workunit.client.0.smithi114.stdout:removed './239/37' 2024-05-27T09:31:47.588 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './239' 2024-05-27T09:31:47.588 INFO:tasks.workunit.client.0.smithi114.stdout:removed './240/21' 2024-05-27T09:31:47.588 INFO:tasks.workunit.client.0.smithi114.stdout:removed './240/27' 2024-05-27T09:31:47.588 INFO:tasks.workunit.client.0.smithi114.stdout:removed './240/31' 2024-05-27T09:31:47.588 INFO:tasks.workunit.client.0.smithi114.stdout:removed './240/36' 2024-05-27T09:31:47.588 INFO:tasks.workunit.client.0.smithi114.stdout:removed './240/20' 2024-05-27T09:31:47.588 INFO:tasks.workunit.client.0.smithi114.stdout:removed './240/39' 2024-05-27T09:31:47.588 INFO:tasks.workunit.client.0.smithi114.stdout:removed './240/34' 2024-05-27T09:31:47.588 INFO:tasks.workunit.client.0.smithi114.stdout:removed './240/38' 2024-05-27T09:31:47.589 INFO:tasks.workunit.client.0.smithi114.stdout:removed './240/28' 2024-05-27T09:31:47.589 INFO:tasks.workunit.client.0.smithi114.stdout:removed './240/24' 2024-05-27T09:31:47.589 INFO:tasks.workunit.client.0.smithi114.stdout:removed './240/29' 2024-05-27T09:31:47.589 INFO:tasks.workunit.client.0.smithi114.stdout:removed './240/32' 2024-05-27T09:31:47.589 INFO:tasks.workunit.client.0.smithi114.stdout:removed './240/22' 2024-05-27T09:31:47.589 INFO:tasks.workunit.client.0.smithi114.stdout:removed './240/25' 2024-05-27T09:31:47.589 INFO:tasks.workunit.client.0.smithi114.stdout:removed './240/30' 2024-05-27T09:31:47.589 INFO:tasks.workunit.client.0.smithi114.stdout:removed './240/33' 2024-05-27T09:31:47.589 INFO:tasks.workunit.client.0.smithi114.stdout:removed './240/23' 2024-05-27T09:31:47.589 INFO:tasks.workunit.client.0.smithi114.stdout:removed './240/35' 2024-05-27T09:31:47.589 INFO:tasks.workunit.client.0.smithi114.stdout:removed './240/26' 2024-05-27T09:31:47.589 INFO:tasks.workunit.client.0.smithi114.stdout:removed './240/37' 2024-05-27T09:31:47.589 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './240' 2024-05-27T09:31:47.590 INFO:tasks.workunit.client.0.smithi114.stdout:removed './241/21' 2024-05-27T09:31:47.590 INFO:tasks.workunit.client.0.smithi114.stdout:removed './241/27' 2024-05-27T09:31:47.590 INFO:tasks.workunit.client.0.smithi114.stdout:removed './241/31' 2024-05-27T09:31:47.590 INFO:tasks.workunit.client.0.smithi114.stdout:removed './241/36' 2024-05-27T09:31:47.590 INFO:tasks.workunit.client.0.smithi114.stdout:removed './241/20' 2024-05-27T09:31:47.590 INFO:tasks.workunit.client.0.smithi114.stdout:removed './241/39' 2024-05-27T09:31:47.590 INFO:tasks.workunit.client.0.smithi114.stdout:removed './241/34' 2024-05-27T09:31:47.590 INFO:tasks.workunit.client.0.smithi114.stdout:removed './241/38' 2024-05-27T09:31:47.590 INFO:tasks.workunit.client.0.smithi114.stdout:removed './241/28' 2024-05-27T09:31:47.590 INFO:tasks.workunit.client.0.smithi114.stdout:removed './241/24' 2024-05-27T09:31:47.590 INFO:tasks.workunit.client.0.smithi114.stdout:removed './241/29' 2024-05-27T09:31:47.590 INFO:tasks.workunit.client.0.smithi114.stdout:removed './241/32' 2024-05-27T09:31:47.590 INFO:tasks.workunit.client.0.smithi114.stdout:removed './241/22' 2024-05-27T09:31:47.591 INFO:tasks.workunit.client.0.smithi114.stdout:removed './241/25' 2024-05-27T09:31:47.591 INFO:tasks.workunit.client.0.smithi114.stdout:removed './241/30' 2024-05-27T09:31:47.591 INFO:tasks.workunit.client.0.smithi114.stdout:removed './241/33' 2024-05-27T09:31:47.591 INFO:tasks.workunit.client.0.smithi114.stdout:removed './241/23' 2024-05-27T09:31:47.591 INFO:tasks.workunit.client.0.smithi114.stdout:removed './241/35' 2024-05-27T09:31:47.591 INFO:tasks.workunit.client.0.smithi114.stdout:removed './241/26' 2024-05-27T09:31:47.591 INFO:tasks.workunit.client.0.smithi114.stdout:removed './241/37' 2024-05-27T09:31:47.591 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './241' 2024-05-27T09:31:47.591 INFO:tasks.workunit.client.0.smithi114.stdout:removed './242/21' 2024-05-27T09:31:47.591 INFO:tasks.workunit.client.0.smithi114.stdout:removed './242/27' 2024-05-27T09:31:47.591 INFO:tasks.workunit.client.0.smithi114.stdout:removed './242/31' 2024-05-27T09:31:47.591 INFO:tasks.workunit.client.0.smithi114.stdout:removed './242/36' 2024-05-27T09:31:47.591 INFO:tasks.workunit.client.0.smithi114.stdout:removed './242/20' 2024-05-27T09:31:47.591 INFO:tasks.workunit.client.0.smithi114.stdout:removed './242/39' 2024-05-27T09:31:47.592 INFO:tasks.workunit.client.0.smithi114.stdout:removed './242/34' 2024-05-27T09:31:47.592 INFO:tasks.workunit.client.0.smithi114.stdout:removed './242/38' 2024-05-27T09:31:47.592 INFO:tasks.workunit.client.0.smithi114.stdout:removed './242/28' 2024-05-27T09:31:47.592 INFO:tasks.workunit.client.0.smithi114.stdout:removed './242/24' 2024-05-27T09:31:47.592 INFO:tasks.workunit.client.0.smithi114.stdout:removed './242/29' 2024-05-27T09:31:47.592 INFO:tasks.workunit.client.0.smithi114.stdout:removed './242/32' 2024-05-27T09:31:47.592 INFO:tasks.workunit.client.0.smithi114.stdout:removed './242/22' 2024-05-27T09:31:47.592 INFO:tasks.workunit.client.0.smithi114.stdout:removed './242/25' 2024-05-27T09:31:47.592 INFO:tasks.workunit.client.0.smithi114.stdout:removed './242/30' 2024-05-27T09:31:47.592 INFO:tasks.workunit.client.0.smithi114.stdout:removed './242/33' 2024-05-27T09:31:47.592 INFO:tasks.workunit.client.0.smithi114.stdout:removed './242/23' 2024-05-27T09:31:47.592 INFO:tasks.workunit.client.0.smithi114.stdout:removed './242/35' 2024-05-27T09:31:47.592 INFO:tasks.workunit.client.0.smithi114.stdout:removed './242/26' 2024-05-27T09:31:47.592 INFO:tasks.workunit.client.0.smithi114.stdout:removed './242/37' 2024-05-27T09:31:47.593 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './242' 2024-05-27T09:31:47.593 INFO:tasks.workunit.client.0.smithi114.stdout:removed './243/21' 2024-05-27T09:31:47.593 INFO:tasks.workunit.client.0.smithi114.stdout:removed './243/27' 2024-05-27T09:31:47.593 INFO:tasks.workunit.client.0.smithi114.stdout:removed './243/31' 2024-05-27T09:31:47.593 INFO:tasks.workunit.client.0.smithi114.stdout:removed './243/36' 2024-05-27T09:31:47.593 INFO:tasks.workunit.client.0.smithi114.stdout:removed './243/20' 2024-05-27T09:31:47.593 INFO:tasks.workunit.client.0.smithi114.stdout:removed './243/39' 2024-05-27T09:31:47.593 INFO:tasks.workunit.client.0.smithi114.stdout:removed './243/34' 2024-05-27T09:31:47.593 INFO:tasks.workunit.client.0.smithi114.stdout:removed './243/38' 2024-05-27T09:31:47.593 INFO:tasks.workunit.client.0.smithi114.stdout:removed './243/28' 2024-05-27T09:31:47.593 INFO:tasks.workunit.client.0.smithi114.stdout:removed './243/24' 2024-05-27T09:31:47.593 INFO:tasks.workunit.client.0.smithi114.stdout:removed './243/29' 2024-05-27T09:31:47.593 INFO:tasks.workunit.client.0.smithi114.stdout:removed './243/32' 2024-05-27T09:31:47.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed './243/22' 2024-05-27T09:31:47.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed './243/25' 2024-05-27T09:31:47.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed './243/30' 2024-05-27T09:31:47.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed './243/33' 2024-05-27T09:31:47.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed './243/23' 2024-05-27T09:31:47.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed './243/35' 2024-05-27T09:31:47.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed './243/26' 2024-05-27T09:31:47.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed './243/37' 2024-05-27T09:31:47.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './243' 2024-05-27T09:31:47.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed './244/21' 2024-05-27T09:31:47.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed './244/27' 2024-05-27T09:31:47.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed './244/31' 2024-05-27T09:31:47.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed './244/36' 2024-05-27T09:31:47.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './244/20' 2024-05-27T09:31:47.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './244/39' 2024-05-27T09:31:47.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './244/34' 2024-05-27T09:31:47.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './244/38' 2024-05-27T09:31:47.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './244/28' 2024-05-27T09:31:47.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './244/24' 2024-05-27T09:31:47.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './244/29' 2024-05-27T09:31:47.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './244/32' 2024-05-27T09:31:47.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './244/22' 2024-05-27T09:31:47.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './244/25' 2024-05-27T09:31:47.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './244/30' 2024-05-27T09:31:47.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './244/33' 2024-05-27T09:31:47.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './244/23' 2024-05-27T09:31:47.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './244/35' 2024-05-27T09:31:47.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './244/26' 2024-05-27T09:31:47.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './244/37' 2024-05-27T09:31:47.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './244' 2024-05-27T09:31:47.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './245/21' 2024-05-27T09:31:47.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './245/27' 2024-05-27T09:31:47.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './245/31' 2024-05-27T09:31:47.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './245/36' 2024-05-27T09:31:47.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './245/20' 2024-05-27T09:31:47.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './245/39' 2024-05-27T09:31:47.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './245/34' 2024-05-27T09:31:47.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './245/38' 2024-05-27T09:31:47.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './245/28' 2024-05-27T09:31:47.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './245/24' 2024-05-27T09:31:47.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './245/29' 2024-05-27T09:31:47.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './245/32' 2024-05-27T09:31:47.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './245/22' 2024-05-27T09:31:47.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './245/25' 2024-05-27T09:31:47.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './245/30' 2024-05-27T09:31:47.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './245/33' 2024-05-27T09:31:47.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './245/23' 2024-05-27T09:31:47.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './245/35' 2024-05-27T09:31:47.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './245/26' 2024-05-27T09:31:47.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './245/37' 2024-05-27T09:31:47.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './245' 2024-05-27T09:31:47.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './246/21' 2024-05-27T09:31:47.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './246/27' 2024-05-27T09:31:47.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './246/31' 2024-05-27T09:31:47.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './246/36' 2024-05-27T09:31:47.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './246/20' 2024-05-27T09:31:47.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './246/39' 2024-05-27T09:31:47.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './246/34' 2024-05-27T09:31:47.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './246/38' 2024-05-27T09:31:47.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './246/28' 2024-05-27T09:31:47.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './246/24' 2024-05-27T09:31:47.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './246/29' 2024-05-27T09:31:47.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './246/32' 2024-05-27T09:31:47.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './246/22' 2024-05-27T09:31:48.695 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:31:48.701 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:31:48.723 INFO:tasks.workunit.client.0.smithi114.stdout:removed './246/25' 2024-05-27T09:31:48.723 INFO:tasks.workunit.client.0.smithi114.stdout:removed './246/30' 2024-05-27T09:31:48.723 INFO:tasks.workunit.client.0.smithi114.stdout:removed './246/33' 2024-05-27T09:31:48.723 INFO:tasks.workunit.client.0.smithi114.stdout:removed './246/23' 2024-05-27T09:31:48.723 INFO:tasks.workunit.client.0.smithi114.stdout:removed './246/35' 2024-05-27T09:31:48.724 INFO:tasks.workunit.client.0.smithi114.stdout:removed './246/26' 2024-05-27T09:31:48.724 INFO:tasks.workunit.client.0.smithi114.stdout:removed './246/37' 2024-05-27T09:31:48.724 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './246' 2024-05-27T09:31:48.724 INFO:tasks.workunit.client.0.smithi114.stdout:removed './247/21' 2024-05-27T09:31:48.724 INFO:tasks.workunit.client.0.smithi114.stdout:removed './247/27' 2024-05-27T09:31:48.724 INFO:tasks.workunit.client.0.smithi114.stdout:removed './247/31' 2024-05-27T09:31:48.724 INFO:tasks.workunit.client.0.smithi114.stdout:removed './247/36' 2024-05-27T09:31:48.724 INFO:tasks.workunit.client.0.smithi114.stdout:removed './247/20' 2024-05-27T09:31:48.724 INFO:tasks.workunit.client.0.smithi114.stdout:removed './247/39' 2024-05-27T09:31:48.724 INFO:tasks.workunit.client.0.smithi114.stdout:removed './247/34' 2024-05-27T09:31:48.724 INFO:tasks.workunit.client.0.smithi114.stdout:removed './247/38' 2024-05-27T09:31:48.724 INFO:tasks.workunit.client.0.smithi114.stdout:removed './247/28' 2024-05-27T09:31:48.724 INFO:tasks.workunit.client.0.smithi114.stdout:removed './247/24' 2024-05-27T09:31:48.724 INFO:tasks.workunit.client.0.smithi114.stdout:removed './247/29' 2024-05-27T09:31:48.725 INFO:tasks.workunit.client.0.smithi114.stdout:removed './247/32' 2024-05-27T09:31:48.725 INFO:tasks.workunit.client.0.smithi114.stdout:removed './247/22' 2024-05-27T09:31:48.725 INFO:tasks.workunit.client.0.smithi114.stdout:removed './247/25' 2024-05-27T09:31:48.725 INFO:tasks.workunit.client.0.smithi114.stdout:removed './247/30' 2024-05-27T09:31:48.725 INFO:tasks.workunit.client.0.smithi114.stdout:removed './247/33' 2024-05-27T09:31:48.725 INFO:tasks.workunit.client.0.smithi114.stdout:removed './247/23' 2024-05-27T09:31:48.725 INFO:tasks.workunit.client.0.smithi114.stdout:removed './247/35' 2024-05-27T09:31:48.725 INFO:tasks.workunit.client.0.smithi114.stdout:removed './247/26' 2024-05-27T09:31:48.725 INFO:tasks.workunit.client.0.smithi114.stdout:removed './247/37' 2024-05-27T09:31:48.725 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './247' 2024-05-27T09:31:48.725 INFO:tasks.workunit.client.0.smithi114.stdout:removed './248/21' 2024-05-27T09:31:48.726 INFO:tasks.workunit.client.0.smithi114.stdout:removed './248/27' 2024-05-27T09:31:48.726 INFO:tasks.workunit.client.0.smithi114.stdout:removed './248/31' 2024-05-27T09:31:48.726 INFO:tasks.workunit.client.0.smithi114.stdout:removed './248/36' 2024-05-27T09:31:48.726 INFO:tasks.workunit.client.0.smithi114.stdout:removed './248/20' 2024-05-27T09:31:48.726 INFO:tasks.workunit.client.0.smithi114.stdout:removed './248/39' 2024-05-27T09:31:48.726 INFO:tasks.workunit.client.0.smithi114.stdout:removed './248/34' 2024-05-27T09:31:48.726 INFO:tasks.workunit.client.0.smithi114.stdout:removed './248/38' 2024-05-27T09:31:48.726 INFO:tasks.workunit.client.0.smithi114.stdout:removed './248/28' 2024-05-27T09:31:48.726 INFO:tasks.workunit.client.0.smithi114.stdout:removed './248/24' 2024-05-27T09:31:48.726 INFO:tasks.workunit.client.0.smithi114.stdout:removed './248/29' 2024-05-27T09:31:48.726 INFO:tasks.workunit.client.0.smithi114.stdout:removed './248/32' 2024-05-27T09:31:48.726 INFO:tasks.workunit.client.0.smithi114.stdout:removed './248/22' 2024-05-27T09:31:48.727 INFO:tasks.workunit.client.0.smithi114.stdout:removed './248/25' 2024-05-27T09:31:48.727 INFO:tasks.workunit.client.0.smithi114.stdout:removed './248/30' 2024-05-27T09:31:48.727 INFO:tasks.workunit.client.0.smithi114.stdout:removed './248/33' 2024-05-27T09:31:48.727 INFO:tasks.workunit.client.0.smithi114.stdout:removed './248/23' 2024-05-27T09:31:48.727 INFO:tasks.workunit.client.0.smithi114.stdout:removed './248/35' 2024-05-27T09:31:48.727 INFO:tasks.workunit.client.0.smithi114.stdout:removed './248/26' 2024-05-27T09:31:48.727 INFO:tasks.workunit.client.0.smithi114.stdout:removed './248/37' 2024-05-27T09:31:48.727 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './248' 2024-05-27T09:31:48.727 INFO:tasks.workunit.client.0.smithi114.stdout:removed './249/21' 2024-05-27T09:31:48.727 INFO:tasks.workunit.client.0.smithi114.stdout:removed './249/27' 2024-05-27T09:31:48.727 INFO:tasks.workunit.client.0.smithi114.stdout:removed './249/31' 2024-05-27T09:31:48.727 INFO:tasks.workunit.client.0.smithi114.stdout:removed './249/36' 2024-05-27T09:31:48.727 INFO:tasks.workunit.client.0.smithi114.stdout:removed './249/20' 2024-05-27T09:31:48.727 INFO:tasks.workunit.client.0.smithi114.stdout:removed './249/39' 2024-05-27T09:31:48.727 INFO:tasks.workunit.client.0.smithi114.stdout:removed './249/34' 2024-05-27T09:31:48.728 INFO:tasks.workunit.client.0.smithi114.stdout:removed './249/38' 2024-05-27T09:31:48.728 INFO:tasks.workunit.client.0.smithi114.stdout:removed './249/28' 2024-05-27T09:31:48.728 INFO:tasks.workunit.client.0.smithi114.stdout:removed './249/24' 2024-05-27T09:31:48.728 INFO:tasks.workunit.client.0.smithi114.stdout:removed './249/29' 2024-05-27T09:31:48.728 INFO:tasks.workunit.client.0.smithi114.stdout:removed './249/32' 2024-05-27T09:31:48.728 INFO:tasks.workunit.client.0.smithi114.stdout:removed './249/22' 2024-05-27T09:31:48.728 INFO:tasks.workunit.client.0.smithi114.stdout:removed './249/25' 2024-05-27T09:31:48.728 INFO:tasks.workunit.client.0.smithi114.stdout:removed './249/30' 2024-05-27T09:31:48.728 INFO:tasks.workunit.client.0.smithi114.stdout:removed './249/33' 2024-05-27T09:31:48.728 INFO:tasks.workunit.client.0.smithi114.stdout:removed './249/23' 2024-05-27T09:31:48.728 INFO:tasks.workunit.client.0.smithi114.stdout:removed './249/35' 2024-05-27T09:31:48.728 INFO:tasks.workunit.client.0.smithi114.stdout:removed './249/26' 2024-05-27T09:31:48.728 INFO:tasks.workunit.client.0.smithi114.stdout:removed './249/37' 2024-05-27T09:31:48.728 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './249' 2024-05-27T09:31:48.729 INFO:tasks.workunit.client.0.smithi114.stdout:removed './250/21' 2024-05-27T09:31:48.729 INFO:tasks.workunit.client.0.smithi114.stdout:removed './250/27' 2024-05-27T09:31:48.729 INFO:tasks.workunit.client.0.smithi114.stdout:removed './250/31' 2024-05-27T09:31:48.729 INFO:tasks.workunit.client.0.smithi114.stdout:removed './250/36' 2024-05-27T09:31:48.729 INFO:tasks.workunit.client.0.smithi114.stdout:removed './250/20' 2024-05-27T09:31:48.729 INFO:tasks.workunit.client.0.smithi114.stdout:removed './250/39' 2024-05-27T09:31:48.729 INFO:tasks.workunit.client.0.smithi114.stdout:removed './250/34' 2024-05-27T09:31:48.729 INFO:tasks.workunit.client.0.smithi114.stdout:removed './250/38' 2024-05-27T09:31:48.729 INFO:tasks.workunit.client.0.smithi114.stdout:removed './250/28' 2024-05-27T09:31:48.729 INFO:tasks.workunit.client.0.smithi114.stdout:removed './250/24' 2024-05-27T09:31:48.729 INFO:tasks.workunit.client.0.smithi114.stdout:removed './250/29' 2024-05-27T09:31:48.729 INFO:tasks.workunit.client.0.smithi114.stdout:removed './250/32' 2024-05-27T09:31:48.729 INFO:tasks.workunit.client.0.smithi114.stdout:removed './250/22' 2024-05-27T09:31:48.729 INFO:tasks.workunit.client.0.smithi114.stdout:removed './250/25' 2024-05-27T09:31:48.730 INFO:tasks.workunit.client.0.smithi114.stdout:removed './250/30' 2024-05-27T09:31:48.730 INFO:tasks.workunit.client.0.smithi114.stdout:removed './250/33' 2024-05-27T09:31:48.730 INFO:tasks.workunit.client.0.smithi114.stdout:removed './250/23' 2024-05-27T09:31:48.730 INFO:tasks.workunit.client.0.smithi114.stdout:removed './250/35' 2024-05-27T09:31:48.730 INFO:tasks.workunit.client.0.smithi114.stdout:removed './250/26' 2024-05-27T09:31:48.730 INFO:tasks.workunit.client.0.smithi114.stdout:removed './250/37' 2024-05-27T09:31:48.730 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './250' 2024-05-27T09:31:48.730 INFO:tasks.workunit.client.0.smithi114.stdout:removed './251/21' 2024-05-27T09:31:48.730 INFO:tasks.workunit.client.0.smithi114.stdout:removed './251/27' 2024-05-27T09:31:48.730 INFO:tasks.workunit.client.0.smithi114.stdout:removed './251/31' 2024-05-27T09:31:48.730 INFO:tasks.workunit.client.0.smithi114.stdout:removed './251/36' 2024-05-27T09:31:48.730 INFO:tasks.workunit.client.0.smithi114.stdout:removed './251/20' 2024-05-27T09:31:48.730 INFO:tasks.workunit.client.0.smithi114.stdout:removed './251/39' 2024-05-27T09:31:48.730 INFO:tasks.workunit.client.0.smithi114.stdout:removed './251/34' 2024-05-27T09:31:48.731 INFO:tasks.workunit.client.0.smithi114.stdout:removed './251/38' 2024-05-27T09:31:48.731 INFO:tasks.workunit.client.0.smithi114.stdout:removed './251/28' 2024-05-27T09:31:48.731 INFO:tasks.workunit.client.0.smithi114.stdout:removed './251/24' 2024-05-27T09:31:48.731 INFO:tasks.workunit.client.0.smithi114.stdout:removed './251/29' 2024-05-27T09:31:48.731 INFO:tasks.workunit.client.0.smithi114.stdout:removed './251/32' 2024-05-27T09:31:48.731 INFO:tasks.workunit.client.0.smithi114.stdout:removed './251/22' 2024-05-27T09:31:48.731 INFO:tasks.workunit.client.0.smithi114.stdout:removed './251/25' 2024-05-27T09:31:48.731 INFO:tasks.workunit.client.0.smithi114.stdout:removed './251/30' 2024-05-27T09:31:48.731 INFO:tasks.workunit.client.0.smithi114.stdout:removed './251/33' 2024-05-27T09:31:48.731 INFO:tasks.workunit.client.0.smithi114.stdout:removed './251/23' 2024-05-27T09:31:48.731 INFO:tasks.workunit.client.0.smithi114.stdout:removed './251/35' 2024-05-27T09:31:48.731 INFO:tasks.workunit.client.0.smithi114.stdout:removed './251/26' 2024-05-27T09:31:48.731 INFO:tasks.workunit.client.0.smithi114.stdout:removed './251/37' 2024-05-27T09:31:48.731 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './251' 2024-05-27T09:31:48.732 INFO:tasks.workunit.client.0.smithi114.stdout:removed './252/21' 2024-05-27T09:31:48.732 INFO:tasks.workunit.client.0.smithi114.stdout:removed './252/27' 2024-05-27T09:31:48.732 INFO:tasks.workunit.client.0.smithi114.stdout:removed './252/31' 2024-05-27T09:31:48.732 INFO:tasks.workunit.client.0.smithi114.stdout:removed './252/36' 2024-05-27T09:31:48.732 INFO:tasks.workunit.client.0.smithi114.stdout:removed './252/20' 2024-05-27T09:31:48.732 INFO:tasks.workunit.client.0.smithi114.stdout:removed './252/39' 2024-05-27T09:31:48.732 INFO:tasks.workunit.client.0.smithi114.stdout:removed './252/34' 2024-05-27T09:31:48.732 INFO:tasks.workunit.client.0.smithi114.stdout:removed './252/38' 2024-05-27T09:31:48.732 INFO:tasks.workunit.client.0.smithi114.stdout:removed './252/28' 2024-05-27T09:31:48.732 INFO:tasks.workunit.client.0.smithi114.stdout:removed './252/24' 2024-05-27T09:31:48.732 INFO:tasks.workunit.client.0.smithi114.stdout:removed './252/29' 2024-05-27T09:31:48.732 INFO:tasks.workunit.client.0.smithi114.stdout:removed './252/32' 2024-05-27T09:31:48.732 INFO:tasks.workunit.client.0.smithi114.stdout:removed './252/22' 2024-05-27T09:31:48.732 INFO:tasks.workunit.client.0.smithi114.stdout:removed './252/25' 2024-05-27T09:31:48.733 INFO:tasks.workunit.client.0.smithi114.stdout:removed './252/30' 2024-05-27T09:31:48.733 INFO:tasks.workunit.client.0.smithi114.stdout:removed './252/33' 2024-05-27T09:31:48.733 INFO:tasks.workunit.client.0.smithi114.stdout:removed './252/23' 2024-05-27T09:31:48.733 INFO:tasks.workunit.client.0.smithi114.stdout:removed './252/35' 2024-05-27T09:31:48.733 INFO:tasks.workunit.client.0.smithi114.stdout:removed './252/26' 2024-05-27T09:31:48.733 INFO:tasks.workunit.client.0.smithi114.stdout:removed './252/37' 2024-05-27T09:31:48.733 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './252' 2024-05-27T09:31:48.733 INFO:tasks.workunit.client.0.smithi114.stdout:removed './253/21' 2024-05-27T09:31:48.733 INFO:tasks.workunit.client.0.smithi114.stdout:removed './253/27' 2024-05-27T09:31:48.733 INFO:tasks.workunit.client.0.smithi114.stdout:removed './253/31' 2024-05-27T09:31:48.733 INFO:tasks.workunit.client.0.smithi114.stdout:removed './253/36' 2024-05-27T09:31:48.733 INFO:tasks.workunit.client.0.smithi114.stdout:removed './253/20' 2024-05-27T09:31:48.733 INFO:tasks.workunit.client.0.smithi114.stdout:removed './253/39' 2024-05-27T09:31:48.733 INFO:tasks.workunit.client.0.smithi114.stdout:removed './253/34' 2024-05-27T09:31:48.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './253/38' 2024-05-27T09:31:48.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './253/28' 2024-05-27T09:31:48.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './253/24' 2024-05-27T09:31:48.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './253/29' 2024-05-27T09:31:48.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './253/32' 2024-05-27T09:31:48.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './253/22' 2024-05-27T09:31:48.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './253/25' 2024-05-27T09:31:48.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './253/30' 2024-05-27T09:31:48.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './253/33' 2024-05-27T09:31:48.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './253/23' 2024-05-27T09:31:48.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './253/35' 2024-05-27T09:31:48.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './253/26' 2024-05-27T09:31:48.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed './253/37' 2024-05-27T09:31:48.734 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './253' 2024-05-27T09:31:48.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './254/21' 2024-05-27T09:31:48.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './254/27' 2024-05-27T09:31:48.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './254/31' 2024-05-27T09:31:48.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './254/36' 2024-05-27T09:31:48.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './254/20' 2024-05-27T09:31:48.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './254/39' 2024-05-27T09:31:48.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './254/34' 2024-05-27T09:31:48.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './254/38' 2024-05-27T09:31:48.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './254/28' 2024-05-27T09:31:48.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './254/24' 2024-05-27T09:31:48.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './254/29' 2024-05-27T09:31:48.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './254/32' 2024-05-27T09:31:48.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './254/22' 2024-05-27T09:31:48.735 INFO:tasks.workunit.client.0.smithi114.stdout:removed './254/25' 2024-05-27T09:31:48.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './254/30' 2024-05-27T09:31:48.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './254/33' 2024-05-27T09:31:48.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './254/23' 2024-05-27T09:31:48.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './254/35' 2024-05-27T09:31:48.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './254/26' 2024-05-27T09:31:48.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './254/37' 2024-05-27T09:31:48.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './254' 2024-05-27T09:31:48.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './255/21' 2024-05-27T09:31:48.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './255/27' 2024-05-27T09:31:48.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './255/31' 2024-05-27T09:31:48.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './255/36' 2024-05-27T09:31:48.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './255/20' 2024-05-27T09:31:48.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './255/39' 2024-05-27T09:31:48.736 INFO:tasks.workunit.client.0.smithi114.stdout:removed './255/34' 2024-05-27T09:31:48.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './255/38' 2024-05-27T09:31:48.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './255/28' 2024-05-27T09:31:48.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './255/24' 2024-05-27T09:31:48.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './255/29' 2024-05-27T09:31:48.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './255/32' 2024-05-27T09:31:48.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './255/22' 2024-05-27T09:31:48.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './255/25' 2024-05-27T09:31:48.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './255/30' 2024-05-27T09:31:48.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './255/33' 2024-05-27T09:31:48.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './255/23' 2024-05-27T09:31:48.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './255/35' 2024-05-27T09:31:48.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './255/26' 2024-05-27T09:31:48.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed './255/37' 2024-05-27T09:31:48.737 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './255' 2024-05-27T09:31:48.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './256/21' 2024-05-27T09:31:48.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './256/27' 2024-05-27T09:31:48.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './256/31' 2024-05-27T09:31:48.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './256/36' 2024-05-27T09:31:48.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './256/20' 2024-05-27T09:31:48.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './256/39' 2024-05-27T09:31:48.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './256/34' 2024-05-27T09:31:48.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './256/38' 2024-05-27T09:31:48.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './256/28' 2024-05-27T09:31:48.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './256/24' 2024-05-27T09:31:48.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './256/29' 2024-05-27T09:31:48.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './256/32' 2024-05-27T09:31:48.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './256/22' 2024-05-27T09:31:48.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './256/25' 2024-05-27T09:31:48.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './256/30' 2024-05-27T09:31:49.620 INFO:tasks.workunit.client.0.smithi114.stdout:removed './256/33' 2024-05-27T09:31:49.621 INFO:tasks.workunit.client.0.smithi114.stdout:removed './256/23' 2024-05-27T09:31:49.621 INFO:tasks.workunit.client.0.smithi114.stdout:removed './256/35' 2024-05-27T09:31:49.621 INFO:tasks.workunit.client.0.smithi114.stdout:removed './256/26' 2024-05-27T09:31:49.621 INFO:tasks.workunit.client.0.smithi114.stdout:removed './256/37' 2024-05-27T09:31:49.621 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './256' 2024-05-27T09:31:49.621 INFO:tasks.workunit.client.0.smithi114.stdout:removed './257/21' 2024-05-27T09:31:49.621 INFO:tasks.workunit.client.0.smithi114.stdout:removed './257/27' 2024-05-27T09:31:49.621 INFO:tasks.workunit.client.0.smithi114.stdout:removed './257/31' 2024-05-27T09:31:49.621 INFO:tasks.workunit.client.0.smithi114.stdout:removed './257/36' 2024-05-27T09:31:49.621 INFO:tasks.workunit.client.0.smithi114.stdout:removed './257/20' 2024-05-27T09:31:49.621 INFO:tasks.workunit.client.0.smithi114.stdout:removed './257/39' 2024-05-27T09:31:49.621 INFO:tasks.workunit.client.0.smithi114.stdout:removed './257/34' 2024-05-27T09:31:49.621 INFO:tasks.workunit.client.0.smithi114.stdout:removed './257/38' 2024-05-27T09:31:49.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './257/28' 2024-05-27T09:31:49.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './257/24' 2024-05-27T09:31:49.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './257/29' 2024-05-27T09:31:49.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './257/32' 2024-05-27T09:31:49.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './257/22' 2024-05-27T09:31:49.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './257/25' 2024-05-27T09:31:49.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './257/30' 2024-05-27T09:31:49.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './257/33' 2024-05-27T09:31:49.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './257/23' 2024-05-27T09:31:49.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './257/35' 2024-05-27T09:31:49.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './257/26' 2024-05-27T09:31:49.622 INFO:tasks.workunit.client.0.smithi114.stdout:removed './257/37' 2024-05-27T09:31:49.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './257' 2024-05-27T09:31:49.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './258/21' 2024-05-27T09:31:49.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './258/27' 2024-05-27T09:31:49.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './258/31' 2024-05-27T09:31:49.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './258/36' 2024-05-27T09:31:49.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './258/20' 2024-05-27T09:31:49.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './258/39' 2024-05-27T09:31:49.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './258/34' 2024-05-27T09:31:49.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './258/38' 2024-05-27T09:31:49.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './258/28' 2024-05-27T09:31:49.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './258/24' 2024-05-27T09:31:49.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './258/29' 2024-05-27T09:31:49.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './258/32' 2024-05-27T09:31:49.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './258/22' 2024-05-27T09:31:49.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './258/25' 2024-05-27T09:31:49.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './258/30' 2024-05-27T09:31:49.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './258/33' 2024-05-27T09:31:49.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './258/23' 2024-05-27T09:31:49.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './258/35' 2024-05-27T09:31:49.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './258/26' 2024-05-27T09:31:49.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './258/37' 2024-05-27T09:31:49.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './258' 2024-05-27T09:31:49.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './259/21' 2024-05-27T09:31:49.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './259/27' 2024-05-27T09:31:49.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './259/31' 2024-05-27T09:31:49.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './259/36' 2024-05-27T09:31:49.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './259/20' 2024-05-27T09:31:49.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './259/39' 2024-05-27T09:31:49.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './259/34' 2024-05-27T09:31:49.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './259/38' 2024-05-27T09:31:49.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './259/28' 2024-05-27T09:31:49.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './259/24' 2024-05-27T09:31:49.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './259/29' 2024-05-27T09:31:49.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './259/32' 2024-05-27T09:31:49.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './259/22' 2024-05-27T09:31:49.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './259/25' 2024-05-27T09:31:49.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './259/30' 2024-05-27T09:31:49.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './259/33' 2024-05-27T09:31:49.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './259/23' 2024-05-27T09:31:49.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './259/35' 2024-05-27T09:31:49.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './259/26' 2024-05-27T09:31:49.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './259/37' 2024-05-27T09:31:49.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './259' 2024-05-27T09:31:49.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './260/21' 2024-05-27T09:31:49.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './260/27' 2024-05-27T09:31:49.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './260/31' 2024-05-27T09:31:49.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './260/36' 2024-05-27T09:31:49.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './260/20' 2024-05-27T09:31:49.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './260/39' 2024-05-27T09:31:49.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './260/34' 2024-05-27T09:31:49.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './260/38' 2024-05-27T09:31:49.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './260/28' 2024-05-27T09:31:49.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './260/24' 2024-05-27T09:31:49.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './260/29' 2024-05-27T09:31:49.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './260/32' 2024-05-27T09:31:49.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './260/22' 2024-05-27T09:31:49.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './260/25' 2024-05-27T09:31:49.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './260/30' 2024-05-27T09:31:49.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './260/33' 2024-05-27T09:31:49.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './260/23' 2024-05-27T09:31:49.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './260/35' 2024-05-27T09:31:49.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './260/26' 2024-05-27T09:31:49.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './260/37' 2024-05-27T09:31:49.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './260' 2024-05-27T09:31:49.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './261/21' 2024-05-27T09:31:49.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './261/27' 2024-05-27T09:31:49.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './261/31' 2024-05-27T09:31:49.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './261/36' 2024-05-27T09:31:49.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './261/20' 2024-05-27T09:31:49.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './261/39' 2024-05-27T09:31:49.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './261/34' 2024-05-27T09:31:49.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './261/38' 2024-05-27T09:31:49.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './261/28' 2024-05-27T09:31:49.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './261/24' 2024-05-27T09:31:49.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './261/29' 2024-05-27T09:31:49.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './261/32' 2024-05-27T09:31:49.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './261/22' 2024-05-27T09:31:49.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './261/25' 2024-05-27T09:31:49.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './261/30' 2024-05-27T09:31:49.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './261/33' 2024-05-27T09:31:49.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './261/23' 2024-05-27T09:31:49.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './261/35' 2024-05-27T09:31:49.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './261/26' 2024-05-27T09:31:49.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './261/37' 2024-05-27T09:31:49.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './261' 2024-05-27T09:31:49.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './262/21' 2024-05-27T09:31:49.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './262/27' 2024-05-27T09:31:49.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './262/31' 2024-05-27T09:31:49.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './262/36' 2024-05-27T09:31:49.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './262/20' 2024-05-27T09:31:49.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './262/39' 2024-05-27T09:31:49.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './262/34' 2024-05-27T09:31:49.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './262/38' 2024-05-27T09:31:49.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './262/28' 2024-05-27T09:31:49.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './262/24' 2024-05-27T09:31:49.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './262/29' 2024-05-27T09:31:49.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './262/32' 2024-05-27T09:31:49.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './262/22' 2024-05-27T09:31:49.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './262/25' 2024-05-27T09:31:49.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './262/30' 2024-05-27T09:31:49.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './262/33' 2024-05-27T09:31:49.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './262/23' 2024-05-27T09:31:49.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './262/35' 2024-05-27T09:31:49.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './262/26' 2024-05-27T09:31:49.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './262/37' 2024-05-27T09:31:49.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './262' 2024-05-27T09:31:49.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './263/21' 2024-05-27T09:31:49.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './263/27' 2024-05-27T09:31:49.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './263/31' 2024-05-27T09:31:49.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './263/36' 2024-05-27T09:31:49.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './263/20' 2024-05-27T09:31:49.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './263/39' 2024-05-27T09:31:49.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './263/34' 2024-05-27T09:31:49.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './263/38' 2024-05-27T09:31:49.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './263/28' 2024-05-27T09:31:49.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './263/24' 2024-05-27T09:31:49.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './263/29' 2024-05-27T09:31:49.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './263/32' 2024-05-27T09:31:49.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './263/22' 2024-05-27T09:31:49.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './263/25' 2024-05-27T09:31:49.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './263/30' 2024-05-27T09:31:49.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './263/33' 2024-05-27T09:31:49.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './263/23' 2024-05-27T09:31:49.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './263/35' 2024-05-27T09:31:49.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './263/26' 2024-05-27T09:31:49.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './263/37' 2024-05-27T09:31:49.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './263' 2024-05-27T09:31:49.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './264/21' 2024-05-27T09:31:49.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './264/27' 2024-05-27T09:31:49.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './264/31' 2024-05-27T09:31:49.632 INFO:tasks.workunit.client.0.smithi114.stdout:removed './264/36' 2024-05-27T09:31:49.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './264/20' 2024-05-27T09:31:49.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './264/39' 2024-05-27T09:31:49.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './264/34' 2024-05-27T09:31:49.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './264/38' 2024-05-27T09:31:49.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './264/28' 2024-05-27T09:31:49.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './264/24' 2024-05-27T09:31:49.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './264/29' 2024-05-27T09:31:49.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './264/32' 2024-05-27T09:31:49.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './264/22' 2024-05-27T09:31:49.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './264/25' 2024-05-27T09:31:49.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './264/30' 2024-05-27T09:31:49.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './264/33' 2024-05-27T09:31:49.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './264/23' 2024-05-27T09:31:49.633 INFO:tasks.workunit.client.0.smithi114.stdout:removed './264/35' 2024-05-27T09:31:49.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './264/26' 2024-05-27T09:31:49.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './264/37' 2024-05-27T09:31:49.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './264' 2024-05-27T09:31:49.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './265/21' 2024-05-27T09:31:49.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './265/27' 2024-05-27T09:31:49.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './265/31' 2024-05-27T09:31:49.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './265/36' 2024-05-27T09:31:49.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './265/20' 2024-05-27T09:31:49.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './265/39' 2024-05-27T09:31:49.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './265/34' 2024-05-27T09:31:49.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './265/38' 2024-05-27T09:31:49.634 INFO:tasks.workunit.client.0.smithi114.stdout:removed './265/28' 2024-05-27T09:31:49.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './265/24' 2024-05-27T09:31:49.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './265/29' 2024-05-27T09:31:49.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './265/32' 2024-05-27T09:31:49.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './265/22' 2024-05-27T09:31:49.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './265/25' 2024-05-27T09:31:49.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './265/30' 2024-05-27T09:31:49.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './265/33' 2024-05-27T09:31:49.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './265/23' 2024-05-27T09:31:49.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './265/35' 2024-05-27T09:31:49.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './265/26' 2024-05-27T09:31:49.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './265/37' 2024-05-27T09:31:49.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './265' 2024-05-27T09:31:49.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './266/21' 2024-05-27T09:31:49.635 INFO:tasks.workunit.client.0.smithi114.stdout:removed './266/27' 2024-05-27T09:31:49.636 INFO:tasks.workunit.client.0.smithi114.stdout:removed './266/31' 2024-05-27T09:31:49.636 INFO:tasks.workunit.client.0.smithi114.stdout:removed './266/36' 2024-05-27T09:31:49.636 INFO:tasks.workunit.client.0.smithi114.stdout:removed './266/20' 2024-05-27T09:31:49.636 INFO:tasks.workunit.client.0.smithi114.stdout:removed './266/39' 2024-05-27T09:31:49.636 INFO:tasks.workunit.client.0.smithi114.stdout:removed './266/34' 2024-05-27T09:31:49.636 INFO:tasks.workunit.client.0.smithi114.stdout:removed './266/38' 2024-05-27T09:31:49.636 INFO:tasks.workunit.client.0.smithi114.stdout:removed './266/28' 2024-05-27T09:31:49.636 INFO:tasks.workunit.client.0.smithi114.stdout:removed './266/24' 2024-05-27T09:31:49.636 INFO:tasks.workunit.client.0.smithi114.stdout:removed './266/29' 2024-05-27T09:31:49.636 INFO:tasks.workunit.client.0.smithi114.stdout:removed './266/32' 2024-05-27T09:31:49.636 INFO:tasks.workunit.client.0.smithi114.stdout:removed './266/22' 2024-05-27T09:31:49.636 INFO:tasks.workunit.client.0.smithi114.stdout:removed './266/25' 2024-05-27T09:31:49.636 INFO:tasks.workunit.client.0.smithi114.stdout:removed './266/30' 2024-05-27T09:31:49.636 INFO:tasks.workunit.client.0.smithi114.stdout:removed './266/33' 2024-05-27T09:31:49.637 INFO:tasks.workunit.client.0.smithi114.stdout:removed './266/23' 2024-05-27T09:31:50.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed './266/35' 2024-05-27T09:31:50.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed './266/26' 2024-05-27T09:31:50.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed './266/37' 2024-05-27T09:31:50.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './266' 2024-05-27T09:31:50.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed './267/21' 2024-05-27T09:31:50.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed './267/27' 2024-05-27T09:31:50.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed './267/31' 2024-05-27T09:31:50.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed './267/36' 2024-05-27T09:31:50.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed './267/20' 2024-05-27T09:31:50.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed './267/39' 2024-05-27T09:31:50.594 INFO:tasks.workunit.client.0.smithi114.stdout:removed './267/34' 2024-05-27T09:31:50.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './267/38' 2024-05-27T09:31:50.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './267/28' 2024-05-27T09:31:50.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './267/24' 2024-05-27T09:31:50.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './267/29' 2024-05-27T09:31:50.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './267/32' 2024-05-27T09:31:50.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './267/22' 2024-05-27T09:31:50.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './267/25' 2024-05-27T09:31:50.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './267/30' 2024-05-27T09:31:50.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './267/33' 2024-05-27T09:31:50.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './267/23' 2024-05-27T09:31:50.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './267/35' 2024-05-27T09:31:50.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './267/26' 2024-05-27T09:31:50.595 INFO:tasks.workunit.client.0.smithi114.stdout:removed './267/37' 2024-05-27T09:31:50.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './267' 2024-05-27T09:31:50.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './268/21' 2024-05-27T09:31:50.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './268/27' 2024-05-27T09:31:50.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './268/31' 2024-05-27T09:31:50.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './268/36' 2024-05-27T09:31:50.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './268/20' 2024-05-27T09:31:50.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './268/39' 2024-05-27T09:31:50.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './268/34' 2024-05-27T09:31:50.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './268/38' 2024-05-27T09:31:50.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './268/28' 2024-05-27T09:31:50.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './268/24' 2024-05-27T09:31:50.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './268/29' 2024-05-27T09:31:50.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './268/32' 2024-05-27T09:31:50.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './268/22' 2024-05-27T09:31:50.596 INFO:tasks.workunit.client.0.smithi114.stdout:removed './268/25' 2024-05-27T09:31:50.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './268/30' 2024-05-27T09:31:50.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './268/33' 2024-05-27T09:31:50.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './268/23' 2024-05-27T09:31:50.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './268/35' 2024-05-27T09:31:50.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './268/26' 2024-05-27T09:31:50.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './268/37' 2024-05-27T09:31:50.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './268' 2024-05-27T09:31:50.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './269/21' 2024-05-27T09:31:50.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './269/27' 2024-05-27T09:31:50.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './269/31' 2024-05-27T09:31:50.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './269/36' 2024-05-27T09:31:50.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './269/20' 2024-05-27T09:31:50.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './269/39' 2024-05-27T09:31:50.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './269/34' 2024-05-27T09:31:50.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './269/38' 2024-05-27T09:31:50.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './269/28' 2024-05-27T09:31:50.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './269/24' 2024-05-27T09:31:50.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './269/29' 2024-05-27T09:31:50.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './269/32' 2024-05-27T09:31:50.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './269/22' 2024-05-27T09:31:50.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './269/25' 2024-05-27T09:31:50.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './269/30' 2024-05-27T09:31:50.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './269/33' 2024-05-27T09:31:50.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './269/23' 2024-05-27T09:31:50.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './269/35' 2024-05-27T09:31:50.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './269/26' 2024-05-27T09:31:50.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './269/37' 2024-05-27T09:31:50.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './269' 2024-05-27T09:31:50.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './270/21' 2024-05-27T09:31:50.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './270/27' 2024-05-27T09:31:50.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './270/31' 2024-05-27T09:31:50.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './270/36' 2024-05-27T09:31:50.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './270/20' 2024-05-27T09:31:50.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './270/39' 2024-05-27T09:31:50.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './270/34' 2024-05-27T09:31:50.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './270/38' 2024-05-27T09:31:50.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './270/28' 2024-05-27T09:31:50.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './270/24' 2024-05-27T09:31:50.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './270/29' 2024-05-27T09:31:50.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './270/32' 2024-05-27T09:31:50.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './270/22' 2024-05-27T09:31:50.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './270/25' 2024-05-27T09:31:50.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './270/30' 2024-05-27T09:31:50.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './270/33' 2024-05-27T09:31:50.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './270/23' 2024-05-27T09:31:50.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './270/35' 2024-05-27T09:31:50.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './270/26' 2024-05-27T09:31:50.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './270/37' 2024-05-27T09:31:50.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './270' 2024-05-27T09:31:50.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './271/21' 2024-05-27T09:31:50.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './271/27' 2024-05-27T09:31:50.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './271/31' 2024-05-27T09:31:50.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './271/36' 2024-05-27T09:31:50.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './271/20' 2024-05-27T09:31:50.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './271/39' 2024-05-27T09:31:50.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './271/34' 2024-05-27T09:31:50.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './271/38' 2024-05-27T09:31:50.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './271/28' 2024-05-27T09:31:50.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './271/24' 2024-05-27T09:31:50.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './271/29' 2024-05-27T09:31:50.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './271/32' 2024-05-27T09:31:50.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './271/22' 2024-05-27T09:31:50.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './271/25' 2024-05-27T09:31:50.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './271/30' 2024-05-27T09:31:50.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './271/33' 2024-05-27T09:31:50.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './271/23' 2024-05-27T09:31:50.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './271/35' 2024-05-27T09:31:50.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './271/26' 2024-05-27T09:31:50.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './271/37' 2024-05-27T09:31:50.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './271' 2024-05-27T09:31:50.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './272/21' 2024-05-27T09:31:50.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './272/27' 2024-05-27T09:31:50.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './272/31' 2024-05-27T09:31:50.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './272/36' 2024-05-27T09:31:50.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './272/20' 2024-05-27T09:31:50.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './272/39' 2024-05-27T09:31:50.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './272/34' 2024-05-27T09:31:50.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './272/38' 2024-05-27T09:31:50.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './272/28' 2024-05-27T09:31:50.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './272/24' 2024-05-27T09:31:50.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './272/29' 2024-05-27T09:31:50.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './272/32' 2024-05-27T09:31:50.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './272/22' 2024-05-27T09:31:50.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './272/25' 2024-05-27T09:31:50.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './272/30' 2024-05-27T09:31:50.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './272/33' 2024-05-27T09:31:50.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './272/23' 2024-05-27T09:31:50.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './272/35' 2024-05-27T09:31:50.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './272/26' 2024-05-27T09:31:50.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './272/37' 2024-05-27T09:31:50.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './272' 2024-05-27T09:31:50.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './273/21' 2024-05-27T09:31:50.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './273/27' 2024-05-27T09:31:50.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './273/31' 2024-05-27T09:31:50.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './273/36' 2024-05-27T09:31:50.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './273/20' 2024-05-27T09:31:50.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './273/39' 2024-05-27T09:31:50.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './273/34' 2024-05-27T09:31:50.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './273/38' 2024-05-27T09:31:50.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './273/28' 2024-05-27T09:31:50.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './273/24' 2024-05-27T09:31:50.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './273/29' 2024-05-27T09:31:50.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './273/32' 2024-05-27T09:31:50.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './273/22' 2024-05-27T09:31:50.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './273/25' 2024-05-27T09:31:50.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './273/30' 2024-05-27T09:31:50.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './273/33' 2024-05-27T09:31:50.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './273/23' 2024-05-27T09:31:50.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './273/35' 2024-05-27T09:31:50.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './273/26' 2024-05-27T09:31:50.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './273/37' 2024-05-27T09:31:50.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './273' 2024-05-27T09:31:50.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './274/21' 2024-05-27T09:31:50.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './274/27' 2024-05-27T09:31:50.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './274/31' 2024-05-27T09:31:50.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './274/36' 2024-05-27T09:31:50.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './274/20' 2024-05-27T09:31:50.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './274/39' 2024-05-27T09:31:50.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './274/34' 2024-05-27T09:31:50.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './274/38' 2024-05-27T09:31:50.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './274/28' 2024-05-27T09:31:50.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './274/24' 2024-05-27T09:31:50.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './274/29' 2024-05-27T09:31:50.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './274/32' 2024-05-27T09:31:50.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './274/22' 2024-05-27T09:31:50.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './274/25' 2024-05-27T09:31:50.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './274/30' 2024-05-27T09:31:50.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './274/33' 2024-05-27T09:31:50.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './274/23' 2024-05-27T09:31:50.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './274/35' 2024-05-27T09:31:50.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './274/26' 2024-05-27T09:31:50.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './274/37' 2024-05-27T09:31:50.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './274' 2024-05-27T09:31:50.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './275/21' 2024-05-27T09:31:50.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './275/27' 2024-05-27T09:31:50.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './275/31' 2024-05-27T09:31:50.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './275/36' 2024-05-27T09:31:50.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './275/20' 2024-05-27T09:31:50.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './275/39' 2024-05-27T09:31:50.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './275/34' 2024-05-27T09:31:50.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './275/38' 2024-05-27T09:31:50.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './275/28' 2024-05-27T09:31:50.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './275/24' 2024-05-27T09:31:50.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './275/29' 2024-05-27T09:31:50.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './275/32' 2024-05-27T09:31:50.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './275/22' 2024-05-27T09:31:50.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './275/25' 2024-05-27T09:31:50.606 INFO:tasks.workunit.client.0.smithi114.stdout:removed './275/30' 2024-05-27T09:31:50.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './275/33' 2024-05-27T09:31:50.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './275/23' 2024-05-27T09:31:50.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './275/35' 2024-05-27T09:31:50.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './275/26' 2024-05-27T09:31:50.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './275/37' 2024-05-27T09:31:50.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './275' 2024-05-27T09:31:50.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './276/21' 2024-05-27T09:31:50.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './276/27' 2024-05-27T09:31:50.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './276/31' 2024-05-27T09:31:50.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './276/36' 2024-05-27T09:31:50.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './276/20' 2024-05-27T09:31:50.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './276/39' 2024-05-27T09:31:50.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './276/34' 2024-05-27T09:31:50.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './276/38' 2024-05-27T09:31:50.607 INFO:tasks.workunit.client.0.smithi114.stdout:removed './276/28' 2024-05-27T09:31:50.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './276/24' 2024-05-27T09:31:50.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './276/29' 2024-05-27T09:31:50.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './276/32' 2024-05-27T09:31:50.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './276/22' 2024-05-27T09:31:50.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './276/25' 2024-05-27T09:31:50.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './276/30' 2024-05-27T09:31:50.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './276/33' 2024-05-27T09:31:50.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './276/23' 2024-05-27T09:31:50.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './276/35' 2024-05-27T09:31:50.608 INFO:tasks.workunit.client.0.smithi114.stdout:removed './276/26' 2024-05-27T09:31:51.701 INFO:tasks.workunit.client.0.smithi114.stdout:removed './276/37' 2024-05-27T09:31:51.701 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './276' 2024-05-27T09:31:51.701 INFO:tasks.workunit.client.0.smithi114.stdout:removed './277/21' 2024-05-27T09:31:51.701 INFO:tasks.workunit.client.0.smithi114.stdout:removed './277/27' 2024-05-27T09:31:51.701 INFO:tasks.workunit.client.0.smithi114.stdout:removed './277/31' 2024-05-27T09:31:51.701 INFO:tasks.workunit.client.0.smithi114.stdout:removed './277/36' 2024-05-27T09:31:51.701 INFO:tasks.workunit.client.0.smithi114.stdout:removed './277/20' 2024-05-27T09:31:51.701 INFO:tasks.workunit.client.0.smithi114.stdout:removed './277/39' 2024-05-27T09:31:51.701 INFO:tasks.workunit.client.0.smithi114.stdout:removed './277/34' 2024-05-27T09:31:51.702 INFO:tasks.workunit.client.0.smithi114.stdout:removed './277/38' 2024-05-27T09:31:51.702 INFO:tasks.workunit.client.0.smithi114.stdout:removed './277/28' 2024-05-27T09:31:51.702 INFO:tasks.workunit.client.0.smithi114.stdout:removed './277/24' 2024-05-27T09:31:51.702 INFO:tasks.workunit.client.0.smithi114.stdout:removed './277/29' 2024-05-27T09:31:51.702 INFO:tasks.workunit.client.0.smithi114.stdout:removed './277/32' 2024-05-27T09:31:51.702 INFO:tasks.workunit.client.0.smithi114.stdout:removed './277/22' 2024-05-27T09:31:51.702 INFO:tasks.workunit.client.0.smithi114.stdout:removed './277/25' 2024-05-27T09:31:51.702 INFO:tasks.workunit.client.0.smithi114.stdout:removed './277/30' 2024-05-27T09:31:51.702 INFO:tasks.workunit.client.0.smithi114.stdout:removed './277/33' 2024-05-27T09:31:51.702 INFO:tasks.workunit.client.0.smithi114.stdout:removed './277/23' 2024-05-27T09:31:51.702 INFO:tasks.workunit.client.0.smithi114.stdout:removed './277/35' 2024-05-27T09:31:51.702 INFO:tasks.workunit.client.0.smithi114.stdout:removed './277/26' 2024-05-27T09:31:51.702 INFO:tasks.workunit.client.0.smithi114.stdout:removed './277/37' 2024-05-27T09:31:51.703 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './277' 2024-05-27T09:31:51.703 INFO:tasks.workunit.client.0.smithi114.stdout:removed './278/21' 2024-05-27T09:31:51.703 INFO:tasks.workunit.client.0.smithi114.stdout:removed './278/27' 2024-05-27T09:31:51.703 INFO:tasks.workunit.client.0.smithi114.stdout:removed './278/31' 2024-05-27T09:31:51.703 INFO:tasks.workunit.client.0.smithi114.stdout:removed './278/36' 2024-05-27T09:31:51.703 INFO:tasks.workunit.client.0.smithi114.stdout:removed './278/20' 2024-05-27T09:31:51.703 INFO:tasks.workunit.client.0.smithi114.stdout:removed './278/39' 2024-05-27T09:31:51.703 INFO:tasks.workunit.client.0.smithi114.stdout:removed './278/34' 2024-05-27T09:31:51.703 INFO:tasks.workunit.client.0.smithi114.stdout:removed './278/38' 2024-05-27T09:31:51.703 INFO:tasks.workunit.client.0.smithi114.stdout:removed './278/28' 2024-05-27T09:31:51.703 INFO:tasks.workunit.client.0.smithi114.stdout:removed './278/24' 2024-05-27T09:31:51.703 INFO:tasks.workunit.client.0.smithi114.stdout:removed './278/29' 2024-05-27T09:31:51.703 INFO:tasks.workunit.client.0.smithi114.stdout:removed './278/32' 2024-05-27T09:31:51.703 INFO:tasks.workunit.client.0.smithi114.stdout:removed './278/22' 2024-05-27T09:31:51.704 INFO:tasks.workunit.client.0.smithi114.stdout:removed './278/25' 2024-05-27T09:31:51.704 INFO:tasks.workunit.client.0.smithi114.stdout:removed './278/30' 2024-05-27T09:31:51.704 INFO:tasks.workunit.client.0.smithi114.stdout:removed './278/33' 2024-05-27T09:31:51.704 INFO:tasks.workunit.client.0.smithi114.stdout:removed './278/23' 2024-05-27T09:31:51.704 INFO:tasks.workunit.client.0.smithi114.stdout:removed './278/35' 2024-05-27T09:31:51.704 INFO:tasks.workunit.client.0.smithi114.stdout:removed './278/26' 2024-05-27T09:31:51.704 INFO:tasks.workunit.client.0.smithi114.stdout:removed './278/37' 2024-05-27T09:31:51.704 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './278' 2024-05-27T09:31:51.704 INFO:tasks.workunit.client.0.smithi114.stdout:removed './279/21' 2024-05-27T09:31:51.704 INFO:tasks.workunit.client.0.smithi114.stdout:removed './279/27' 2024-05-27T09:31:51.704 INFO:tasks.workunit.client.0.smithi114.stdout:removed './279/31' 2024-05-27T09:31:51.704 INFO:tasks.workunit.client.0.smithi114.stdout:removed './279/36' 2024-05-27T09:31:51.704 INFO:tasks.workunit.client.0.smithi114.stdout:removed './279/20' 2024-05-27T09:31:51.705 INFO:tasks.workunit.client.0.smithi114.stdout:removed './279/39' 2024-05-27T09:31:51.705 INFO:tasks.workunit.client.0.smithi114.stdout:removed './279/34' 2024-05-27T09:31:51.705 INFO:tasks.workunit.client.0.smithi114.stdout:removed './279/38' 2024-05-27T09:31:51.705 INFO:tasks.workunit.client.0.smithi114.stdout:removed './279/28' 2024-05-27T09:31:51.705 INFO:tasks.workunit.client.0.smithi114.stdout:removed './279/24' 2024-05-27T09:31:51.705 INFO:tasks.workunit.client.0.smithi114.stdout:removed './279/29' 2024-05-27T09:31:51.705 INFO:tasks.workunit.client.0.smithi114.stdout:removed './279/32' 2024-05-27T09:31:51.705 INFO:tasks.workunit.client.0.smithi114.stdout:removed './279/22' 2024-05-27T09:31:51.705 INFO:tasks.workunit.client.0.smithi114.stdout:removed './279/25' 2024-05-27T09:31:51.705 INFO:tasks.workunit.client.0.smithi114.stdout:removed './279/30' 2024-05-27T09:31:51.705 INFO:tasks.workunit.client.0.smithi114.stdout:removed './279/33' 2024-05-27T09:31:51.705 INFO:tasks.workunit.client.0.smithi114.stdout:removed './279/23' 2024-05-27T09:31:51.705 INFO:tasks.workunit.client.0.smithi114.stdout:removed './279/35' 2024-05-27T09:31:51.705 INFO:tasks.workunit.client.0.smithi114.stdout:removed './279/26' 2024-05-27T09:31:51.706 INFO:tasks.workunit.client.0.smithi114.stdout:removed './279/37' 2024-05-27T09:31:51.706 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './279' 2024-05-27T09:31:51.706 INFO:tasks.workunit.client.0.smithi114.stdout:removed './280/21' 2024-05-27T09:31:51.706 INFO:tasks.workunit.client.0.smithi114.stdout:removed './280/27' 2024-05-27T09:31:51.706 INFO:tasks.workunit.client.0.smithi114.stdout:removed './280/31' 2024-05-27T09:31:51.706 INFO:tasks.workunit.client.0.smithi114.stdout:removed './280/36' 2024-05-27T09:31:51.706 INFO:tasks.workunit.client.0.smithi114.stdout:removed './280/20' 2024-05-27T09:31:51.706 INFO:tasks.workunit.client.0.smithi114.stdout:removed './280/39' 2024-05-27T09:31:51.706 INFO:tasks.workunit.client.0.smithi114.stdout:removed './280/34' 2024-05-27T09:31:51.706 INFO:tasks.workunit.client.0.smithi114.stdout:removed './280/38' 2024-05-27T09:31:51.706 INFO:tasks.workunit.client.0.smithi114.stdout:removed './280/28' 2024-05-27T09:31:51.706 INFO:tasks.workunit.client.0.smithi114.stdout:removed './280/24' 2024-05-27T09:31:51.706 INFO:tasks.workunit.client.0.smithi114.stdout:removed './280/29' 2024-05-27T09:31:51.706 INFO:tasks.workunit.client.0.smithi114.stdout:removed './280/32' 2024-05-27T09:31:51.707 INFO:tasks.workunit.client.0.smithi114.stdout:removed './280/22' 2024-05-27T09:31:51.707 INFO:tasks.workunit.client.0.smithi114.stdout:removed './280/25' 2024-05-27T09:31:51.707 INFO:tasks.workunit.client.0.smithi114.stdout:removed './280/30' 2024-05-27T09:31:51.707 INFO:tasks.workunit.client.0.smithi114.stdout:removed './280/33' 2024-05-27T09:31:51.707 INFO:tasks.workunit.client.0.smithi114.stdout:removed './280/23' 2024-05-27T09:31:51.707 INFO:tasks.workunit.client.0.smithi114.stdout:removed './280/35' 2024-05-27T09:31:51.707 INFO:tasks.workunit.client.0.smithi114.stdout:removed './280/26' 2024-05-27T09:31:51.707 INFO:tasks.workunit.client.0.smithi114.stdout:removed './280/37' 2024-05-27T09:31:51.707 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './280' 2024-05-27T09:31:51.707 INFO:tasks.workunit.client.0.smithi114.stdout:removed './281/21' 2024-05-27T09:31:51.707 INFO:tasks.workunit.client.0.smithi114.stdout:removed './281/27' 2024-05-27T09:31:51.708 INFO:tasks.workunit.client.0.smithi114.stdout:removed './281/31' 2024-05-27T09:31:51.708 INFO:tasks.workunit.client.0.smithi114.stdout:removed './281/36' 2024-05-27T09:31:51.708 INFO:tasks.workunit.client.0.smithi114.stdout:removed './281/20' 2024-05-27T09:31:51.708 INFO:tasks.workunit.client.0.smithi114.stdout:removed './281/39' 2024-05-27T09:31:51.708 INFO:tasks.workunit.client.0.smithi114.stdout:removed './281/34' 2024-05-27T09:31:51.708 INFO:tasks.workunit.client.0.smithi114.stdout:removed './281/38' 2024-05-27T09:31:51.708 INFO:tasks.workunit.client.0.smithi114.stdout:removed './281/28' 2024-05-27T09:31:51.708 INFO:tasks.workunit.client.0.smithi114.stdout:removed './281/24' 2024-05-27T09:31:51.708 INFO:tasks.workunit.client.0.smithi114.stdout:removed './281/29' 2024-05-27T09:31:51.708 INFO:tasks.workunit.client.0.smithi114.stdout:removed './281/32' 2024-05-27T09:31:51.708 INFO:tasks.workunit.client.0.smithi114.stdout:removed './281/22' 2024-05-27T09:31:51.708 INFO:tasks.workunit.client.0.smithi114.stdout:removed './281/25' 2024-05-27T09:31:51.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './281/30' 2024-05-27T09:31:51.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './281/33' 2024-05-27T09:31:51.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './281/23' 2024-05-27T09:31:51.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './281/35' 2024-05-27T09:31:51.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './281/26' 2024-05-27T09:31:51.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './281/37' 2024-05-27T09:31:51.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './281' 2024-05-27T09:31:51.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './282/21' 2024-05-27T09:31:51.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './282/27' 2024-05-27T09:31:51.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './282/31' 2024-05-27T09:31:51.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './282/36' 2024-05-27T09:31:51.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './282/20' 2024-05-27T09:31:51.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './282/39' 2024-05-27T09:31:51.709 INFO:tasks.workunit.client.0.smithi114.stdout:removed './282/34' 2024-05-27T09:31:51.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './282/38' 2024-05-27T09:31:51.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './282/28' 2024-05-27T09:31:51.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './282/24' 2024-05-27T09:31:51.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './282/29' 2024-05-27T09:31:51.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './282/32' 2024-05-27T09:31:51.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './282/22' 2024-05-27T09:31:51.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './282/25' 2024-05-27T09:31:51.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './282/30' 2024-05-27T09:31:51.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './282/33' 2024-05-27T09:31:51.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './282/23' 2024-05-27T09:31:51.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './282/35' 2024-05-27T09:31:51.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './282/26' 2024-05-27T09:31:51.710 INFO:tasks.workunit.client.0.smithi114.stdout:removed './282/37' 2024-05-27T09:31:51.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './282' 2024-05-27T09:31:51.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './283/21' 2024-05-27T09:31:51.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './283/27' 2024-05-27T09:31:51.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './283/31' 2024-05-27T09:31:51.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './283/36' 2024-05-27T09:31:51.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './283/20' 2024-05-27T09:31:51.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './283/39' 2024-05-27T09:31:51.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './283/34' 2024-05-27T09:31:51.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './283/38' 2024-05-27T09:31:51.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './283/28' 2024-05-27T09:31:51.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './283/24' 2024-05-27T09:31:51.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './283/29' 2024-05-27T09:31:51.711 INFO:tasks.workunit.client.0.smithi114.stdout:removed './283/32' 2024-05-27T09:31:51.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './283/22' 2024-05-27T09:31:51.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './283/25' 2024-05-27T09:31:51.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './283/30' 2024-05-27T09:31:51.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './283/33' 2024-05-27T09:31:51.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './283/23' 2024-05-27T09:31:51.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './283/35' 2024-05-27T09:31:51.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './283/26' 2024-05-27T09:31:51.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './283/37' 2024-05-27T09:31:51.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './283' 2024-05-27T09:31:51.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './284/21' 2024-05-27T09:31:51.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './284/27' 2024-05-27T09:31:51.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './284/31' 2024-05-27T09:31:51.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './284/36' 2024-05-27T09:31:51.712 INFO:tasks.workunit.client.0.smithi114.stdout:removed './284/20' 2024-05-27T09:31:51.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './284/39' 2024-05-27T09:31:51.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './284/34' 2024-05-27T09:31:51.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './284/38' 2024-05-27T09:31:51.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './284/28' 2024-05-27T09:31:51.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './284/24' 2024-05-27T09:31:51.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './284/29' 2024-05-27T09:31:51.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './284/32' 2024-05-27T09:31:51.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './284/22' 2024-05-27T09:31:51.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './284/25' 2024-05-27T09:31:51.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './284/30' 2024-05-27T09:31:51.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './284/33' 2024-05-27T09:31:51.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './284/23' 2024-05-27T09:31:51.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './284/35' 2024-05-27T09:31:51.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './284/26' 2024-05-27T09:31:51.713 INFO:tasks.workunit.client.0.smithi114.stdout:removed './284/37' 2024-05-27T09:31:51.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './284' 2024-05-27T09:31:51.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './285/21' 2024-05-27T09:31:51.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './285/27' 2024-05-27T09:31:51.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './285/31' 2024-05-27T09:31:51.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './285/36' 2024-05-27T09:31:51.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './285/20' 2024-05-27T09:31:51.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './285/39' 2024-05-27T09:31:51.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './285/34' 2024-05-27T09:31:51.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './285/38' 2024-05-27T09:31:51.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './285/28' 2024-05-27T09:31:51.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './285/24' 2024-05-27T09:31:51.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './285/29' 2024-05-27T09:31:51.714 INFO:tasks.workunit.client.0.smithi114.stdout:removed './285/32' 2024-05-27T09:31:51.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './285/22' 2024-05-27T09:31:51.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './285/25' 2024-05-27T09:31:51.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './285/30' 2024-05-27T09:31:51.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './285/33' 2024-05-27T09:31:51.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './285/23' 2024-05-27T09:31:51.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './285/35' 2024-05-27T09:31:51.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './285/26' 2024-05-27T09:31:51.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './285/37' 2024-05-27T09:31:51.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './285' 2024-05-27T09:31:51.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './286/21' 2024-05-27T09:31:51.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './286/27' 2024-05-27T09:31:51.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './286/31' 2024-05-27T09:31:51.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './286/36' 2024-05-27T09:31:51.715 INFO:tasks.workunit.client.0.smithi114.stdout:removed './286/20' 2024-05-27T09:31:51.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './286/39' 2024-05-27T09:31:51.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './286/34' 2024-05-27T09:31:51.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './286/38' 2024-05-27T09:31:51.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './286/28' 2024-05-27T09:31:51.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './286/24' 2024-05-27T09:31:51.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './286/29' 2024-05-27T09:31:51.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './286/32' 2024-05-27T09:31:51.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './286/22' 2024-05-27T09:31:51.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './286/25' 2024-05-27T09:31:51.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './286/30' 2024-05-27T09:31:51.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './286/33' 2024-05-27T09:31:51.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './286/23' 2024-05-27T09:31:51.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './286/35' 2024-05-27T09:31:51.716 INFO:tasks.workunit.client.0.smithi114.stdout:removed './286/26' 2024-05-27T09:31:51.717 INFO:tasks.workunit.client.0.smithi114.stdout:removed './286/37' 2024-05-27T09:31:51.717 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './286' 2024-05-27T09:31:52.657 INFO:tasks.workunit.client.0.smithi114.stdout:removed './287/21' 2024-05-27T09:31:52.657 INFO:tasks.workunit.client.0.smithi114.stdout:removed './287/27' 2024-05-27T09:31:52.658 INFO:tasks.workunit.client.0.smithi114.stdout:removed './287/31' 2024-05-27T09:31:52.658 INFO:tasks.workunit.client.0.smithi114.stdout:removed './287/36' 2024-05-27T09:31:52.658 INFO:tasks.workunit.client.0.smithi114.stdout:removed './287/20' 2024-05-27T09:31:52.658 INFO:tasks.workunit.client.0.smithi114.stdout:removed './287/39' 2024-05-27T09:31:52.658 INFO:tasks.workunit.client.0.smithi114.stdout:removed './287/34' 2024-05-27T09:31:52.658 INFO:tasks.workunit.client.0.smithi114.stdout:removed './287/38' 2024-05-27T09:31:52.658 INFO:tasks.workunit.client.0.smithi114.stdout:removed './287/28' 2024-05-27T09:31:52.658 INFO:tasks.workunit.client.0.smithi114.stdout:removed './287/24' 2024-05-27T09:31:52.658 INFO:tasks.workunit.client.0.smithi114.stdout:removed './287/29' 2024-05-27T09:31:52.658 INFO:tasks.workunit.client.0.smithi114.stdout:removed './287/32' 2024-05-27T09:31:52.658 INFO:tasks.workunit.client.0.smithi114.stdout:removed './287/22' 2024-05-27T09:31:52.658 INFO:tasks.workunit.client.0.smithi114.stdout:removed './287/25' 2024-05-27T09:31:52.659 INFO:tasks.workunit.client.0.smithi114.stdout:removed './287/30' 2024-05-27T09:31:52.659 INFO:tasks.workunit.client.0.smithi114.stdout:removed './287/33' 2024-05-27T09:31:52.659 INFO:tasks.workunit.client.0.smithi114.stdout:removed './287/23' 2024-05-27T09:31:52.659 INFO:tasks.workunit.client.0.smithi114.stdout:removed './287/35' 2024-05-27T09:31:52.659 INFO:tasks.workunit.client.0.smithi114.stdout:removed './287/26' 2024-05-27T09:31:52.659 INFO:tasks.workunit.client.0.smithi114.stdout:removed './287/37' 2024-05-27T09:31:52.659 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './287' 2024-05-27T09:31:52.659 INFO:tasks.workunit.client.0.smithi114.stdout:removed './288/21' 2024-05-27T09:31:52.659 INFO:tasks.workunit.client.0.smithi114.stdout:removed './288/27' 2024-05-27T09:31:52.659 INFO:tasks.workunit.client.0.smithi114.stdout:removed './288/31' 2024-05-27T09:31:52.659 INFO:tasks.workunit.client.0.smithi114.stdout:removed './288/36' 2024-05-27T09:31:52.659 INFO:tasks.workunit.client.0.smithi114.stdout:removed './288/20' 2024-05-27T09:31:52.659 INFO:tasks.workunit.client.0.smithi114.stdout:removed './288/39' 2024-05-27T09:31:52.659 INFO:tasks.workunit.client.0.smithi114.stdout:removed './288/34' 2024-05-27T09:31:52.659 INFO:tasks.workunit.client.0.smithi114.stdout:removed './288/38' 2024-05-27T09:31:52.660 INFO:tasks.workunit.client.0.smithi114.stdout:removed './288/28' 2024-05-27T09:31:52.660 INFO:tasks.workunit.client.0.smithi114.stdout:removed './288/24' 2024-05-27T09:31:52.660 INFO:tasks.workunit.client.0.smithi114.stdout:removed './288/29' 2024-05-27T09:31:52.660 INFO:tasks.workunit.client.0.smithi114.stdout:removed './288/32' 2024-05-27T09:31:52.660 INFO:tasks.workunit.client.0.smithi114.stdout:removed './288/22' 2024-05-27T09:31:52.660 INFO:tasks.workunit.client.0.smithi114.stdout:removed './288/25' 2024-05-27T09:31:52.660 INFO:tasks.workunit.client.0.smithi114.stdout:removed './288/30' 2024-05-27T09:31:52.660 INFO:tasks.workunit.client.0.smithi114.stdout:removed './288/33' 2024-05-27T09:31:52.660 INFO:tasks.workunit.client.0.smithi114.stdout:removed './288/23' 2024-05-27T09:31:52.660 INFO:tasks.workunit.client.0.smithi114.stdout:removed './288/35' 2024-05-27T09:31:52.660 INFO:tasks.workunit.client.0.smithi114.stdout:removed './288/26' 2024-05-27T09:31:52.660 INFO:tasks.workunit.client.0.smithi114.stdout:removed './288/37' 2024-05-27T09:31:52.660 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './288' 2024-05-27T09:31:52.661 INFO:tasks.workunit.client.0.smithi114.stdout:removed './289/21' 2024-05-27T09:31:52.661 INFO:tasks.workunit.client.0.smithi114.stdout:removed './289/27' 2024-05-27T09:31:52.661 INFO:tasks.workunit.client.0.smithi114.stdout:removed './289/31' 2024-05-27T09:31:52.661 INFO:tasks.workunit.client.0.smithi114.stdout:removed './289/36' 2024-05-27T09:31:52.661 INFO:tasks.workunit.client.0.smithi114.stdout:removed './289/20' 2024-05-27T09:31:52.661 INFO:tasks.workunit.client.0.smithi114.stdout:removed './289/39' 2024-05-27T09:31:52.661 INFO:tasks.workunit.client.0.smithi114.stdout:removed './289/34' 2024-05-27T09:31:52.661 INFO:tasks.workunit.client.0.smithi114.stdout:removed './289/38' 2024-05-27T09:31:52.661 INFO:tasks.workunit.client.0.smithi114.stdout:removed './289/28' 2024-05-27T09:31:52.661 INFO:tasks.workunit.client.0.smithi114.stdout:removed './289/24' 2024-05-27T09:31:52.661 INFO:tasks.workunit.client.0.smithi114.stdout:removed './289/29' 2024-05-27T09:31:52.661 INFO:tasks.workunit.client.0.smithi114.stdout:removed './289/32' 2024-05-27T09:31:52.661 INFO:tasks.workunit.client.0.smithi114.stdout:removed './289/22' 2024-05-27T09:31:52.661 INFO:tasks.workunit.client.0.smithi114.stdout:removed './289/25' 2024-05-27T09:31:52.661 INFO:tasks.workunit.client.0.smithi114.stdout:removed './289/30' 2024-05-27T09:31:52.662 INFO:tasks.workunit.client.0.smithi114.stdout:removed './289/33' 2024-05-27T09:31:52.662 INFO:tasks.workunit.client.0.smithi114.stdout:removed './289/23' 2024-05-27T09:31:52.662 INFO:tasks.workunit.client.0.smithi114.stdout:removed './289/35' 2024-05-27T09:31:52.662 INFO:tasks.workunit.client.0.smithi114.stdout:removed './289/26' 2024-05-27T09:31:52.662 INFO:tasks.workunit.client.0.smithi114.stdout:removed './289/37' 2024-05-27T09:31:52.662 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './289' 2024-05-27T09:31:52.662 INFO:tasks.workunit.client.0.smithi114.stdout:removed './290/21' 2024-05-27T09:31:52.662 INFO:tasks.workunit.client.0.smithi114.stdout:removed './290/27' 2024-05-27T09:31:52.662 INFO:tasks.workunit.client.0.smithi114.stdout:removed './290/31' 2024-05-27T09:31:52.662 INFO:tasks.workunit.client.0.smithi114.stdout:removed './290/36' 2024-05-27T09:31:52.662 INFO:tasks.workunit.client.0.smithi114.stdout:removed './290/20' 2024-05-27T09:31:52.662 INFO:tasks.workunit.client.0.smithi114.stdout:removed './290/39' 2024-05-27T09:31:52.662 INFO:tasks.workunit.client.0.smithi114.stdout:removed './290/34' 2024-05-27T09:31:52.663 INFO:tasks.workunit.client.0.smithi114.stdout:removed './290/38' 2024-05-27T09:31:52.663 INFO:tasks.workunit.client.0.smithi114.stdout:removed './290/28' 2024-05-27T09:31:52.663 INFO:tasks.workunit.client.0.smithi114.stdout:removed './290/24' 2024-05-27T09:31:52.663 INFO:tasks.workunit.client.0.smithi114.stdout:removed './290/29' 2024-05-27T09:31:52.663 INFO:tasks.workunit.client.0.smithi114.stdout:removed './290/32' 2024-05-27T09:31:52.663 INFO:tasks.workunit.client.0.smithi114.stdout:removed './290/22' 2024-05-27T09:31:52.663 INFO:tasks.workunit.client.0.smithi114.stdout:removed './290/25' 2024-05-27T09:31:52.663 INFO:tasks.workunit.client.0.smithi114.stdout:removed './290/30' 2024-05-27T09:31:52.663 INFO:tasks.workunit.client.0.smithi114.stdout:removed './290/33' 2024-05-27T09:31:52.663 INFO:tasks.workunit.client.0.smithi114.stdout:removed './290/23' 2024-05-27T09:31:52.663 INFO:tasks.workunit.client.0.smithi114.stdout:removed './290/35' 2024-05-27T09:31:52.663 INFO:tasks.workunit.client.0.smithi114.stdout:removed './290/26' 2024-05-27T09:31:52.663 INFO:tasks.workunit.client.0.smithi114.stdout:removed './290/37' 2024-05-27T09:31:52.663 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './290' 2024-05-27T09:31:52.663 INFO:tasks.workunit.client.0.smithi114.stdout:removed './291/21' 2024-05-27T09:31:52.664 INFO:tasks.workunit.client.0.smithi114.stdout:removed './291/27' 2024-05-27T09:31:52.664 INFO:tasks.workunit.client.0.smithi114.stdout:removed './291/31' 2024-05-27T09:31:52.664 INFO:tasks.workunit.client.0.smithi114.stdout:removed './291/36' 2024-05-27T09:31:52.664 INFO:tasks.workunit.client.0.smithi114.stdout:removed './291/20' 2024-05-27T09:31:52.664 INFO:tasks.workunit.client.0.smithi114.stdout:removed './291/39' 2024-05-27T09:31:52.664 INFO:tasks.workunit.client.0.smithi114.stdout:removed './291/34' 2024-05-27T09:31:52.664 INFO:tasks.workunit.client.0.smithi114.stdout:removed './291/38' 2024-05-27T09:31:52.664 INFO:tasks.workunit.client.0.smithi114.stdout:removed './291/28' 2024-05-27T09:31:52.664 INFO:tasks.workunit.client.0.smithi114.stdout:removed './291/24' 2024-05-27T09:31:52.664 INFO:tasks.workunit.client.0.smithi114.stdout:removed './291/29' 2024-05-27T09:31:52.664 INFO:tasks.workunit.client.0.smithi114.stdout:removed './291/32' 2024-05-27T09:31:52.664 INFO:tasks.workunit.client.0.smithi114.stdout:removed './291/22' 2024-05-27T09:31:52.664 INFO:tasks.workunit.client.0.smithi114.stdout:removed './291/25' 2024-05-27T09:31:52.664 INFO:tasks.workunit.client.0.smithi114.stdout:removed './291/30' 2024-05-27T09:31:52.665 INFO:tasks.workunit.client.0.smithi114.stdout:removed './291/33' 2024-05-27T09:31:52.665 INFO:tasks.workunit.client.0.smithi114.stdout:removed './291/23' 2024-05-27T09:31:52.665 INFO:tasks.workunit.client.0.smithi114.stdout:removed './291/35' 2024-05-27T09:31:52.665 INFO:tasks.workunit.client.0.smithi114.stdout:removed './291/26' 2024-05-27T09:31:52.665 INFO:tasks.workunit.client.0.smithi114.stdout:removed './291/37' 2024-05-27T09:31:52.665 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './291' 2024-05-27T09:31:52.665 INFO:tasks.workunit.client.0.smithi114.stdout:removed './292/21' 2024-05-27T09:31:52.665 INFO:tasks.workunit.client.0.smithi114.stdout:removed './292/27' 2024-05-27T09:31:52.665 INFO:tasks.workunit.client.0.smithi114.stdout:removed './292/31' 2024-05-27T09:31:52.665 INFO:tasks.workunit.client.0.smithi114.stdout:removed './292/36' 2024-05-27T09:31:52.665 INFO:tasks.workunit.client.0.smithi114.stdout:removed './292/20' 2024-05-27T09:31:52.665 INFO:tasks.workunit.client.0.smithi114.stdout:removed './292/39' 2024-05-27T09:31:52.665 INFO:tasks.workunit.client.0.smithi114.stdout:removed './292/34' 2024-05-27T09:31:52.666 INFO:tasks.workunit.client.0.smithi114.stdout:removed './292/38' 2024-05-27T09:31:52.666 INFO:tasks.workunit.client.0.smithi114.stdout:removed './292/28' 2024-05-27T09:31:52.666 INFO:tasks.workunit.client.0.smithi114.stdout:removed './292/24' 2024-05-27T09:31:52.666 INFO:tasks.workunit.client.0.smithi114.stdout:removed './292/29' 2024-05-27T09:31:52.666 INFO:tasks.workunit.client.0.smithi114.stdout:removed './292/32' 2024-05-27T09:31:52.666 INFO:tasks.workunit.client.0.smithi114.stdout:removed './292/22' 2024-05-27T09:31:52.666 INFO:tasks.workunit.client.0.smithi114.stdout:removed './292/25' 2024-05-27T09:31:52.666 INFO:tasks.workunit.client.0.smithi114.stdout:removed './292/30' 2024-05-27T09:31:52.666 INFO:tasks.workunit.client.0.smithi114.stdout:removed './292/33' 2024-05-27T09:31:52.666 INFO:tasks.workunit.client.0.smithi114.stdout:removed './292/23' 2024-05-27T09:31:52.666 INFO:tasks.workunit.client.0.smithi114.stdout:removed './292/35' 2024-05-27T09:31:52.666 INFO:tasks.workunit.client.0.smithi114.stdout:removed './292/26' 2024-05-27T09:31:52.666 INFO:tasks.workunit.client.0.smithi114.stdout:removed './292/37' 2024-05-27T09:31:52.667 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './292' 2024-05-27T09:31:52.667 INFO:tasks.workunit.client.0.smithi114.stdout:removed './293/21' 2024-05-27T09:31:52.667 INFO:tasks.workunit.client.0.smithi114.stdout:removed './293/27' 2024-05-27T09:31:52.667 INFO:tasks.workunit.client.0.smithi114.stdout:removed './293/31' 2024-05-27T09:31:52.667 INFO:tasks.workunit.client.0.smithi114.stdout:removed './293/36' 2024-05-27T09:31:52.667 INFO:tasks.workunit.client.0.smithi114.stdout:removed './293/20' 2024-05-27T09:31:52.667 INFO:tasks.workunit.client.0.smithi114.stdout:removed './293/39' 2024-05-27T09:31:52.667 INFO:tasks.workunit.client.0.smithi114.stdout:removed './293/34' 2024-05-27T09:31:52.667 INFO:tasks.workunit.client.0.smithi114.stdout:removed './293/38' 2024-05-27T09:31:52.667 INFO:tasks.workunit.client.0.smithi114.stdout:removed './293/28' 2024-05-27T09:31:52.667 INFO:tasks.workunit.client.0.smithi114.stdout:removed './293/24' 2024-05-27T09:31:52.668 INFO:tasks.workunit.client.0.smithi114.stdout:removed './293/29' 2024-05-27T09:31:52.668 INFO:tasks.workunit.client.0.smithi114.stdout:removed './293/32' 2024-05-27T09:31:52.668 INFO:tasks.workunit.client.0.smithi114.stdout:removed './293/22' 2024-05-27T09:31:52.668 INFO:tasks.workunit.client.0.smithi114.stdout:removed './293/25' 2024-05-27T09:31:52.668 INFO:tasks.workunit.client.0.smithi114.stdout:removed './293/30' 2024-05-27T09:31:52.668 INFO:tasks.workunit.client.0.smithi114.stdout:removed './293/33' 2024-05-27T09:31:52.668 INFO:tasks.workunit.client.0.smithi114.stdout:removed './293/23' 2024-05-27T09:31:52.668 INFO:tasks.workunit.client.0.smithi114.stdout:removed './293/35' 2024-05-27T09:31:52.668 INFO:tasks.workunit.client.0.smithi114.stdout:removed './293/26' 2024-05-27T09:31:52.668 INFO:tasks.workunit.client.0.smithi114.stdout:removed './293/37' 2024-05-27T09:31:52.668 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './293' 2024-05-27T09:31:52.668 INFO:tasks.workunit.client.0.smithi114.stdout:removed './294/21' 2024-05-27T09:31:52.668 INFO:tasks.workunit.client.0.smithi114.stdout:removed './294/27' 2024-05-27T09:31:52.668 INFO:tasks.workunit.client.0.smithi114.stdout:removed './294/31' 2024-05-27T09:31:52.669 INFO:tasks.workunit.client.0.smithi114.stdout:removed './294/36' 2024-05-27T09:31:52.669 INFO:tasks.workunit.client.0.smithi114.stdout:removed './294/20' 2024-05-27T09:31:52.669 INFO:tasks.workunit.client.0.smithi114.stdout:removed './294/39' 2024-05-27T09:31:52.669 INFO:tasks.workunit.client.0.smithi114.stdout:removed './294/34' 2024-05-27T09:31:52.669 INFO:tasks.workunit.client.0.smithi114.stdout:removed './294/38' 2024-05-27T09:31:52.669 INFO:tasks.workunit.client.0.smithi114.stdout:removed './294/28' 2024-05-27T09:31:52.669 INFO:tasks.workunit.client.0.smithi114.stdout:removed './294/24' 2024-05-27T09:31:52.669 INFO:tasks.workunit.client.0.smithi114.stdout:removed './294/29' 2024-05-27T09:31:52.669 INFO:tasks.workunit.client.0.smithi114.stdout:removed './294/32' 2024-05-27T09:31:52.669 INFO:tasks.workunit.client.0.smithi114.stdout:removed './294/22' 2024-05-27T09:31:52.669 INFO:tasks.workunit.client.0.smithi114.stdout:removed './294/25' 2024-05-27T09:31:52.669 INFO:tasks.workunit.client.0.smithi114.stdout:removed './294/30' 2024-05-27T09:31:52.669 INFO:tasks.workunit.client.0.smithi114.stdout:removed './294/33' 2024-05-27T09:31:52.670 INFO:tasks.workunit.client.0.smithi114.stdout:removed './294/23' 2024-05-27T09:31:52.670 INFO:tasks.workunit.client.0.smithi114.stdout:removed './294/35' 2024-05-27T09:31:52.670 INFO:tasks.workunit.client.0.smithi114.stdout:removed './294/26' 2024-05-27T09:31:52.670 INFO:tasks.workunit.client.0.smithi114.stdout:removed './294/37' 2024-05-27T09:31:52.670 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './294' 2024-05-27T09:31:52.670 INFO:tasks.workunit.client.0.smithi114.stdout:removed './295/21' 2024-05-27T09:31:52.670 INFO:tasks.workunit.client.0.smithi114.stdout:removed './295/27' 2024-05-27T09:31:52.670 INFO:tasks.workunit.client.0.smithi114.stdout:removed './295/31' 2024-05-27T09:31:52.670 INFO:tasks.workunit.client.0.smithi114.stdout:removed './295/36' 2024-05-27T09:31:52.670 INFO:tasks.workunit.client.0.smithi114.stdout:removed './295/20' 2024-05-27T09:31:52.670 INFO:tasks.workunit.client.0.smithi114.stdout:removed './295/39' 2024-05-27T09:31:52.670 INFO:tasks.workunit.client.0.smithi114.stdout:removed './295/34' 2024-05-27T09:31:52.670 INFO:tasks.workunit.client.0.smithi114.stdout:removed './295/38' 2024-05-27T09:31:52.671 INFO:tasks.workunit.client.0.smithi114.stdout:removed './295/28' 2024-05-27T09:31:52.671 INFO:tasks.workunit.client.0.smithi114.stdout:removed './295/24' 2024-05-27T09:31:52.671 INFO:tasks.workunit.client.0.smithi114.stdout:removed './295/29' 2024-05-27T09:31:52.671 INFO:tasks.workunit.client.0.smithi114.stdout:removed './295/32' 2024-05-27T09:31:52.671 INFO:tasks.workunit.client.0.smithi114.stdout:removed './295/22' 2024-05-27T09:31:52.671 INFO:tasks.workunit.client.0.smithi114.stdout:removed './295/25' 2024-05-27T09:31:52.671 INFO:tasks.workunit.client.0.smithi114.stdout:removed './295/30' 2024-05-27T09:31:52.671 INFO:tasks.workunit.client.0.smithi114.stdout:removed './295/33' 2024-05-27T09:31:52.671 INFO:tasks.workunit.client.0.smithi114.stdout:removed './295/23' 2024-05-27T09:31:52.671 INFO:tasks.workunit.client.0.smithi114.stdout:removed './295/35' 2024-05-27T09:31:52.671 INFO:tasks.workunit.client.0.smithi114.stdout:removed './295/26' 2024-05-27T09:31:52.671 INFO:tasks.workunit.client.0.smithi114.stdout:removed './295/37' 2024-05-27T09:31:52.671 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './295' 2024-05-27T09:31:52.671 INFO:tasks.workunit.client.0.smithi114.stdout:removed './296/21' 2024-05-27T09:31:52.672 INFO:tasks.workunit.client.0.smithi114.stdout:removed './296/27' 2024-05-27T09:31:52.672 INFO:tasks.workunit.client.0.smithi114.stdout:removed './296/31' 2024-05-27T09:31:52.672 INFO:tasks.workunit.client.0.smithi114.stdout:removed './296/36' 2024-05-27T09:31:52.672 INFO:tasks.workunit.client.0.smithi114.stdout:removed './296/20' 2024-05-27T09:31:52.672 INFO:tasks.workunit.client.0.smithi114.stdout:removed './296/39' 2024-05-27T09:31:52.672 INFO:tasks.workunit.client.0.smithi114.stdout:removed './296/34' 2024-05-27T09:31:52.672 INFO:tasks.workunit.client.0.smithi114.stdout:removed './296/38' 2024-05-27T09:31:52.672 INFO:tasks.workunit.client.0.smithi114.stdout:removed './296/28' 2024-05-27T09:31:52.672 INFO:tasks.workunit.client.0.smithi114.stdout:removed './296/24' 2024-05-27T09:31:52.672 INFO:tasks.workunit.client.0.smithi114.stdout:removed './296/29' 2024-05-27T09:31:52.672 INFO:tasks.workunit.client.0.smithi114.stdout:removed './296/32' 2024-05-27T09:31:52.672 INFO:tasks.workunit.client.0.smithi114.stdout:removed './296/22' 2024-05-27T09:31:52.673 INFO:tasks.workunit.client.0.smithi114.stdout:removed './296/25' 2024-05-27T09:31:52.673 INFO:tasks.workunit.client.0.smithi114.stdout:removed './296/30' 2024-05-27T09:31:52.673 INFO:tasks.workunit.client.0.smithi114.stdout:removed './296/33' 2024-05-27T09:31:52.673 INFO:tasks.workunit.client.0.smithi114.stdout:removed './296/23' 2024-05-27T09:31:52.673 INFO:tasks.workunit.client.0.smithi114.stdout:removed './296/35' 2024-05-27T09:31:52.673 INFO:tasks.workunit.client.0.smithi114.stdout:removed './296/26' 2024-05-27T09:31:52.673 INFO:tasks.workunit.client.0.smithi114.stdout:removed './296/37' 2024-05-27T09:31:52.673 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './296' 2024-05-27T09:31:52.673 INFO:tasks.workunit.client.0.smithi114.stdout:removed './297/21' 2024-05-27T09:31:52.673 INFO:tasks.workunit.client.0.smithi114.stdout:removed './297/27' 2024-05-27T09:31:53.433 INFO:tasks.workunit.client.0.smithi114.stdout:removed './297/31' 2024-05-27T09:31:53.433 INFO:tasks.workunit.client.0.smithi114.stdout:removed './297/36' 2024-05-27T09:31:53.433 INFO:tasks.workunit.client.0.smithi114.stdout:removed './297/20' 2024-05-27T09:31:53.433 INFO:tasks.workunit.client.0.smithi114.stdout:removed './297/39' 2024-05-27T09:31:53.433 INFO:tasks.workunit.client.0.smithi114.stdout:removed './297/34' 2024-05-27T09:31:53.433 INFO:tasks.workunit.client.0.smithi114.stdout:removed './297/38' 2024-05-27T09:31:53.433 INFO:tasks.workunit.client.0.smithi114.stdout:removed './297/28' 2024-05-27T09:31:53.434 INFO:tasks.workunit.client.0.smithi114.stdout:removed './297/24' 2024-05-27T09:31:53.434 INFO:tasks.workunit.client.0.smithi114.stdout:removed './297/29' 2024-05-27T09:31:53.434 INFO:tasks.workunit.client.0.smithi114.stdout:removed './297/32' 2024-05-27T09:31:53.434 INFO:tasks.workunit.client.0.smithi114.stdout:removed './297/22' 2024-05-27T09:31:53.434 INFO:tasks.workunit.client.0.smithi114.stdout:removed './297/25' 2024-05-27T09:31:53.434 INFO:tasks.workunit.client.0.smithi114.stdout:removed './297/30' 2024-05-27T09:31:53.434 INFO:tasks.workunit.client.0.smithi114.stdout:removed './297/33' 2024-05-27T09:31:53.434 INFO:tasks.workunit.client.0.smithi114.stdout:removed './297/23' 2024-05-27T09:31:53.434 INFO:tasks.workunit.client.0.smithi114.stdout:removed './297/35' 2024-05-27T09:31:53.434 INFO:tasks.workunit.client.0.smithi114.stdout:removed './297/26' 2024-05-27T09:31:53.434 INFO:tasks.workunit.client.0.smithi114.stdout:removed './297/37' 2024-05-27T09:31:53.434 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './297' 2024-05-27T09:31:53.434 INFO:tasks.workunit.client.0.smithi114.stdout:removed './298/21' 2024-05-27T09:31:53.435 INFO:tasks.workunit.client.0.smithi114.stdout:removed './298/27' 2024-05-27T09:31:53.435 INFO:tasks.workunit.client.0.smithi114.stdout:removed './298/31' 2024-05-27T09:31:53.435 INFO:tasks.workunit.client.0.smithi114.stdout:removed './298/36' 2024-05-27T09:31:53.435 INFO:tasks.workunit.client.0.smithi114.stdout:removed './298/20' 2024-05-27T09:31:53.435 INFO:tasks.workunit.client.0.smithi114.stdout:removed './298/39' 2024-05-27T09:31:53.435 INFO:tasks.workunit.client.0.smithi114.stdout:removed './298/34' 2024-05-27T09:31:53.435 INFO:tasks.workunit.client.0.smithi114.stdout:removed './298/38' 2024-05-27T09:31:53.435 INFO:tasks.workunit.client.0.smithi114.stdout:removed './298/28' 2024-05-27T09:31:53.435 INFO:tasks.workunit.client.0.smithi114.stdout:removed './298/24' 2024-05-27T09:31:53.435 INFO:tasks.workunit.client.0.smithi114.stdout:removed './298/29' 2024-05-27T09:31:53.435 INFO:tasks.workunit.client.0.smithi114.stdout:removed './298/32' 2024-05-27T09:31:53.435 INFO:tasks.workunit.client.0.smithi114.stdout:removed './298/22' 2024-05-27T09:31:53.435 INFO:tasks.workunit.client.0.smithi114.stdout:removed './298/25' 2024-05-27T09:31:53.436 INFO:tasks.workunit.client.0.smithi114.stdout:removed './298/30' 2024-05-27T09:31:53.436 INFO:tasks.workunit.client.0.smithi114.stdout:removed './298/33' 2024-05-27T09:31:53.436 INFO:tasks.workunit.client.0.smithi114.stdout:removed './298/23' 2024-05-27T09:31:53.436 INFO:tasks.workunit.client.0.smithi114.stdout:removed './298/35' 2024-05-27T09:31:53.436 INFO:tasks.workunit.client.0.smithi114.stdout:removed './298/26' 2024-05-27T09:31:53.436 INFO:tasks.workunit.client.0.smithi114.stdout:removed './298/37' 2024-05-27T09:31:53.436 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './298' 2024-05-27T09:31:53.436 INFO:tasks.workunit.client.0.smithi114.stdout:removed './299/21' 2024-05-27T09:31:53.436 INFO:tasks.workunit.client.0.smithi114.stdout:removed './299/27' 2024-05-27T09:31:53.436 INFO:tasks.workunit.client.0.smithi114.stdout:removed './299/31' 2024-05-27T09:31:53.436 INFO:tasks.workunit.client.0.smithi114.stdout:removed './299/36' 2024-05-27T09:31:53.436 INFO:tasks.workunit.client.0.smithi114.stdout:removed './299/20' 2024-05-27T09:31:53.436 INFO:tasks.workunit.client.0.smithi114.stdout:removed './299/39' 2024-05-27T09:31:53.436 INFO:tasks.workunit.client.0.smithi114.stdout:removed './299/34' 2024-05-27T09:31:53.436 INFO:tasks.workunit.client.0.smithi114.stdout:removed './299/38' 2024-05-27T09:31:53.437 INFO:tasks.workunit.client.0.smithi114.stdout:removed './299/28' 2024-05-27T09:31:53.437 INFO:tasks.workunit.client.0.smithi114.stdout:removed './299/24' 2024-05-27T09:31:53.437 INFO:tasks.workunit.client.0.smithi114.stdout:removed './299/29' 2024-05-27T09:31:53.437 INFO:tasks.workunit.client.0.smithi114.stdout:removed './299/32' 2024-05-27T09:31:53.437 INFO:tasks.workunit.client.0.smithi114.stdout:removed './299/22' 2024-05-27T09:31:53.437 INFO:tasks.workunit.client.0.smithi114.stdout:removed './299/25' 2024-05-27T09:31:53.437 INFO:tasks.workunit.client.0.smithi114.stdout:removed './299/30' 2024-05-27T09:31:53.437 INFO:tasks.workunit.client.0.smithi114.stdout:removed './299/33' 2024-05-27T09:31:53.437 INFO:tasks.workunit.client.0.smithi114.stdout:removed './299/23' 2024-05-27T09:31:53.437 INFO:tasks.workunit.client.0.smithi114.stdout:removed './299/35' 2024-05-27T09:31:53.437 INFO:tasks.workunit.client.0.smithi114.stdout:removed './299/26' 2024-05-27T09:31:53.437 INFO:tasks.workunit.client.0.smithi114.stdout:removed './299/37' 2024-05-27T09:31:53.437 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './299' 2024-05-27T09:31:53.437 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/166/19' 2024-05-27T09:31:53.437 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/166/15' 2024-05-27T09:31:53.438 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/166/20' 2024-05-27T09:31:53.438 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/166/12' 2024-05-27T09:31:53.438 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/166/10' 2024-05-27T09:31:53.438 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/166/11' 2024-05-27T09:31:53.438 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/166/16' 2024-05-27T09:31:53.438 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/166/14' 2024-05-27T09:31:53.438 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/166/18' 2024-05-27T09:31:53.438 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/166/13' 2024-05-27T09:31:53.438 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/166/17' 2024-05-27T09:31:53.438 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/166' 2024-05-27T09:31:53.438 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/147/19' 2024-05-27T09:31:53.438 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/147/15' 2024-05-27T09:31:53.438 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/147/20' 2024-05-27T09:31:53.439 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/147/12' 2024-05-27T09:31:53.439 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/147/10' 2024-05-27T09:31:53.439 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/147/11' 2024-05-27T09:31:53.439 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/147/16' 2024-05-27T09:31:53.439 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/147/14' 2024-05-27T09:31:53.439 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/147/18' 2024-05-27T09:31:53.439 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/147/13' 2024-05-27T09:31:53.439 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/147/17' 2024-05-27T09:31:53.439 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/147' 2024-05-27T09:31:53.439 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/151/19' 2024-05-27T09:31:53.439 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/151/15' 2024-05-27T09:31:53.439 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/151/20' 2024-05-27T09:31:53.439 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/151/12' 2024-05-27T09:31:53.440 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/151/10' 2024-05-27T09:31:53.440 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/151/11' 2024-05-27T09:31:53.440 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/151/16' 2024-05-27T09:31:53.440 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/151/14' 2024-05-27T09:31:53.440 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/151/18' 2024-05-27T09:31:53.440 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/151/13' 2024-05-27T09:31:53.440 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/151/17' 2024-05-27T09:31:53.440 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/151' 2024-05-27T09:31:53.440 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/170/19' 2024-05-27T09:31:53.440 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/170/15' 2024-05-27T09:31:53.440 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/170/20' 2024-05-27T09:31:53.440 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/170/12' 2024-05-27T09:31:53.440 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/170/10' 2024-05-27T09:31:53.440 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/170/11' 2024-05-27T09:31:53.441 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/170/16' 2024-05-27T09:31:53.441 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/170/14' 2024-05-27T09:31:53.441 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/170/18' 2024-05-27T09:31:53.441 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/170/13' 2024-05-27T09:31:53.441 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/170/17' 2024-05-27T09:31:53.441 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/170' 2024-05-27T09:31:53.441 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/167/19' 2024-05-27T09:31:53.441 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/167/15' 2024-05-27T09:31:53.441 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/167/20' 2024-05-27T09:31:53.441 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/167/12' 2024-05-27T09:31:53.441 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/167/10' 2024-05-27T09:31:53.441 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/167/11' 2024-05-27T09:31:53.441 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/167/16' 2024-05-27T09:31:53.441 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/167/14' 2024-05-27T09:31:53.441 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/167/18' 2024-05-27T09:31:53.442 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/167/13' 2024-05-27T09:31:53.442 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/167/17' 2024-05-27T09:31:53.442 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/167' 2024-05-27T09:31:53.442 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/140/19' 2024-05-27T09:31:53.442 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/140/15' 2024-05-27T09:31:53.442 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/140/20' 2024-05-27T09:31:53.442 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/140/12' 2024-05-27T09:31:53.442 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/140/10' 2024-05-27T09:31:53.442 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/140/11' 2024-05-27T09:31:53.442 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/140/16' 2024-05-27T09:31:53.442 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/140/14' 2024-05-27T09:31:53.442 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/140/18' 2024-05-27T09:31:53.442 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/140/13' 2024-05-27T09:31:53.443 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/140/17' 2024-05-27T09:31:53.443 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/140' 2024-05-27T09:31:53.443 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/142/19' 2024-05-27T09:31:53.443 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/142/15' 2024-05-27T09:31:53.443 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/142/20' 2024-05-27T09:31:53.443 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/142/12' 2024-05-27T09:31:53.443 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/142/10' 2024-05-27T09:31:53.810 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/142/11' 2024-05-27T09:31:53.810 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/142/16' 2024-05-27T09:31:53.810 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/142/14' 2024-05-27T09:31:53.810 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/142/18' 2024-05-27T09:31:53.810 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/142/13' 2024-05-27T09:31:53.810 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/142/17' 2024-05-27T09:31:53.810 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/142' 2024-05-27T09:31:53.810 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/150/19' 2024-05-27T09:31:53.810 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/150/15' 2024-05-27T09:31:53.810 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/150/20' 2024-05-27T09:31:53.811 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/150/12' 2024-05-27T09:31:53.811 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/150/10' 2024-05-27T09:31:53.811 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/150/11' 2024-05-27T09:31:53.811 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/150/16' 2024-05-27T09:31:53.811 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/150/14' 2024-05-27T09:31:53.811 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/150/18' 2024-05-27T09:31:53.811 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/150/13' 2024-05-27T09:31:53.811 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/150/17' 2024-05-27T09:31:53.811 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/150' 2024-05-27T09:31:53.811 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/105/19' 2024-05-27T09:31:53.811 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/105/15' 2024-05-27T09:31:53.811 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/105/20' 2024-05-27T09:31:53.811 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/105/12' 2024-05-27T09:31:53.812 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/105/10' 2024-05-27T09:31:53.812 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/105/11' 2024-05-27T09:31:53.812 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/105/16' 2024-05-27T09:31:53.812 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/105/14' 2024-05-27T09:31:53.812 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/105/18' 2024-05-27T09:31:53.812 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/105/13' 2024-05-27T09:31:53.812 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/105/17' 2024-05-27T09:31:53.812 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/105' 2024-05-27T09:31:53.812 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/149/19' 2024-05-27T09:31:53.812 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/149/15' 2024-05-27T09:31:53.812 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/149/20' 2024-05-27T09:31:53.812 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/149/12' 2024-05-27T09:31:53.812 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/149/10' 2024-05-27T09:31:53.812 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/149/11' 2024-05-27T09:31:53.813 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/149/16' 2024-05-27T09:31:53.813 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/149/14' 2024-05-27T09:31:53.813 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/149/18' 2024-05-27T09:31:53.813 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/149/13' 2024-05-27T09:31:53.813 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/149/17' 2024-05-27T09:31:53.813 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/149' 2024-05-27T09:31:53.813 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/109/19' 2024-05-27T09:31:53.813 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/109/15' 2024-05-27T09:31:53.813 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/109/20' 2024-05-27T09:31:53.813 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/109/12' 2024-05-27T09:31:53.813 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/109/10' 2024-05-27T09:31:53.813 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/109/11' 2024-05-27T09:31:53.813 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/109/16' 2024-05-27T09:31:53.814 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/109/14' 2024-05-27T09:31:53.814 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/109/18' 2024-05-27T09:31:53.814 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/109/13' 2024-05-27T09:31:53.814 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/109/17' 2024-05-27T09:31:53.814 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/109' 2024-05-27T09:31:53.814 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/158/19' 2024-05-27T09:31:53.814 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/158/15' 2024-05-27T09:31:53.814 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/158/20' 2024-05-27T09:31:53.814 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/158/12' 2024-05-27T09:31:53.814 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/158/10' 2024-05-27T09:31:53.814 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/158/11' 2024-05-27T09:31:53.814 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/158/16' 2024-05-27T09:31:53.814 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/158/14' 2024-05-27T09:31:53.815 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/158/18' 2024-05-27T09:31:53.815 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/158/13' 2024-05-27T09:31:53.815 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/158/17' 2024-05-27T09:31:53.815 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/158' 2024-05-27T09:31:53.815 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/106/19' 2024-05-27T09:31:53.815 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/106/15' 2024-05-27T09:31:53.815 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/106/20' 2024-05-27T09:31:53.815 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/106/12' 2024-05-27T09:31:53.815 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/106/10' 2024-05-27T09:31:53.815 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/106/11' 2024-05-27T09:31:53.815 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/106/16' 2024-05-27T09:31:53.815 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/106/14' 2024-05-27T09:31:53.815 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/106/18' 2024-05-27T09:31:53.816 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/106/13' 2024-05-27T09:31:53.816 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/106/17' 2024-05-27T09:31:53.816 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/106' 2024-05-27T09:31:53.816 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/189/19' 2024-05-27T09:31:53.816 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/189/15' 2024-05-27T09:31:53.816 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/189/20' 2024-05-27T09:31:53.816 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/189/12' 2024-05-27T09:31:53.816 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/189/10' 2024-05-27T09:31:53.816 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/189/11' 2024-05-27T09:31:53.816 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/189/16' 2024-05-27T09:31:53.816 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/189/14' 2024-05-27T09:31:53.816 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/189/18' 2024-05-27T09:31:53.816 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/189/13' 2024-05-27T09:31:53.816 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/189/17' 2024-05-27T09:31:53.817 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/189' 2024-05-27T09:31:53.817 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/133/19' 2024-05-27T09:31:53.817 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/133/15' 2024-05-27T09:31:53.817 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/133/20' 2024-05-27T09:31:53.817 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/133/12' 2024-05-27T09:31:53.817 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/133/10' 2024-05-27T09:31:53.817 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/133/11' 2024-05-27T09:31:53.817 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/133/16' 2024-05-27T09:31:53.817 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/133/14' 2024-05-27T09:31:53.817 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/133/18' 2024-05-27T09:31:53.817 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/133/13' 2024-05-27T09:31:53.817 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/133/17' 2024-05-27T09:31:53.817 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/133' 2024-05-27T09:31:53.818 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/173/19' 2024-05-27T09:31:53.818 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/173/15' 2024-05-27T09:31:53.818 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/173/20' 2024-05-27T09:31:53.818 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/173/12' 2024-05-27T09:31:53.818 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/173/10' 2024-05-27T09:31:53.818 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/173/11' 2024-05-27T09:31:54.414 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/173/16' 2024-05-27T09:31:54.414 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/173/14' 2024-05-27T09:31:54.414 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/173/18' 2024-05-27T09:31:54.414 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/173/13' 2024-05-27T09:31:54.414 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/173/17' 2024-05-27T09:31:54.415 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/173' 2024-05-27T09:31:54.415 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/174/19' 2024-05-27T09:31:54.415 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/174/15' 2024-05-27T09:31:54.415 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/174/20' 2024-05-27T09:31:54.415 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/174/12' 2024-05-27T09:31:54.416 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/174/10' 2024-05-27T09:31:54.416 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/174/11' 2024-05-27T09:31:54.416 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/174/16' 2024-05-27T09:31:54.416 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/174/14' 2024-05-27T09:31:54.416 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/174/18' 2024-05-27T09:31:54.416 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/174/13' 2024-05-27T09:31:54.417 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/174/17' 2024-05-27T09:31:54.417 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/174' 2024-05-27T09:31:54.417 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/155/19' 2024-05-27T09:31:54.417 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/155/15' 2024-05-27T09:31:54.417 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/155/20' 2024-05-27T09:31:54.417 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/155/12' 2024-05-27T09:31:54.418 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/155/10' 2024-05-27T09:31:54.418 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/155/11' 2024-05-27T09:31:54.418 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/155/16' 2024-05-27T09:31:54.418 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/155/14' 2024-05-27T09:31:54.418 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/155/18' 2024-05-27T09:31:54.419 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/155/13' 2024-05-27T09:31:54.419 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/155/17' 2024-05-27T09:31:54.419 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/155' 2024-05-27T09:31:54.419 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/114/19' 2024-05-27T09:31:54.419 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/114/15' 2024-05-27T09:31:54.420 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/114/20' 2024-05-27T09:31:54.420 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/114/12' 2024-05-27T09:31:54.420 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/114/10' 2024-05-27T09:31:54.420 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/114/11' 2024-05-27T09:31:54.420 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/114/16' 2024-05-27T09:31:54.420 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/114/14' 2024-05-27T09:31:54.421 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/114/18' 2024-05-27T09:31:54.421 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/114/13' 2024-05-27T09:31:54.421 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/114/17' 2024-05-27T09:31:54.421 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/114' 2024-05-27T09:31:54.421 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/159/19' 2024-05-27T09:31:54.421 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/159/15' 2024-05-27T09:31:54.422 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/159/20' 2024-05-27T09:31:54.422 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/159/12' 2024-05-27T09:31:54.422 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/159/10' 2024-05-27T09:31:54.422 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/159/11' 2024-05-27T09:31:54.422 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/159/16' 2024-05-27T09:31:54.423 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/159/14' 2024-05-27T09:31:54.423 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/159/18' 2024-05-27T09:31:54.423 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/159/13' 2024-05-27T09:31:54.423 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/159/17' 2024-05-27T09:31:54.423 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/159' 2024-05-27T09:31:54.423 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/187/19' 2024-05-27T09:31:54.424 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/187/15' 2024-05-27T09:31:54.424 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/187/20' 2024-05-27T09:31:54.424 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/187/12' 2024-05-27T09:31:54.424 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/187/10' 2024-05-27T09:31:54.424 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/187/11' 2024-05-27T09:31:54.425 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/187/16' 2024-05-27T09:31:54.425 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/187/14' 2024-05-27T09:31:54.425 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/187/18' 2024-05-27T09:31:54.425 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/187/13' 2024-05-27T09:31:54.425 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/187/17' 2024-05-27T09:31:54.425 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/187' 2024-05-27T09:31:54.426 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/185/19' 2024-05-27T09:31:54.426 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/185/15' 2024-05-27T09:31:54.426 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/185/20' 2024-05-27T09:31:54.426 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/185/12' 2024-05-27T09:31:54.426 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/185/10' 2024-05-27T09:31:54.427 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/185/11' 2024-05-27T09:31:54.427 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/185/16' 2024-05-27T09:31:54.427 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/185/14' 2024-05-27T09:31:54.427 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/185/18' 2024-05-27T09:31:54.427 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/185/13' 2024-05-27T09:31:54.427 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/185/17' 2024-05-27T09:31:54.428 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/185' 2024-05-27T09:31:54.428 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/120/19' 2024-05-27T09:31:54.428 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/120/15' 2024-05-27T09:31:54.428 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/120/20' 2024-05-27T09:31:54.428 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/120/12' 2024-05-27T09:31:54.429 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/120/10' 2024-05-27T09:31:54.429 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/120/11' 2024-05-27T09:31:54.429 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/120/16' 2024-05-27T09:31:54.429 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/120/14' 2024-05-27T09:31:54.429 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/120/18' 2024-05-27T09:31:54.429 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/120/13' 2024-05-27T09:31:54.429 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/120/17' 2024-05-27T09:31:54.429 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/120' 2024-05-27T09:31:54.429 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/110/19' 2024-05-27T09:31:54.430 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/110/15' 2024-05-27T09:31:54.430 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/110/20' 2024-05-27T09:31:54.430 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/110/12' 2024-05-27T09:31:54.430 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/110/10' 2024-05-27T09:31:54.430 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/110/11' 2024-05-27T09:31:54.430 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/110/16' 2024-05-27T09:31:54.430 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/110/14' 2024-05-27T09:31:54.431 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/110/18' 2024-05-27T09:31:54.431 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/110/13' 2024-05-27T09:31:54.431 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/110/17' 2024-05-27T09:31:54.431 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/110' 2024-05-27T09:31:54.431 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/137/19' 2024-05-27T09:31:54.431 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/137/15' 2024-05-27T09:31:54.431 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/137/20' 2024-05-27T09:31:54.431 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/137/12' 2024-05-27T09:31:54.431 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/137/10' 2024-05-27T09:31:54.431 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/137/11' 2024-05-27T09:31:54.431 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/137/16' 2024-05-27T09:31:54.910 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/137/14' 2024-05-27T09:31:54.910 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/137/18' 2024-05-27T09:31:54.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/137/13' 2024-05-27T09:31:54.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/137/17' 2024-05-27T09:31:54.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/137' 2024-05-27T09:31:54.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/198/19' 2024-05-27T09:31:54.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/198/15' 2024-05-27T09:31:54.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/198/20' 2024-05-27T09:31:54.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/198/12' 2024-05-27T09:31:54.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/198/10' 2024-05-27T09:31:54.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/198/11' 2024-05-27T09:31:54.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/198/16' 2024-05-27T09:31:54.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/198/14' 2024-05-27T09:31:54.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/198/18' 2024-05-27T09:31:54.911 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/198/13' 2024-05-27T09:31:54.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/198/17' 2024-05-27T09:31:54.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/198' 2024-05-27T09:31:54.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/136/19' 2024-05-27T09:31:54.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/136/15' 2024-05-27T09:31:54.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/136/20' 2024-05-27T09:31:54.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/136/12' 2024-05-27T09:31:54.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/136/10' 2024-05-27T09:31:54.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/136/11' 2024-05-27T09:31:54.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/136/16' 2024-05-27T09:31:54.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/136/14' 2024-05-27T09:31:54.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/136/18' 2024-05-27T09:31:54.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/136/13' 2024-05-27T09:31:54.912 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/136/17' 2024-05-27T09:31:54.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/136' 2024-05-27T09:31:54.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/125/19' 2024-05-27T09:31:54.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/125/15' 2024-05-27T09:31:54.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/125/20' 2024-05-27T09:31:54.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/125/12' 2024-05-27T09:31:54.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/125/10' 2024-05-27T09:31:54.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/125/11' 2024-05-27T09:31:54.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/125/16' 2024-05-27T09:31:54.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/125/14' 2024-05-27T09:31:54.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/125/18' 2024-05-27T09:31:54.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/125/13' 2024-05-27T09:31:54.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/125/17' 2024-05-27T09:31:54.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/125' 2024-05-27T09:31:54.913 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/194/19' 2024-05-27T09:31:54.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/194/15' 2024-05-27T09:31:54.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/194/20' 2024-05-27T09:31:54.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/194/12' 2024-05-27T09:31:54.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/194/10' 2024-05-27T09:31:54.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/194/11' 2024-05-27T09:31:54.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/194/16' 2024-05-27T09:31:54.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/194/14' 2024-05-27T09:31:54.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/194/18' 2024-05-27T09:31:54.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/194/13' 2024-05-27T09:31:54.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/194/17' 2024-05-27T09:31:54.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/194' 2024-05-27T09:31:54.914 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/119/19' 2024-05-27T09:31:54.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/119/15' 2024-05-27T09:31:54.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/119/20' 2024-05-27T09:31:54.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/119/12' 2024-05-27T09:31:54.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/119/10' 2024-05-27T09:31:54.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/119/11' 2024-05-27T09:31:54.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/119/16' 2024-05-27T09:31:54.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/119/14' 2024-05-27T09:31:54.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/119/18' 2024-05-27T09:31:54.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/119/13' 2024-05-27T09:31:54.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/119/17' 2024-05-27T09:31:54.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/119' 2024-05-27T09:31:54.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/157/19' 2024-05-27T09:31:54.915 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/157/15' 2024-05-27T09:31:54.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/157/20' 2024-05-27T09:31:54.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/157/12' 2024-05-27T09:31:54.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/157/10' 2024-05-27T09:31:54.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/157/11' 2024-05-27T09:31:54.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/157/16' 2024-05-27T09:31:54.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/157/14' 2024-05-27T09:31:54.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/157/18' 2024-05-27T09:31:54.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/157/13' 2024-05-27T09:31:54.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/157/17' 2024-05-27T09:31:54.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/157' 2024-05-27T09:31:54.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/172/19' 2024-05-27T09:31:54.916 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/172/15' 2024-05-27T09:31:54.917 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/172/20' 2024-05-27T09:31:54.917 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/172/12' 2024-05-27T09:31:54.917 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/172/10' 2024-05-27T09:31:54.917 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/172/11' 2024-05-27T09:31:54.917 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/172/16' 2024-05-27T09:31:54.917 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/172/14' 2024-05-27T09:31:54.917 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/172/18' 2024-05-27T09:31:54.917 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/172/13' 2024-05-27T09:31:54.917 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/172/17' 2024-05-27T09:31:54.917 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/172' 2024-05-27T09:31:54.917 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/186/19' 2024-05-27T09:31:54.917 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/186/15' 2024-05-27T09:31:54.917 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/186/20' 2024-05-27T09:31:54.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/186/12' 2024-05-27T09:31:54.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/186/10' 2024-05-27T09:31:54.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/186/11' 2024-05-27T09:31:54.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/186/16' 2024-05-27T09:31:54.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/186/14' 2024-05-27T09:31:54.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/186/18' 2024-05-27T09:31:54.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/186/13' 2024-05-27T09:31:54.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/186/17' 2024-05-27T09:31:54.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/186' 2024-05-27T09:31:54.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/190/19' 2024-05-27T09:31:54.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/190/15' 2024-05-27T09:31:54.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/190/20' 2024-05-27T09:31:54.918 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/190/12' 2024-05-27T09:31:54.919 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/190/10' 2024-05-27T09:31:54.919 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/190/11' 2024-05-27T09:31:54.919 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/190/16' 2024-05-27T09:31:54.919 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/190/14' 2024-05-27T09:31:55.322 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/190/18' 2024-05-27T09:31:55.322 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/190/13' 2024-05-27T09:31:55.322 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/190/17' 2024-05-27T09:31:55.323 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/190' 2024-05-27T09:31:55.323 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/104/19' 2024-05-27T09:31:55.323 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/104/15' 2024-05-27T09:31:55.323 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/104/20' 2024-05-27T09:31:55.323 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/104/12' 2024-05-27T09:31:55.323 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/104/10' 2024-05-27T09:31:55.323 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/104/11' 2024-05-27T09:31:55.323 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/104/16' 2024-05-27T09:31:55.323 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/104/14' 2024-05-27T09:31:55.323 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/104/18' 2024-05-27T09:31:55.323 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/104/13' 2024-05-27T09:31:55.323 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/104/17' 2024-05-27T09:31:55.323 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/104' 2024-05-27T09:31:55.323 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/164/19' 2024-05-27T09:31:55.324 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/164/15' 2024-05-27T09:31:55.324 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/164/20' 2024-05-27T09:31:55.324 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/164/12' 2024-05-27T09:31:55.324 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/164/10' 2024-05-27T09:31:55.324 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/164/11' 2024-05-27T09:31:55.324 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/164/16' 2024-05-27T09:31:55.324 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/164/14' 2024-05-27T09:31:55.324 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/164/18' 2024-05-27T09:31:55.324 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/164/13' 2024-05-27T09:31:55.324 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/164/17' 2024-05-27T09:31:55.324 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/164' 2024-05-27T09:31:55.324 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/112/19' 2024-05-27T09:31:55.325 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/112/15' 2024-05-27T09:31:55.325 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/112/20' 2024-05-27T09:31:55.325 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/112/12' 2024-05-27T09:31:55.325 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/112/10' 2024-05-27T09:31:55.325 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/112/11' 2024-05-27T09:31:55.325 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/112/16' 2024-05-27T09:31:55.325 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/112/14' 2024-05-27T09:31:55.325 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/112/18' 2024-05-27T09:31:55.325 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/112/13' 2024-05-27T09:31:55.325 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/112/17' 2024-05-27T09:31:55.325 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/112' 2024-05-27T09:31:55.325 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/100/19' 2024-05-27T09:31:55.325 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/100/15' 2024-05-27T09:31:55.325 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/100/20' 2024-05-27T09:31:55.326 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/100/12' 2024-05-27T09:31:55.326 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/100/10' 2024-05-27T09:31:55.326 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/100/11' 2024-05-27T09:31:55.326 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/100/16' 2024-05-27T09:31:55.326 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/100/14' 2024-05-27T09:31:55.326 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/100/18' 2024-05-27T09:31:55.326 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/100/13' 2024-05-27T09:31:55.326 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/100/17' 2024-05-27T09:31:55.326 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/100' 2024-05-27T09:31:55.326 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/134/19' 2024-05-27T09:31:55.326 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/134/15' 2024-05-27T09:31:55.326 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/134/20' 2024-05-27T09:31:55.327 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/134/12' 2024-05-27T09:31:55.327 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/134/10' 2024-05-27T09:31:55.327 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/134/11' 2024-05-27T09:31:55.327 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/134/16' 2024-05-27T09:31:55.327 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/134/14' 2024-05-27T09:31:55.327 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/134/18' 2024-05-27T09:31:55.327 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/134/13' 2024-05-27T09:31:55.327 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/134/17' 2024-05-27T09:31:55.327 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/134' 2024-05-27T09:31:55.327 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/115/19' 2024-05-27T09:31:55.327 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/115/15' 2024-05-27T09:31:55.327 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/115/20' 2024-05-27T09:31:55.328 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/115/12' 2024-05-27T09:31:55.328 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/115/10' 2024-05-27T09:31:55.328 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/115/11' 2024-05-27T09:31:55.328 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/115/16' 2024-05-27T09:31:55.328 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/115/14' 2024-05-27T09:31:55.328 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/115/18' 2024-05-27T09:31:55.328 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/115/13' 2024-05-27T09:31:55.328 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/115/17' 2024-05-27T09:31:55.328 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/115' 2024-05-27T09:31:55.328 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/116/19' 2024-05-27T09:31:55.328 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/116/15' 2024-05-27T09:31:55.328 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/116/20' 2024-05-27T09:31:55.329 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/116/12' 2024-05-27T09:31:55.329 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/116/10' 2024-05-27T09:31:55.329 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/116/11' 2024-05-27T09:31:55.329 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/116/16' 2024-05-27T09:31:55.329 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/116/14' 2024-05-27T09:31:55.329 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/116/18' 2024-05-27T09:31:55.329 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/116/13' 2024-05-27T09:31:55.329 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/116/17' 2024-05-27T09:31:55.329 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/116' 2024-05-27T09:31:55.329 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/195/19' 2024-05-27T09:31:55.329 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/195/15' 2024-05-27T09:31:55.329 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/195/20' 2024-05-27T09:31:55.329 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/195/12' 2024-05-27T09:31:55.329 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/195/10' 2024-05-27T09:31:55.330 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/195/11' 2024-05-27T09:31:55.330 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/195/16' 2024-05-27T09:31:55.330 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/195/14' 2024-05-27T09:31:55.330 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/195/18' 2024-05-27T09:31:55.330 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/195/13' 2024-05-27T09:31:55.330 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/195/17' 2024-05-27T09:31:55.330 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/195' 2024-05-27T09:31:55.330 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/138/19' 2024-05-27T09:31:55.330 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/138/15' 2024-05-27T09:31:55.330 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/138/20' 2024-05-27T09:31:55.330 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/138/12' 2024-05-27T09:31:55.330 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/138/10' 2024-05-27T09:31:55.331 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/138/11' 2024-05-27T09:31:55.331 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/138/16' 2024-05-27T09:31:55.331 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/138/14' 2024-05-27T09:31:55.331 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/138/18' 2024-05-27T09:31:55.893 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/138/13' 2024-05-27T09:31:55.893 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/138/17' 2024-05-27T09:31:55.893 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/138' 2024-05-27T09:31:55.894 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/135/19' 2024-05-27T09:31:55.894 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/135/15' 2024-05-27T09:31:55.894 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/135/20' 2024-05-27T09:31:55.894 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/135/12' 2024-05-27T09:31:55.894 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/135/10' 2024-05-27T09:31:55.894 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/135/11' 2024-05-27T09:31:55.894 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/135/16' 2024-05-27T09:31:55.894 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/135/14' 2024-05-27T09:31:55.894 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/135/18' 2024-05-27T09:31:55.894 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/135/13' 2024-05-27T09:31:55.894 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/135/17' 2024-05-27T09:31:55.894 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/135' 2024-05-27T09:31:55.895 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/117/19' 2024-05-27T09:31:55.895 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/117/15' 2024-05-27T09:31:55.895 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/117/20' 2024-05-27T09:31:55.895 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/117/12' 2024-05-27T09:31:55.895 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/117/10' 2024-05-27T09:31:55.895 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/117/11' 2024-05-27T09:31:55.895 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/117/16' 2024-05-27T09:31:55.895 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/117/14' 2024-05-27T09:31:55.895 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/117/18' 2024-05-27T09:31:55.895 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/117/13' 2024-05-27T09:31:55.895 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/117/17' 2024-05-27T09:31:55.895 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/117' 2024-05-27T09:31:55.895 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/143/19' 2024-05-27T09:31:55.895 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/143/15' 2024-05-27T09:31:55.896 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/143/20' 2024-05-27T09:31:55.896 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/143/12' 2024-05-27T09:31:55.896 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/143/10' 2024-05-27T09:31:55.896 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/143/11' 2024-05-27T09:31:55.896 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/143/16' 2024-05-27T09:31:55.896 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/143/14' 2024-05-27T09:31:55.896 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/143/18' 2024-05-27T09:31:55.896 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/143/13' 2024-05-27T09:31:55.896 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/143/17' 2024-05-27T09:31:55.896 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/143' 2024-05-27T09:31:55.896 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/180/19' 2024-05-27T09:31:55.896 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/180/15' 2024-05-27T09:31:55.896 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/180/20' 2024-05-27T09:31:55.896 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/180/12' 2024-05-27T09:31:55.897 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/180/10' 2024-05-27T09:31:55.897 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/180/11' 2024-05-27T09:31:55.897 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/180/16' 2024-05-27T09:31:55.897 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/180/14' 2024-05-27T09:31:55.897 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/180/18' 2024-05-27T09:31:55.897 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/180/13' 2024-05-27T09:31:55.897 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/180/17' 2024-05-27T09:31:55.897 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/180' 2024-05-27T09:31:55.897 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/126/19' 2024-05-27T09:31:55.897 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/126/15' 2024-05-27T09:31:55.897 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/126/20' 2024-05-27T09:31:55.897 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/126/12' 2024-05-27T09:31:55.897 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/126/10' 2024-05-27T09:31:55.897 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/126/11' 2024-05-27T09:31:55.898 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/126/16' 2024-05-27T09:31:55.898 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/126/14' 2024-05-27T09:31:55.898 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/126/18' 2024-05-27T09:31:55.898 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/126/13' 2024-05-27T09:31:55.898 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/126/17' 2024-05-27T09:31:55.898 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/126' 2024-05-27T09:31:55.898 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/161/19' 2024-05-27T09:31:55.898 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/161/15' 2024-05-27T09:31:55.898 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/161/20' 2024-05-27T09:31:55.898 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/161/12' 2024-05-27T09:31:55.898 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/161/10' 2024-05-27T09:31:55.898 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/161/11' 2024-05-27T09:31:55.898 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/161/16' 2024-05-27T09:31:55.899 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/161/14' 2024-05-27T09:31:55.899 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/161/18' 2024-05-27T09:31:55.899 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/161/13' 2024-05-27T09:31:55.899 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/161/17' 2024-05-27T09:31:55.899 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/161' 2024-05-27T09:31:55.899 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/176/19' 2024-05-27T09:31:55.899 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/176/15' 2024-05-27T09:31:55.899 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/176/20' 2024-05-27T09:31:55.899 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/176/12' 2024-05-27T09:31:55.899 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/176/10' 2024-05-27T09:31:55.899 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/176/11' 2024-05-27T09:31:55.899 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/176/16' 2024-05-27T09:31:55.899 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/176/14' 2024-05-27T09:31:55.900 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/176/18' 2024-05-27T09:31:55.900 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/176/13' 2024-05-27T09:31:55.900 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/176/17' 2024-05-27T09:31:55.900 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/176' 2024-05-27T09:31:55.900 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/111/19' 2024-05-27T09:31:55.900 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/111/15' 2024-05-27T09:31:55.900 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/111/20' 2024-05-27T09:31:55.900 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/111/12' 2024-05-27T09:31:55.900 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/111/10' 2024-05-27T09:31:55.900 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/111/11' 2024-05-27T09:31:55.900 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/111/16' 2024-05-27T09:31:55.900 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/111/14' 2024-05-27T09:31:55.900 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/111/18' 2024-05-27T09:31:55.901 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/111/13' 2024-05-27T09:31:55.901 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/111/17' 2024-05-27T09:31:55.901 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/111' 2024-05-27T09:31:55.901 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/191/19' 2024-05-27T09:31:55.901 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/191/15' 2024-05-27T09:31:55.901 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/191/20' 2024-05-27T09:31:55.901 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/191/12' 2024-05-27T09:31:55.901 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/191/10' 2024-05-27T09:31:55.901 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/191/11' 2024-05-27T09:31:55.901 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/191/16' 2024-05-27T09:31:55.901 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/191/14' 2024-05-27T09:31:55.901 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/191/18' 2024-05-27T09:31:55.901 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/191/13' 2024-05-27T09:31:56.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/191/17' 2024-05-27T09:31:56.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/191' 2024-05-27T09:31:56.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/184/19' 2024-05-27T09:31:56.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/184/15' 2024-05-27T09:31:56.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/184/20' 2024-05-27T09:31:56.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/184/12' 2024-05-27T09:31:56.597 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/184/10' 2024-05-27T09:31:56.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/184/11' 2024-05-27T09:31:56.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/184/16' 2024-05-27T09:31:56.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/184/14' 2024-05-27T09:31:56.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/184/18' 2024-05-27T09:31:56.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/184/13' 2024-05-27T09:31:56.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/184/17' 2024-05-27T09:31:56.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/184' 2024-05-27T09:31:56.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/192/19' 2024-05-27T09:31:56.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/192/15' 2024-05-27T09:31:56.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/192/20' 2024-05-27T09:31:56.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/192/12' 2024-05-27T09:31:56.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/192/10' 2024-05-27T09:31:56.598 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/192/11' 2024-05-27T09:31:56.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/192/16' 2024-05-27T09:31:56.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/192/14' 2024-05-27T09:31:56.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/192/18' 2024-05-27T09:31:56.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/192/13' 2024-05-27T09:31:56.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/192/17' 2024-05-27T09:31:56.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/192' 2024-05-27T09:31:56.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/103/19' 2024-05-27T09:31:56.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/103/15' 2024-05-27T09:31:56.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/103/20' 2024-05-27T09:31:56.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/103/12' 2024-05-27T09:31:56.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/103/10' 2024-05-27T09:31:56.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/103/11' 2024-05-27T09:31:56.599 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/103/16' 2024-05-27T09:31:56.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/103/14' 2024-05-27T09:31:56.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/103/18' 2024-05-27T09:31:56.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/103/13' 2024-05-27T09:31:56.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/103/17' 2024-05-27T09:31:56.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/103' 2024-05-27T09:31:56.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/128/19' 2024-05-27T09:31:56.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/128/15' 2024-05-27T09:31:56.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/128/20' 2024-05-27T09:31:56.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/128/12' 2024-05-27T09:31:56.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/128/10' 2024-05-27T09:31:56.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/128/11' 2024-05-27T09:31:56.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/128/16' 2024-05-27T09:31:56.600 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/128/14' 2024-05-27T09:31:56.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/128/18' 2024-05-27T09:31:56.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/128/13' 2024-05-27T09:31:56.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/128/17' 2024-05-27T09:31:56.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/128' 2024-05-27T09:31:56.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/144/19' 2024-05-27T09:31:56.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/144/15' 2024-05-27T09:31:56.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/144/20' 2024-05-27T09:31:56.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/144/12' 2024-05-27T09:31:56.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/144/10' 2024-05-27T09:31:56.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/144/11' 2024-05-27T09:31:56.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/144/16' 2024-05-27T09:31:56.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/144/14' 2024-05-27T09:31:56.601 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/144/18' 2024-05-27T09:31:56.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/144/13' 2024-05-27T09:31:56.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/144/17' 2024-05-27T09:31:56.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/144' 2024-05-27T09:31:56.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/146/19' 2024-05-27T09:31:56.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/146/15' 2024-05-27T09:31:56.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/146/20' 2024-05-27T09:31:56.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/146/12' 2024-05-27T09:31:56.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/146/10' 2024-05-27T09:31:56.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/146/11' 2024-05-27T09:31:56.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/146/16' 2024-05-27T09:31:56.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/146/14' 2024-05-27T09:31:56.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/146/18' 2024-05-27T09:31:56.602 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/146/13' 2024-05-27T09:31:56.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/146/17' 2024-05-27T09:31:56.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/146' 2024-05-27T09:31:56.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/141/19' 2024-05-27T09:31:56.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/141/15' 2024-05-27T09:31:56.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/141/20' 2024-05-27T09:31:56.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/141/12' 2024-05-27T09:31:56.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/141/10' 2024-05-27T09:31:56.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/141/11' 2024-05-27T09:31:56.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/141/16' 2024-05-27T09:31:56.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/141/14' 2024-05-27T09:31:56.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/141/18' 2024-05-27T09:31:56.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/141/13' 2024-05-27T09:31:56.603 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/141/17' 2024-05-27T09:31:56.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/141' 2024-05-27T09:31:56.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/182/19' 2024-05-27T09:31:56.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/182/15' 2024-05-27T09:31:56.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/182/20' 2024-05-27T09:31:56.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/182/12' 2024-05-27T09:31:56.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/182/10' 2024-05-27T09:31:56.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/182/11' 2024-05-27T09:31:56.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/182/16' 2024-05-27T09:31:56.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/182/14' 2024-05-27T09:31:56.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/182/18' 2024-05-27T09:31:56.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/182/13' 2024-05-27T09:31:56.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/182/17' 2024-05-27T09:31:56.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/182' 2024-05-27T09:31:56.604 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/107/19' 2024-05-27T09:31:56.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/107/15' 2024-05-27T09:31:56.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/107/20' 2024-05-27T09:31:56.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/107/12' 2024-05-27T09:31:56.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/107/10' 2024-05-27T09:31:56.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/107/11' 2024-05-27T09:31:56.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/107/16' 2024-05-27T09:31:56.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/107/14' 2024-05-27T09:31:56.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/107/18' 2024-05-27T09:31:56.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/107/13' 2024-05-27T09:31:56.605 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/107/17' 2024-05-27T09:31:57.009 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/107' 2024-05-27T09:31:57.009 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/178/19' 2024-05-27T09:31:57.009 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/178/15' 2024-05-27T09:31:57.009 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/178/20' 2024-05-27T09:31:57.009 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/178/12' 2024-05-27T09:31:57.009 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/178/10' 2024-05-27T09:31:57.009 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/178/11' 2024-05-27T09:31:57.009 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/178/16' 2024-05-27T09:31:57.010 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/178/14' 2024-05-27T09:31:57.010 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/178/18' 2024-05-27T09:31:57.010 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/178/13' 2024-05-27T09:31:57.010 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/178/17' 2024-05-27T09:31:57.010 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/178' 2024-05-27T09:31:57.010 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/156/19' 2024-05-27T09:31:57.010 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/156/15' 2024-05-27T09:31:57.010 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/156/20' 2024-05-27T09:31:57.010 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/156/12' 2024-05-27T09:31:57.010 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/156/10' 2024-05-27T09:31:57.010 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/156/11' 2024-05-27T09:31:57.010 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/156/16' 2024-05-27T09:31:57.011 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/156/14' 2024-05-27T09:31:57.011 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/156/18' 2024-05-27T09:31:57.011 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/156/13' 2024-05-27T09:31:57.011 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/156/17' 2024-05-27T09:31:57.011 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/156' 2024-05-27T09:31:57.011 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/131/19' 2024-05-27T09:31:57.011 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/131/15' 2024-05-27T09:31:57.011 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/131/20' 2024-05-27T09:31:57.011 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/131/12' 2024-05-27T09:31:57.011 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/131/10' 2024-05-27T09:31:57.011 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/131/11' 2024-05-27T09:31:57.011 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/131/16' 2024-05-27T09:31:57.011 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/131/14' 2024-05-27T09:31:57.012 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/131/18' 2024-05-27T09:31:57.012 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/131/13' 2024-05-27T09:31:57.012 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/131/17' 2024-05-27T09:31:57.012 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/131' 2024-05-27T09:31:57.012 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/123/19' 2024-05-27T09:31:57.012 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/123/15' 2024-05-27T09:31:57.012 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/123/20' 2024-05-27T09:31:57.012 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/123/12' 2024-05-27T09:31:57.012 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/123/10' 2024-05-27T09:31:57.012 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/123/11' 2024-05-27T09:31:57.012 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/123/16' 2024-05-27T09:31:57.012 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/123/14' 2024-05-27T09:31:57.012 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/123/18' 2024-05-27T09:31:57.012 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/123/13' 2024-05-27T09:31:57.013 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/123/17' 2024-05-27T09:31:57.013 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/123' 2024-05-27T09:31:57.013 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/160/19' 2024-05-27T09:31:57.013 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/160/15' 2024-05-27T09:31:57.013 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/160/20' 2024-05-27T09:31:57.013 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/160/12' 2024-05-27T09:31:57.013 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/160/10' 2024-05-27T09:31:57.013 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/160/11' 2024-05-27T09:31:57.013 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/160/16' 2024-05-27T09:31:57.013 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/160/14' 2024-05-27T09:31:57.013 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/160/18' 2024-05-27T09:31:57.013 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/160/13' 2024-05-27T09:31:57.013 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/160/17' 2024-05-27T09:31:57.013 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/160' 2024-05-27T09:31:57.014 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/165/19' 2024-05-27T09:31:57.014 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/165/15' 2024-05-27T09:31:57.014 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/165/20' 2024-05-27T09:31:57.014 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/165/12' 2024-05-27T09:31:57.014 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/165/10' 2024-05-27T09:31:57.014 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/165/11' 2024-05-27T09:31:57.014 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/165/16' 2024-05-27T09:31:57.014 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/165/14' 2024-05-27T09:31:57.014 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/165/18' 2024-05-27T09:31:57.014 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/165/13' 2024-05-27T09:31:57.014 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/165/17' 2024-05-27T09:31:57.014 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/165' 2024-05-27T09:31:57.014 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/101/19' 2024-05-27T09:31:57.015 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/101/15' 2024-05-27T09:31:57.015 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/101/20' 2024-05-27T09:31:57.015 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/101/12' 2024-05-27T09:31:57.015 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/101/10' 2024-05-27T09:31:57.015 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/101/11' 2024-05-27T09:31:57.015 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/101/16' 2024-05-27T09:31:57.015 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/101/14' 2024-05-27T09:31:57.015 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/101/18' 2024-05-27T09:31:57.015 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/101/13' 2024-05-27T09:31:57.015 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/101/17' 2024-05-27T09:31:57.015 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/101' 2024-05-27T09:31:57.015 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/148/19' 2024-05-27T09:31:57.016 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/148/15' 2024-05-27T09:31:57.016 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/148/20' 2024-05-27T09:31:57.016 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/148/12' 2024-05-27T09:31:57.016 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/148/10' 2024-05-27T09:31:57.016 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/148/11' 2024-05-27T09:31:57.016 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/148/16' 2024-05-27T09:31:57.016 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/148/14' 2024-05-27T09:31:57.016 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/148/18' 2024-05-27T09:31:57.016 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/148/13' 2024-05-27T09:31:57.016 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/148/17' 2024-05-27T09:31:57.016 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/148' 2024-05-27T09:31:57.016 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/168/19' 2024-05-27T09:31:57.016 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/168/15' 2024-05-27T09:31:57.016 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/168/20' 2024-05-27T09:31:57.016 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/168/12' 2024-05-27T09:31:57.017 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/168/10' 2024-05-27T09:31:57.017 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/168/11' 2024-05-27T09:31:57.017 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/168/16' 2024-05-27T09:31:57.017 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/168/14' 2024-05-27T09:31:57.017 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/168/18' 2024-05-27T09:31:57.017 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/168/13' 2024-05-27T09:31:57.017 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/168/17' 2024-05-27T09:31:57.017 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/168' 2024-05-27T09:31:57.679 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/179/19' 2024-05-27T09:31:57.679 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/179/15' 2024-05-27T09:31:57.679 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/179/20' 2024-05-27T09:31:57.679 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/179/12' 2024-05-27T09:31:57.680 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/179/10' 2024-05-27T09:31:57.680 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/179/11' 2024-05-27T09:31:57.680 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/179/16' 2024-05-27T09:31:57.680 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/179/14' 2024-05-27T09:31:57.680 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/179/18' 2024-05-27T09:31:57.680 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/179/13' 2024-05-27T09:31:57.680 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/179/17' 2024-05-27T09:31:57.680 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/179' 2024-05-27T09:31:57.680 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/132/19' 2024-05-27T09:31:57.680 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/132/15' 2024-05-27T09:31:57.680 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/132/20' 2024-05-27T09:31:57.680 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/132/12' 2024-05-27T09:31:57.680 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/132/10' 2024-05-27T09:31:57.681 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/132/11' 2024-05-27T09:31:57.681 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/132/16' 2024-05-27T09:31:57.681 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/132/14' 2024-05-27T09:31:57.681 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/132/18' 2024-05-27T09:31:57.681 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/132/13' 2024-05-27T09:31:57.681 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/132/17' 2024-05-27T09:31:57.681 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/132' 2024-05-27T09:31:57.681 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/199/19' 2024-05-27T09:31:57.681 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/199/15' 2024-05-27T09:31:57.681 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/199/20' 2024-05-27T09:31:57.681 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/199/12' 2024-05-27T09:31:57.681 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/199/10' 2024-05-27T09:31:57.681 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/199/11' 2024-05-27T09:31:57.681 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/199/16' 2024-05-27T09:31:57.682 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/199/14' 2024-05-27T09:31:57.682 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/199/18' 2024-05-27T09:31:57.682 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/199/13' 2024-05-27T09:31:57.682 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/199/17' 2024-05-27T09:31:57.682 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/199' 2024-05-27T09:31:57.682 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/163/19' 2024-05-27T09:31:57.682 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/163/15' 2024-05-27T09:31:57.682 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/163/20' 2024-05-27T09:31:57.682 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/163/12' 2024-05-27T09:31:57.682 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/163/10' 2024-05-27T09:31:57.682 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/163/11' 2024-05-27T09:31:57.682 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/163/16' 2024-05-27T09:31:57.683 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/163/14' 2024-05-27T09:31:57.683 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/163/18' 2024-05-27T09:31:57.683 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/163/13' 2024-05-27T09:31:57.683 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/163/17' 2024-05-27T09:31:57.683 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/163' 2024-05-27T09:31:57.683 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/153/19' 2024-05-27T09:31:57.683 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/153/15' 2024-05-27T09:31:57.683 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/153/20' 2024-05-27T09:31:57.683 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/153/12' 2024-05-27T09:31:57.683 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/153/10' 2024-05-27T09:31:57.683 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/153/11' 2024-05-27T09:31:57.683 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/153/16' 2024-05-27T09:31:57.683 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/153/14' 2024-05-27T09:31:57.683 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/153/18' 2024-05-27T09:31:57.684 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/153/13' 2024-05-27T09:31:57.684 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/153/17' 2024-05-27T09:31:57.684 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/153' 2024-05-27T09:31:57.684 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/122/19' 2024-05-27T09:31:57.684 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/122/15' 2024-05-27T09:31:57.684 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/122/20' 2024-05-27T09:31:57.684 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/122/12' 2024-05-27T09:31:57.684 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/122/10' 2024-05-27T09:31:57.684 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/122/11' 2024-05-27T09:31:57.684 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/122/16' 2024-05-27T09:31:57.684 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/122/14' 2024-05-27T09:31:57.684 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/122/18' 2024-05-27T09:31:57.684 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/122/13' 2024-05-27T09:31:57.685 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/122/17' 2024-05-27T09:31:57.685 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/122' 2024-05-27T09:31:57.685 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/162/19' 2024-05-27T09:31:57.685 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/162/15' 2024-05-27T09:31:57.685 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/162/20' 2024-05-27T09:31:57.685 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/162/12' 2024-05-27T09:31:57.685 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/162/10' 2024-05-27T09:31:57.685 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/162/11' 2024-05-27T09:31:57.685 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/162/16' 2024-05-27T09:31:57.685 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/162/14' 2024-05-27T09:31:57.685 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/162/18' 2024-05-27T09:31:57.685 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/162/13' 2024-05-27T09:31:57.685 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/162/17' 2024-05-27T09:31:57.685 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/162' 2024-05-27T09:31:57.686 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/197/19' 2024-05-27T09:31:57.686 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/197/15' 2024-05-27T09:31:57.686 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/197/20' 2024-05-27T09:31:57.686 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/197/12' 2024-05-27T09:31:57.686 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/197/10' 2024-05-27T09:31:57.686 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/197/11' 2024-05-27T09:31:57.686 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/197/16' 2024-05-27T09:31:57.686 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/197/14' 2024-05-27T09:31:57.686 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/197/18' 2024-05-27T09:31:57.686 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/197/13' 2024-05-27T09:31:57.686 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/197/17' 2024-05-27T09:31:57.686 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/197' 2024-05-27T09:31:57.686 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/113/19' 2024-05-27T09:31:57.687 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/113/15' 2024-05-27T09:31:57.687 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/113/20' 2024-05-27T09:31:57.687 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/113/12' 2024-05-27T09:31:57.687 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/113/10' 2024-05-27T09:31:57.687 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/113/11' 2024-05-27T09:31:57.687 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/113/16' 2024-05-27T09:31:57.687 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/113/14' 2024-05-27T09:31:57.687 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/113/18' 2024-05-27T09:31:57.687 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/113/13' 2024-05-27T09:31:57.687 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/113/17' 2024-05-27T09:31:57.687 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/113' 2024-05-27T09:31:57.687 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/152/19' 2024-05-27T09:31:58.091 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/152/15' 2024-05-27T09:31:58.091 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/152/20' 2024-05-27T09:31:58.091 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/152/12' 2024-05-27T09:31:58.091 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/152/10' 2024-05-27T09:31:58.091 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/152/11' 2024-05-27T09:31:58.091 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/152/16' 2024-05-27T09:31:58.092 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/152/14' 2024-05-27T09:31:58.092 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/152/18' 2024-05-27T09:31:58.092 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/152/13' 2024-05-27T09:31:58.092 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/152/17' 2024-05-27T09:31:58.092 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/152' 2024-05-27T09:31:58.092 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/121/19' 2024-05-27T09:31:58.092 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/121/15' 2024-05-27T09:31:58.092 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/121/20' 2024-05-27T09:31:58.092 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/121/12' 2024-05-27T09:31:58.092 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/121/10' 2024-05-27T09:31:58.092 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/121/11' 2024-05-27T09:31:58.092 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/121/16' 2024-05-27T09:31:58.092 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/121/14' 2024-05-27T09:31:58.093 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/121/18' 2024-05-27T09:31:58.093 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/121/13' 2024-05-27T09:31:58.093 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/121/17' 2024-05-27T09:31:58.093 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/121' 2024-05-27T09:31:58.093 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/118/19' 2024-05-27T09:31:58.093 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/118/15' 2024-05-27T09:31:58.093 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/118/20' 2024-05-27T09:31:58.093 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/118/12' 2024-05-27T09:31:58.093 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/118/10' 2024-05-27T09:31:58.093 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/118/11' 2024-05-27T09:31:58.093 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/118/16' 2024-05-27T09:31:58.093 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/118/14' 2024-05-27T09:31:58.093 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/118/18' 2024-05-27T09:31:58.094 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/118/13' 2024-05-27T09:31:58.094 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/118/17' 2024-05-27T09:31:58.094 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/118' 2024-05-27T09:31:58.094 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/196/19' 2024-05-27T09:31:58.094 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/196/15' 2024-05-27T09:31:58.094 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/196/20' 2024-05-27T09:31:58.094 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/196/12' 2024-05-27T09:31:58.094 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/196/10' 2024-05-27T09:31:58.094 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/196/11' 2024-05-27T09:31:58.094 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/196/16' 2024-05-27T09:31:58.094 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/196/14' 2024-05-27T09:31:58.094 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/196/18' 2024-05-27T09:31:58.094 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/196/13' 2024-05-27T09:31:58.095 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/196/17' 2024-05-27T09:31:58.095 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/196' 2024-05-27T09:31:58.095 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/124/19' 2024-05-27T09:31:58.095 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/124/15' 2024-05-27T09:31:58.095 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/124/20' 2024-05-27T09:31:58.095 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/124/12' 2024-05-27T09:31:58.095 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/124/10' 2024-05-27T09:31:58.095 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/124/11' 2024-05-27T09:31:58.095 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/124/16' 2024-05-27T09:31:58.095 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/124/14' 2024-05-27T09:31:58.095 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/124/18' 2024-05-27T09:31:58.095 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/124/13' 2024-05-27T09:31:58.095 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/124/17' 2024-05-27T09:31:58.096 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/124' 2024-05-27T09:31:58.096 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/188/19' 2024-05-27T09:31:58.096 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/188/15' 2024-05-27T09:31:58.096 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/188/20' 2024-05-27T09:31:58.096 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/188/12' 2024-05-27T09:31:58.096 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/188/10' 2024-05-27T09:31:58.096 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/188/11' 2024-05-27T09:31:58.096 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/188/16' 2024-05-27T09:31:58.096 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/188/14' 2024-05-27T09:31:58.096 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/188/18' 2024-05-27T09:31:58.096 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/188/13' 2024-05-27T09:31:58.096 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/188/17' 2024-05-27T09:31:58.097 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/188' 2024-05-27T09:31:58.097 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/145/19' 2024-05-27T09:31:58.097 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/145/15' 2024-05-27T09:31:58.097 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/145/20' 2024-05-27T09:31:58.097 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/145/12' 2024-05-27T09:31:58.097 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/145/10' 2024-05-27T09:31:58.097 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/145/11' 2024-05-27T09:31:58.097 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/145/16' 2024-05-27T09:31:58.097 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/145/14' 2024-05-27T09:31:58.097 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/145/18' 2024-05-27T09:31:58.097 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/145/13' 2024-05-27T09:31:58.097 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/145/17' 2024-05-27T09:31:58.097 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/145' 2024-05-27T09:31:58.097 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/130/19' 2024-05-27T09:31:58.098 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/130/15' 2024-05-27T09:31:58.098 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/130/20' 2024-05-27T09:31:58.098 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/130/12' 2024-05-27T09:31:58.098 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/130/10' 2024-05-27T09:31:58.098 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/130/11' 2024-05-27T09:31:58.098 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/130/16' 2024-05-27T09:31:58.098 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/130/14' 2024-05-27T09:31:58.098 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/130/18' 2024-05-27T09:31:58.098 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/130/13' 2024-05-27T09:31:58.098 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/130/17' 2024-05-27T09:31:58.098 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/130' 2024-05-27T09:31:58.098 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/193/19' 2024-05-27T09:31:58.099 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/193/15' 2024-05-27T09:31:58.099 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/193/20' 2024-05-27T09:31:58.099 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/193/12' 2024-05-27T09:31:58.099 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/193/10' 2024-05-27T09:31:58.099 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/193/11' 2024-05-27T09:31:58.099 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/193/16' 2024-05-27T09:31:58.099 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/193/14' 2024-05-27T09:31:58.100 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/193/18' 2024-05-27T09:31:58.100 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/193/13' 2024-05-27T09:31:58.100 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/193/17' 2024-05-27T09:31:58.100 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/193' 2024-05-27T09:31:58.100 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/175/19' 2024-05-27T09:31:58.100 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/175/15' 2024-05-27T09:31:58.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/175/20' 2024-05-27T09:31:58.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/175/12' 2024-05-27T09:31:58.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/175/10' 2024-05-27T09:31:58.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/175/11' 2024-05-27T09:31:58.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/175/16' 2024-05-27T09:31:58.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/175/14' 2024-05-27T09:31:58.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/175/18' 2024-05-27T09:31:58.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/175/13' 2024-05-27T09:31:58.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/175/17' 2024-05-27T09:31:58.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/175' 2024-05-27T09:31:58.623 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/181/19' 2024-05-27T09:31:58.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/181/15' 2024-05-27T09:31:58.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/181/20' 2024-05-27T09:31:58.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/181/12' 2024-05-27T09:31:58.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/181/10' 2024-05-27T09:31:58.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/181/11' 2024-05-27T09:31:58.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/181/16' 2024-05-27T09:31:58.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/181/14' 2024-05-27T09:31:58.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/181/18' 2024-05-27T09:31:58.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/181/13' 2024-05-27T09:31:58.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/181/17' 2024-05-27T09:31:58.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/181' 2024-05-27T09:31:58.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/127/19' 2024-05-27T09:31:58.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/127/15' 2024-05-27T09:31:58.624 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/127/20' 2024-05-27T09:31:58.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/127/12' 2024-05-27T09:31:58.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/127/10' 2024-05-27T09:31:58.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/127/11' 2024-05-27T09:31:58.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/127/16' 2024-05-27T09:31:58.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/127/14' 2024-05-27T09:31:58.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/127/18' 2024-05-27T09:31:58.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/127/13' 2024-05-27T09:31:58.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/127/17' 2024-05-27T09:31:58.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/127' 2024-05-27T09:31:58.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/108/19' 2024-05-27T09:31:58.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/108/15' 2024-05-27T09:31:58.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/108/20' 2024-05-27T09:31:58.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/108/12' 2024-05-27T09:31:58.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/108/10' 2024-05-27T09:31:58.625 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/108/11' 2024-05-27T09:31:58.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/108/16' 2024-05-27T09:31:58.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/108/14' 2024-05-27T09:31:58.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/108/18' 2024-05-27T09:31:58.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/108/13' 2024-05-27T09:31:58.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/108/17' 2024-05-27T09:31:58.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/108' 2024-05-27T09:31:58.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/183/19' 2024-05-27T09:31:58.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/183/15' 2024-05-27T09:31:58.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/183/20' 2024-05-27T09:31:58.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/183/12' 2024-05-27T09:31:58.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/183/10' 2024-05-27T09:31:58.626 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/183/11' 2024-05-27T09:31:58.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/183/16' 2024-05-27T09:31:58.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/183/14' 2024-05-27T09:31:58.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/183/18' 2024-05-27T09:31:58.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/183/13' 2024-05-27T09:31:58.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/183/17' 2024-05-27T09:31:58.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/183' 2024-05-27T09:31:58.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/177/19' 2024-05-27T09:31:58.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/177/15' 2024-05-27T09:31:58.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/177/20' 2024-05-27T09:31:58.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/177/12' 2024-05-27T09:31:58.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/177/10' 2024-05-27T09:31:58.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/177/11' 2024-05-27T09:31:58.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/177/16' 2024-05-27T09:31:58.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/177/14' 2024-05-27T09:31:58.627 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/177/18' 2024-05-27T09:31:58.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/177/13' 2024-05-27T09:31:58.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/177/17' 2024-05-27T09:31:58.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/177' 2024-05-27T09:31:58.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/169/19' 2024-05-27T09:31:58.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/169/15' 2024-05-27T09:31:58.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/169/20' 2024-05-27T09:31:58.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/169/12' 2024-05-27T09:31:58.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/169/10' 2024-05-27T09:31:58.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/169/11' 2024-05-27T09:31:58.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/169/16' 2024-05-27T09:31:58.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/169/14' 2024-05-27T09:31:58.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/169/18' 2024-05-27T09:31:58.628 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/169/13' 2024-05-27T09:31:58.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/169/17' 2024-05-27T09:31:58.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/169' 2024-05-27T09:31:58.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/154/19' 2024-05-27T09:31:58.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/154/15' 2024-05-27T09:31:58.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/154/20' 2024-05-27T09:31:58.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/154/12' 2024-05-27T09:31:58.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/154/10' 2024-05-27T09:31:58.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/154/11' 2024-05-27T09:31:58.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/154/16' 2024-05-27T09:31:58.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/154/14' 2024-05-27T09:31:58.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/154/18' 2024-05-27T09:31:58.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/154/13' 2024-05-27T09:31:58.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/154/17' 2024-05-27T09:31:58.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/154' 2024-05-27T09:31:58.629 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/171/19' 2024-05-27T09:31:58.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/171/15' 2024-05-27T09:31:58.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/171/20' 2024-05-27T09:31:58.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/171/12' 2024-05-27T09:31:58.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/171/10' 2024-05-27T09:31:58.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/171/11' 2024-05-27T09:31:58.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/171/16' 2024-05-27T09:31:58.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/171/14' 2024-05-27T09:31:58.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/171/18' 2024-05-27T09:31:58.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/171/13' 2024-05-27T09:31:58.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/171/17' 2024-05-27T09:31:58.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/171' 2024-05-27T09:31:58.630 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/139/19' 2024-05-27T09:31:58.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/139/15' 2024-05-27T09:31:58.631 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/139/20' 2024-05-27T09:31:58.738 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/139/12' 2024-05-27T09:31:58.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/139/10' 2024-05-27T09:31:58.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/139/11' 2024-05-27T09:31:58.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/139/16' 2024-05-27T09:31:58.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/139/14' 2024-05-27T09:31:58.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/139/18' 2024-05-27T09:31:58.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/139/13' 2024-05-27T09:31:58.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/139/17' 2024-05-27T09:31:58.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/139' 2024-05-27T09:31:58.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/102/19' 2024-05-27T09:31:58.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/102/15' 2024-05-27T09:31:58.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/102/20' 2024-05-27T09:31:58.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/102/12' 2024-05-27T09:31:58.739 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/102/10' 2024-05-27T09:31:58.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/102/11' 2024-05-27T09:31:58.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/102/16' 2024-05-27T09:31:58.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/102/14' 2024-05-27T09:31:58.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/102/18' 2024-05-27T09:31:58.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/102/13' 2024-05-27T09:31:58.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/102/17' 2024-05-27T09:31:58.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/102' 2024-05-27T09:31:58.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/129/19' 2024-05-27T09:31:58.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/129/15' 2024-05-27T09:31:58.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/129/20' 2024-05-27T09:31:58.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/129/12' 2024-05-27T09:31:58.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/129/10' 2024-05-27T09:31:58.740 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/129/11' 2024-05-27T09:31:58.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/129/16' 2024-05-27T09:31:58.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/129/14' 2024-05-27T09:31:58.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/129/18' 2024-05-27T09:31:58.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/129/13' 2024-05-27T09:31:58.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed './copyofsnap1/test1/129/17' 2024-05-27T09:31:58.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1/129' 2024-05-27T09:31:58.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1/test1' 2024-05-27T09:31:58.741 INFO:tasks.workunit.client.0.smithi114.stdout:removed directory './copyofsnap1' 2024-05-27T09:31:58.742 INFO:tasks.workunit.client.0.smithi114.stdout:OK 2024-05-27T09:31:58.742 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-27T09:31:58.743 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-27T09:31:58.816 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-authwb.sh... 2024-05-27T09:31:58.816 DEBUG:teuthology.orchestra.run.smithi114:workunit test fs/snaps/snaptest-authwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6507388dd5057528934822c0163b0c347ef1d5d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-authwb.sh 2024-05-27T09:32:03.635 INFO:tasks.workunit.client.0.smithi114.stderr:+ set -e 2024-05-27T09:32:03.635 INFO:tasks.workunit.client.0.smithi114.stderr:+ touch foo 2024-05-27T09:32:03.638 INFO:tasks.workunit.client.0.smithi114.stderr:+ chmod +x foo 2024-05-27T09:32:03.640 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir .snap/s 2024-05-27T09:32:03.676 INFO:tasks.workunit.client.0.smithi114.stderr:+ find .snap/s/foo -executable 2024-05-27T09:32:03.677 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep foo 2024-05-27T09:32:03.682 INFO:tasks.workunit.client.0.smithi114.stdout:.snap/s/foo 2024-05-27T09:32:03.682 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir .snap/s 2024-05-27T09:32:03.693 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm foo 2024-05-27T09:32:03.699 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo OK 2024-05-27T09:32:03.699 INFO:tasks.workunit.client.0.smithi114.stdout:OK 2024-05-27T09:32:03.700 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-27T09:32:03.700 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-27T09:32:03.795 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-capwb.sh... 2024-05-27T09:32:03.796 DEBUG:teuthology.orchestra.run.smithi114:workunit test fs/snaps/snaptest-capwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6507388dd5057528934822c0163b0c347ef1d5d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-capwb.sh 2024-05-27T09:32:08.657 INFO:tasks.workunit.client.0.smithi114.stderr:+ set -e 2024-05-27T09:32:08.658 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir foo 2024-05-27T09:32:08.661 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo x 2024-05-27T09:32:08.663 INFO:tasks.workunit.client.0.smithi114.stderr:+ sync 2024-05-27T09:32:12.168 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir foo/.snap/ss 2024-05-27T09:32:12.204 INFO:tasks.workunit.client.0.smithi114.stderr:+ ln foo/x foo/xx 2024-05-27T09:32:12.210 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat foo/.snap/ss/x 2024-05-27T09:32:12.226 INFO:tasks.workunit.client.0.smithi114.stdout:x 2024-05-27T09:32:12.227 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir foo/.snap/ss 2024-05-27T09:32:12.238 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo a 2024-05-27T09:32:12.240 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo b 2024-05-27T09:32:12.243 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir foo/.snap/s 2024-05-27T09:32:12.290 INFO:tasks.workunit.client.0.smithi114.stderr:++ cat foo/.snap/s/a 2024-05-27T09:32:12.310 INFO:tasks.workunit.client.0.smithi114.stderr:+ r=a 2024-05-27T09:32:12.310 INFO:tasks.workunit.client.0.smithi114.stderr:+ '[' -z a ']' 2024-05-27T09:32:12.310 INFO:tasks.workunit.client.0.smithi114.stderr:+ ln foo/b foo/b2 2024-05-27T09:32:12.316 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat foo/.snap/s/b 2024-05-27T09:32:12.331 INFO:tasks.workunit.client.0.smithi114.stdout:b 2024-05-27T09:32:12.331 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo 'this used to hang:' 2024-05-27T09:32:12.332 INFO:tasks.workunit.client.0.smithi114.stdout:this used to hang: 2024-05-27T09:32:12.332 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo more 2024-05-27T09:32:12.332 INFO:tasks.workunit.client.0.smithi114.stdout:oh, it didn't hang! good job. 2024-05-27T09:32:12.332 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo 'oh, it didn'\''t hang! good job.' 2024-05-27T09:32:12.333 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat foo/b 2024-05-27T09:32:12.334 INFO:tasks.workunit.client.0.smithi114.stdout:b 2024-05-27T09:32:12.335 INFO:tasks.workunit.client.0.smithi114.stdout:more 2024-05-27T09:32:12.335 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir foo/.snap/s 2024-05-27T09:32:12.347 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm -r foo 2024-05-27T09:32:12.401 INFO:tasks.workunit.client.0.smithi114.stdout:OK 2024-05-27T09:32:12.401 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo OK 2024-05-27T09:32:12.403 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-27T09:32:12.404 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-27T09:32:12.495 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-dir-rename.sh... 2024-05-27T09:32:12.496 DEBUG:teuthology.orchestra.run.smithi114:workunit test fs/snaps/snaptest-dir-rename.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6507388dd5057528934822c0163b0c347ef1d5d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-dir-rename.sh 2024-05-27T09:32:13.649 INFO:tasks.workunit.client.0.smithi114.stderr:+ set -e 2024-05-27T09:32:13.649 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a 2024-05-27T09:32:13.654 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir .snap/bar 2024-05-27T09:32:13.692 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/foo 2024-05-27T09:32:13.725 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a/.snap/foo 2024-05-27T09:32:13.738 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a 2024-05-27T09:32:13.746 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/bar/a 2024-05-27T09:32:13.756 INFO:tasks.workunit.client.0.smithi114.stdout: File: .snap/bar/a 2024-05-27T09:32:13.756 INFO:tasks.workunit.client.0.smithi114.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-27T09:32:13.756 INFO:tasks.workunit.client.0.smithi114.stdout:Device: fch/252d Inode: 1099511633797 Links: 2 2024-05-27T09:32:13.756 INFO:tasks.workunit.client.0.smithi114.stdout:Access: (0755/drwxr-xr-x) Uid: ( 1000/ ubuntu) Gid: ( 1316/ ubuntu) 2024-05-27T09:32:13.756 INFO:tasks.workunit.client.0.smithi114.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-05-27T09:32:13.756 INFO:tasks.workunit.client.0.smithi114.stdout:Access: 2024-05-27 09:32:13.650395528 +0000 2024-05-27T09:32:13.757 INFO:tasks.workunit.client.0.smithi114.stdout:Modify: 2024-05-27 09:32:13.650395528 +0000 2024-05-27T09:32:13.757 INFO:tasks.workunit.client.0.smithi114.stdout:Change: 2024-05-27 09:32:13.650395528 +0000 2024-05-27T09:32:13.757 INFO:tasks.workunit.client.0.smithi114.stdout: Birth: 2024-05-27 09:32:13.650395528 +0000 2024-05-27T09:32:13.757 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir .snap/bar 2024-05-27T09:32:13.771 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo OK 2024-05-27T09:32:13.772 INFO:tasks.workunit.client.0.smithi114.stdout:OK 2024-05-27T09:32:13.773 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-27T09:32:13.774 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-27T09:32:13.864 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-double-null.sh... 2024-05-27T09:32:13.864 DEBUG:teuthology.orchestra.run.smithi114:workunit test fs/snaps/snaptest-double-null.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6507388dd5057528934822c0163b0c347ef1d5d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-double-null.sh 2024-05-27T09:32:14.969 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:32:15.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.227+0000 7f348e400640 1 Processor -- start 2024-05-27T09:32:15.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.227+0000 7f348e400640 1 -- start start 2024-05-27T09:32:15.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.229+0000 7f348e400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3488154820 0x7f34880ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:15.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.229+0000 7f348e400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f34880ca810 0x7f348815dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:15.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.229+0000 7f348e400640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34880cac20 con 0x7f3488074a60 2024-05-27T09:32:15.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.229+0000 7f348e400640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f3488154c30 con 0x7f34880ca810 2024-05-27T09:32:15.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.229+0000 7f348e400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f348815db60 con 0x7f3488154820 2024-05-27T09:32:15.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.230+0000 7f3487e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f34880ca810 0x7f348815dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:15.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.230+0000 7f3486a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3488154820 0x7f34880ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:15.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.230+0000 7f3486a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3488154820 0x7f34880ca2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:44960/0 (socket says 172.21.15.114:44960) 2024-05-27T09:32:15.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.230+0000 7f3486a00640 1 -- 172.21.15.114:0/4117830557 learned_addr learned my addr 172.21.15.114:0/4117830557 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:15.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.230+0000 7f3487400640 1 --1- 172.21.15.114:0/4117830557 >> v1:172.21.15.114:6789/0 conn(0x7f3488074a60 0x7f3488073260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:42542/0 (socket says 172.21.15.114:42542) 2024-05-27T09:32:15.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.230+0000 7f3486a00640 1 -- 172.21.15.114:0/4117830557 >> v2:172.21.15.114:3300/0 conn(0x7f34880ca810 msgr2=0x7f348815dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:32:15.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.230+0000 7f3486a00640 1 --2- 172.21.15.114:0/4117830557 >> v2:172.21.15.114:3300/0 conn(0x7f34880ca810 0x7f348815dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:15.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.230+0000 7f3486a00640 1 -- 172.21.15.114:0/4117830557 >> v1:172.21.15.114:6789/0 conn(0x7f3488074a60 legacy=0x7f3488073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:15.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.230+0000 7f3486a00640 1 -- 172.21.15.114:0/4117830557 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f348815a810 con 0x7f3488154820 2024-05-27T09:32:15.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.231+0000 7f3486a00640 1 --2- 172.21.15.114:0/4117830557 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3488154820 0x7f34880ca2d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3470009870 tx=0x7f347002a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b669c8145bf91bba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:15.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.232+0000 7f3486000640 1 -- 172.21.15.114:0/4117830557 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470037070 con 0x7f3488154820 2024-05-27T09:32:15.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.232+0000 7f3486000640 1 -- 172.21.15.114:0/4117830557 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f347002ae50 con 0x7f3488154820 2024-05-27T09:32:15.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.232+0000 7f3486000640 1 -- 172.21.15.114:0/4117830557 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470032860 con 0x7f3488154820 2024-05-27T09:32:15.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.233+0000 7f348e400640 1 -- 172.21.15.114:0/4117830557 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3488154820 msgr2=0x7f34880ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:15.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.233+0000 7f348e400640 1 --2- 172.21.15.114:0/4117830557 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3488154820 0x7f34880ca2d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3470009870 tx=0x7f347002a300 comp rx=0 tx=0).stop 2024-05-27T09:32:15.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.233+0000 7f348e400640 1 -- 172.21.15.114:0/4117830557 shutdown_connections 2024-05-27T09:32:15.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.233+0000 7f348e400640 1 --2- 172.21.15.114:0/4117830557 >> v2:172.21.15.114:3300/0 conn(0x7f34880ca810 0x7f348815dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:15.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.233+0000 7f348e400640 1 --2- 172.21.15.114:0/4117830557 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3488154820 0x7f34880ca2d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:15.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.233+0000 7f348e400640 1 -- 172.21.15.114:0/4117830557 >> 172.21.15.114:0/4117830557 conn(0x7f348806f6e0 msgr2=0x7f3488070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:15.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.233+0000 7f348e400640 1 -- 172.21.15.114:0/4117830557 shutdown_connections 2024-05-27T09:32:15.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.233+0000 7f348e400640 1 -- 172.21.15.114:0/4117830557 wait complete. 2024-05-27T09:32:15.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.235+0000 7f348e400640 1 Processor -- start 2024-05-27T09:32:15.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.235+0000 7f348e400640 1 -- start start 2024-05-27T09:32:15.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.237+0000 7f348e400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3488074a60 0x7f34881e7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:15.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.237+0000 7f348e400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34880ca810 0x7f34880cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:15.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.237+0000 7f348e400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3488154820 0x7f348815cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:15.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.237+0000 7f348e400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f348815b0b0 con 0x7f34880ca810 2024-05-27T09:32:15.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.237+0000 7f348e400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f348815add0 con 0x7f3488154820 2024-05-27T09:32:15.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.237+0000 7f348e400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f348815b390 con 0x7f3488074a60 2024-05-27T09:32:15.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.237+0000 7f3486a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34880ca810 0x7f34880cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:15.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.237+0000 7f3486a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34880ca810 0x7f34880cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59468/0 (socket says 172.21.15.114:59468) 2024-05-27T09:32:15.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.237+0000 7f3486a00640 1 -- 172.21.15.114:0/976846495 learned_addr learned my addr 172.21.15.114:0/976846495 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:15.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.237+0000 7f3487400640 1 --2- 172.21.15.114:0/976846495 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3488074a60 0x7f34881e7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:15.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.237+0000 7f3487e00640 1 --2- 172.21.15.114:0/976846495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3488154820 0x7f348815cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:15.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.238+0000 7f3486a00640 1 -- 172.21.15.114:0/976846495 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3488074a60 msgr2=0x7f34881e7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:15.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.238+0000 7f3486a00640 1 --2- 172.21.15.114:0/976846495 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3488074a60 0x7f34881e7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:15.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.238+0000 7f3486a00640 1 -- 172.21.15.114:0/976846495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3488154820 msgr2=0x7f348815cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:15.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.238+0000 7f3486a00640 1 --2- 172.21.15.114:0/976846495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3488154820 0x7f348815cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:15.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.238+0000 7f3486a00640 1 -- 172.21.15.114:0/976846495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34880cd5a0 con 0x7f34880ca810 2024-05-27T09:32:15.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.238+0000 7f3486a00640 1 --2- 172.21.15.114:0/976846495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34880ca810 0x7f34880cd060 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3470000c00 tx=0x7f34700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:15.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.238+0000 7f347f400640 1 -- 172.21.15.114:0/976846495 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470041070 con 0x7f34880ca810 2024-05-27T09:32:15.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.238+0000 7f347f400640 1 -- 172.21.15.114:0/976846495 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f347003c660 con 0x7f34880ca810 2024-05-27T09:32:15.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.238+0000 7f348e400640 1 -- 172.21.15.114:0/976846495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34880cbc80 con 0x7f34880ca810 2024-05-27T09:32:15.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.238+0000 7f347f400640 1 -- 172.21.15.114:0/976846495 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470037040 con 0x7f34880ca810 2024-05-27T09:32:15.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.239+0000 7f348e400640 1 -- 172.21.15.114:0/976846495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34880cc580 con 0x7f34880ca810 2024-05-27T09:32:15.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.240+0000 7f348e400640 1 -- 172.21.15.114:0/976846495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f348806a380 con 0x7f34880ca810 2024-05-27T09:32:15.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.241+0000 7f347f400640 1 -- 172.21.15.114:0/976846495 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3470044070 con 0x7f34880ca810 2024-05-27T09:32:15.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.242+0000 7f347f400640 1 --2- 172.21.15.114:0/976846495 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f34640762d0 0x7f34640787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:15.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.242+0000 7f347f400640 1 -- 172.21.15.114:0/976846495 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5982+0+0 (secure 0 0 0) 0x7f34700b7ea0 con 0x7f34880ca810 2024-05-27T09:32:15.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.242+0000 7f3487400640 1 --2- 172.21.15.114:0/976846495 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f34640762d0 0x7f34640787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:15.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.243+0000 7f3487400640 1 --2- 172.21.15.114:0/976846495 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f34640762d0 0x7f34640787e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3478000f60 tx=0x7f3478000bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:15.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.247+0000 7f347f400640 1 -- 172.21.15.114:0/976846495 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3470081090 con 0x7f34880ca810 2024-05-27T09:32:15.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.510+0000 7f348e400640 1 -- 172.21.15.114:0/976846495 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34880c4d10 con 0x7f34880ca810 2024-05-27T09:32:15.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.511+0000 7f347f400640 1 -- 172.21.15.114:0/976846495 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+6883 (secure 0 0 0) 0x7f3470085390 con 0x7f34880ca810 2024-05-27T09:32:15.511 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:32:15.511 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4875,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:31:20.888182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4828,4300,4297]},"id":1}]} 2024-05-27T09:32:15.512 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 24 2024-05-27T09:32:15.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.514+0000 7f348e400640 1 -- 172.21.15.114:0/976846495 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f34640762d0 msgr2=0x7f34640787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:15.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.514+0000 7f348e400640 1 --2- 172.21.15.114:0/976846495 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f34640762d0 0x7f34640787e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3478000f60 tx=0x7f3478000bc0 comp rx=0 tx=0).stop 2024-05-27T09:32:15.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.514+0000 7f348e400640 1 -- 172.21.15.114:0/976846495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34880ca810 msgr2=0x7f34880cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:15.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.514+0000 7f348e400640 1 --2- 172.21.15.114:0/976846495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34880ca810 0x7f34880cd060 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3470000c00 tx=0x7f34700046c0 comp rx=0 tx=0).stop 2024-05-27T09:32:15.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.514+0000 7f348e400640 1 -- 172.21.15.114:0/976846495 shutdown_connections 2024-05-27T09:32:15.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.514+0000 7f348e400640 1 --2- 172.21.15.114:0/976846495 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f34640762d0 0x7f34640787e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:15.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.514+0000 7f348e400640 1 --2- 172.21.15.114:0/976846495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3488154820 0x7f348815cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:15.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.514+0000 7f348e400640 1 --2- 172.21.15.114:0/976846495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f34880ca810 0x7f34880cd060 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:15.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.514+0000 7f348e400640 1 --2- 172.21.15.114:0/976846495 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3488074a60 0x7f34881e7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:15.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.514+0000 7f348e400640 1 -- 172.21.15.114:0/976846495 >> 172.21.15.114:0/976846495 conn(0x7f348806f6e0 msgr2=0x7f3488073240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:15.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.514+0000 7f348e400640 1 -- 172.21.15.114:0/976846495 shutdown_connections 2024-05-27T09:32:15.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.514+0000 7f348e400640 1 -- 172.21.15.114:0/976846495 wait complete. 2024-05-27T09:32:15.540 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=1) 2024-05-27T09:32:15.540 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-27T09:32:15.540 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:32:15.598 INFO:tasks.ceph.mds.c:Stopped 2024-05-27T09:32:15.598 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:32:15.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.874+0000 7fc565000640 1 Processor -- start 2024-05-27T09:32:15.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.874+0000 7fc565000640 1 -- start start 2024-05-27T09:32:15.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.876+0000 7fc565000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc560074a80 0x7fc5600c8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:15.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.876+0000 7fc565000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc5600c9dd0 0x7fc5600c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:15.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.876+0000 7fc565000640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5600c8470 con 0x7fc56015ad10 2024-05-27T09:32:15.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.876+0000 7fc565000640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fc5600c8220 con 0x7fc5600c9dd0 2024-05-27T09:32:15.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.876+0000 7fc565000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fc5600c91d0 con 0x7fc560074a80 2024-05-27T09:32:15.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.876+0000 7fc55fe00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc5600c9dd0 0x7fc5600c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:15.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.876+0000 7fc55f400640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc56015ad10 0x7fc5600c96c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:48952/0 (socket says 172.21.15.114:48952) 2024-05-27T09:32:15.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.876+0000 7fc55f400640 1 -- 172.21.15.114:0/1317456663 learned_addr learned my addr 172.21.15.114:0/1317456663 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:32:15.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.876+0000 7fc556a00640 1 --2- 172.21.15.114:0/1317456663 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc560074a80 0x7fc5600c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:15.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.877+0000 7fc55ea00640 1 -- 172.21.15.114:0/1317456663 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2314217265 0 0) 0x7fc5600c8470 con 0x7fc56015ad10 2024-05-27T09:32:15.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.877+0000 7fc55ea00640 1 -- 172.21.15.114:0/1317456663 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc544003520 con 0x7fc56015ad10 2024-05-27T09:32:15.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.877+0000 7fc55fe00640 1 -- 172.21.15.114:0/1317456663 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc560074a80 msgr2=0x7fc5600c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:32:15.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.877+0000 7fc55fe00640 1 --2- 172.21.15.114:0/1317456663 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc560074a80 0x7fc5600c8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:15.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.877+0000 7fc55fe00640 1 -- 172.21.15.114:0/1317456663 >> v1:172.21.15.163:6789/0 conn(0x7fc56015ad10 legacy=0x7fc5600c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:15.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.877+0000 7fc55fe00640 1 -- 172.21.15.114:0/1317456663 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc56006ef90 con 0x7fc5600c9dd0 2024-05-27T09:32:15.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.877+0000 7fc556a00640 1 --2- 172.21.15.114:0/1317456663 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc560074a80 0x7fc5600c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:32:15.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.878+0000 7fc55fe00640 1 --2- 172.21.15.114:0/1317456663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5600c9dd0 0x7fc5600c8c90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc5580097a0 tx=0x7fc55802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df6c6e2c11bc1408 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:15.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.878+0000 7fc55ea00640 1 -- 172.21.15.114:0/1317456663 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc558037070 con 0x7fc5600c9dd0 2024-05-27T09:32:15.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.878+0000 7fc55ea00640 1 -- 172.21.15.114:0/1317456663 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc558032440 con 0x7fc5600c9dd0 2024-05-27T09:32:15.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.878+0000 7fc55ea00640 1 -- 172.21.15.114:0/1317456663 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5580327f0 con 0x7fc5600c9dd0 2024-05-27T09:32:15.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.879+0000 7fc565000640 1 -- 172.21.15.114:0/1317456663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5600c9dd0 msgr2=0x7fc5600c8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:15.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.879+0000 7fc565000640 1 --2- 172.21.15.114:0/1317456663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5600c9dd0 0x7fc5600c8c90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc5580097a0 tx=0x7fc55802a210 comp rx=0 tx=0).stop 2024-05-27T09:32:15.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.879+0000 7fc565000640 1 -- 172.21.15.114:0/1317456663 shutdown_connections 2024-05-27T09:32:15.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.879+0000 7fc565000640 1 --2- 172.21.15.114:0/1317456663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5600c9dd0 0x7fc5600c8c90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:15.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.879+0000 7fc565000640 1 --2- 172.21.15.114:0/1317456663 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc560074a80 0x7fc5600c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:15.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.879+0000 7fc565000640 1 -- 172.21.15.114:0/1317456663 >> 172.21.15.114:0/1317456663 conn(0x7fc5600ca320 msgr2=0x7fc5600ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:15.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.879+0000 7fc565000640 1 -- 172.21.15.114:0/1317456663 shutdown_connections 2024-05-27T09:32:15.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.879+0000 7fc565000640 1 -- 172.21.15.114:0/1317456663 wait complete. 2024-05-27T09:32:15.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.881+0000 7fc565000640 1 Processor -- start 2024-05-27T09:32:15.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.882+0000 7fc565000640 1 -- start start 2024-05-27T09:32:15.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.883+0000 7fc565000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc560074a80 0x7fc56014d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:15.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.883+0000 7fc565000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5600c9dd0 0x7fc5601f9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:15.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.883+0000 7fc565000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc56015ad10 0x7fc5601fa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:15.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.883+0000 7fc565000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc56006f830 con 0x7fc560074a80 2024-05-27T09:32:15.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.883+0000 7fc565000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc56006f550 con 0x7fc5600c9dd0 2024-05-27T09:32:15.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.883+0000 7fc565000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fc560070060 con 0x7fc56015ad10 2024-05-27T09:32:15.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.883+0000 7fc55fe00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc56015ad10 0x7fc5601fa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:15.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.883+0000 7fc556a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5600c9dd0 0x7fc5601f9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:15.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.883+0000 7fc55fe00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc56015ad10 0x7fc5601fa1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:44986/0 (socket says 172.21.15.114:44986) 2024-05-27T09:32:15.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.883+0000 7fc556a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5600c9dd0 0x7fc5601f9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:52336/0 (socket says 172.21.15.114:52336) 2024-05-27T09:32:15.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.883+0000 7fc55fe00640 1 -- 172.21.15.114:0/1649385754 learned_addr learned my addr 172.21.15.114:0/1649385754 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:15.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.883+0000 7fc55f400640 1 --2- 172.21.15.114:0/1649385754 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc560074a80 0x7fc56014d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:15.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.884+0000 7fc55fe00640 1 -- 172.21.15.114:0/1649385754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5600c9dd0 msgr2=0x7fc5601f9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:15.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.884+0000 7fc55fe00640 1 --2- 172.21.15.114:0/1649385754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5600c9dd0 0x7fc5601f9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:15.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.884+0000 7fc55fe00640 1 -- 172.21.15.114:0/1649385754 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc560074a80 msgr2=0x7fc56014d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:15.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.884+0000 7fc55fe00640 1 --2- 172.21.15.114:0/1649385754 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc560074a80 0x7fc56014d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:15.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.884+0000 7fc55fe00640 1 -- 172.21.15.114:0/1649385754 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5601fa880 con 0x7fc56015ad10 2024-05-27T09:32:15.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.884+0000 7fc55fe00640 1 --2- 172.21.15.114:0/1649385754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc56015ad10 0x7fc5601fa1a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc5580097a0 tx=0x7fc5580046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:15.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.884+0000 7fc557e00640 1 -- 172.21.15.114:0/1649385754 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc558037050 con 0x7fc56015ad10 2024-05-27T09:32:15.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.885+0000 7fc557e00640 1 -- 172.21.15.114:0/1649385754 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc558038040 con 0x7fc56015ad10 2024-05-27T09:32:15.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.885+0000 7fc557e00640 1 -- 172.21.15.114:0/1649385754 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc558004450 con 0x7fc56015ad10 2024-05-27T09:32:15.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.885+0000 7fc565000640 1 -- 172.21.15.114:0/1649385754 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc56014d5d0 con 0x7fc56015ad10 2024-05-27T09:32:15.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.885+0000 7fc565000640 1 -- 172.21.15.114:0/1649385754 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5600c7770 con 0x7fc56015ad10 2024-05-27T09:32:15.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.886+0000 7fc565000640 1 -- 172.21.15.114:0/1649385754 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc560050860 con 0x7fc56015ad10 2024-05-27T09:32:15.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.887+0000 7fc557e00640 1 -- 172.21.15.114:0/1649385754 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc55803c6d0 con 0x7fc56015ad10 2024-05-27T09:32:15.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.888+0000 7fc557e00640 1 --2- 172.21.15.114:0/1649385754 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc53c0762d0 0x7fc53c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:15.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.888+0000 7fc557e00640 1 -- 172.21.15.114:0/1649385754 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(33..33 src has 1..33) ==== 5982+0+0 (secure 0 0 0) 0x7fc5580b8b60 con 0x7fc56015ad10 2024-05-27T09:32:15.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.888+0000 7fc55f400640 1 --2- 172.21.15.114:0/1649385754 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc53c0762d0 0x7fc53c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:15.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.889+0000 7fc55f400640 1 --2- 172.21.15.114:0/1649385754 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc53c0762d0 0x7fc53c0787e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc550005880 tx=0x7fc550006aa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:15.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:15.892+0000 7fc557e00640 1 -- 172.21.15.114:0/1649385754 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc558081dd0 con 0x7fc56015ad10 2024-05-27T09:32:16.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:16.140+0000 7fc565000640 1 -- 172.21.15.114:0/1649385754 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc560074860 con 0x7fc56015ad10 2024-05-27T09:32:16.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:16.141+0000 7fc557e00640 1 -- 172.21.15.114:0/1649385754 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+6883 (secure 0 0 0) 0x7fc5580860d0 con 0x7fc56015ad10 2024-05-27T09:32:16.142 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:32:16.142 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4875,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:31:20.888182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4828,4300,4297]},"id":1}]} 2024-05-27T09:32:16.142 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 24 2024-05-27T09:32:16.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:16.143+0000 7fc565000640 1 -- 172.21.15.114:0/1649385754 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc53c0762d0 msgr2=0x7fc53c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:16.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:16.143+0000 7fc565000640 1 --2- 172.21.15.114:0/1649385754 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc53c0762d0 0x7fc53c0787e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc550005880 tx=0x7fc550006aa0 comp rx=0 tx=0).stop 2024-05-27T09:32:16.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:16.143+0000 7fc565000640 1 -- 172.21.15.114:0/1649385754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc56015ad10 msgr2=0x7fc5601fa1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:16.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:16.143+0000 7fc565000640 1 --2- 172.21.15.114:0/1649385754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc56015ad10 0x7fc5601fa1a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc5580097a0 tx=0x7fc5580046c0 comp rx=0 tx=0).stop 2024-05-27T09:32:16.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:16.143+0000 7fc565000640 1 -- 172.21.15.114:0/1649385754 shutdown_connections 2024-05-27T09:32:16.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:16.143+0000 7fc565000640 1 --2- 172.21.15.114:0/1649385754 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc53c0762d0 0x7fc53c0787e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:16.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:16.143+0000 7fc565000640 1 --2- 172.21.15.114:0/1649385754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc56015ad10 0x7fc5601fa1a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:16.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:16.143+0000 7fc565000640 1 --2- 172.21.15.114:0/1649385754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5600c9dd0 0x7fc5601f9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:16.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:16.143+0000 7fc565000640 1 --2- 172.21.15.114:0/1649385754 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc560074a80 0x7fc56014d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:16.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:16.143+0000 7fc565000640 1 -- 172.21.15.114:0/1649385754 >> 172.21.15.114:0/1649385754 conn(0x7fc5600ca320 msgr2=0x7fc5600b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:16.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:16.144+0000 7fc565000640 1 -- 172.21.15.114:0/1649385754 shutdown_connections 2024-05-27T09:32:16.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:16.144+0000 7fc565000640 1 -- 172.21.15.114:0/1649385754 wait complete. 2024-05-27T09:32:16.171 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-05-27T09:32:18.172 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:32:18.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.454+0000 7ff102e00640 1 Processor -- start 2024-05-27T09:32:18.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.454+0000 7ff102e00640 1 -- start start 2024-05-27T09:32:18.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.456+0000 7ff102e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0fc0741f0 0x7ff0fc0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:18.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.456+0000 7ff102e00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0fc06fe50 con 0x7ff0fc1547d0 2024-05-27T09:32:18.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.456+0000 7ff102e00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0fc0700f0 con 0x7ff0fc074840 2024-05-27T09:32:18.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.456+0000 7ff102e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7ff0fc0703d0 con 0x7ff0fc0741f0 2024-05-27T09:32:18.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.457+0000 7ff102400640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff0fc074840 0x7ff0fc164b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:48964/0 (socket says 172.21.15.114:48964) 2024-05-27T09:32:18.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.457+0000 7ff102400640 1 -- 172.21.15.114:0/2608671281 learned_addr learned my addr 172.21.15.114:0/2608671281 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:32:18.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.457+0000 7ff101a00640 1 --2- 172.21.15.114:0/2608671281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0fc0741f0 0x7ff0fc0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:18.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.457+0000 7ff101000640 1 --1- 172.21.15.114:0/2608671281 >> v1:172.21.15.114:6789/0 conn(0x7ff0fc1547d0 0x7ff0fc06edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:42544/0 (socket says 172.21.15.114:42544) 2024-05-27T09:32:18.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.457+0000 7ff0fbe00640 1 -- 172.21.15.114:0/2608671281 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 691679262 0 0) 0x7ff0fc0700f0 con 0x7ff0fc074840 2024-05-27T09:32:18.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.457+0000 7ff101a00640 1 -- 172.21.15.114:0/2608671281 >> v1:172.21.15.163:6789/0 conn(0x7ff0fc074840 legacy=0x7ff0fc164b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:18.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.457+0000 7ff101a00640 1 -- 172.21.15.114:0/2608671281 >> v1:172.21.15.114:6789/0 conn(0x7ff0fc1547d0 legacy=0x7ff0fc06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:18.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.457+0000 7ff101a00640 1 -- 172.21.15.114:0/2608671281 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0fc0731d0 con 0x7ff0fc0741f0 2024-05-27T09:32:18.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.458+0000 7ff101a00640 1 --2- 172.21.15.114:0/2608671281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0fc0741f0 0x7ff0fc0c63a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00097d0 tx=0x7ff0f002a2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=167265de8732a67e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:18.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.459+0000 7ff0fbe00640 1 -- 172.21.15.114:0/2608671281 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0037070 con 0x7ff0fc0741f0 2024-05-27T09:32:18.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.459+0000 7ff0fbe00640 1 -- 172.21.15.114:0/2608671281 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f0032440 con 0x7ff0fc0741f0 2024-05-27T09:32:18.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.459+0000 7ff0fbe00640 1 -- 172.21.15.114:0/2608671281 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0032810 con 0x7ff0fc0741f0 2024-05-27T09:32:18.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.459+0000 7ff102e00640 1 -- 172.21.15.114:0/2608671281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0fc0741f0 msgr2=0x7ff0fc0c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:18.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.459+0000 7ff102e00640 1 --2- 172.21.15.114:0/2608671281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0fc0741f0 0x7ff0fc0c63a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00097d0 tx=0x7ff0f002a2a0 comp rx=0 tx=0).stop 2024-05-27T09:32:18.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.459+0000 7ff102e00640 1 -- 172.21.15.114:0/2608671281 shutdown_connections 2024-05-27T09:32:18.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.459+0000 7ff102e00640 1 --2- 172.21.15.114:0/2608671281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0fc0741f0 0x7ff0fc0c63a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:18.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.459+0000 7ff102e00640 1 -- 172.21.15.114:0/2608671281 >> 172.21.15.114:0/2608671281 conn(0x7ff0fc0ca320 msgr2=0x7ff0fc0ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:18.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.460+0000 7ff102e00640 1 -- 172.21.15.114:0/2608671281 shutdown_connections 2024-05-27T09:32:18.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.460+0000 7ff102e00640 1 -- 172.21.15.114:0/2608671281 wait complete. 2024-05-27T09:32:18.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.462+0000 7ff102e00640 1 Processor -- start 2024-05-27T09:32:18.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.462+0000 7ff102e00640 1 -- start start 2024-05-27T09:32:18.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.463+0000 7ff102e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0fc0741f0 0x7ff0fc0af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:18.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.463+0000 7ff102e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0fc074840 0x7ff0fc15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:18.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.463+0000 7ff102e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0fc1547d0 0x7ff0fc1582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:18.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.463+0000 7ff102e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc06f500 con 0x7ff0fc1547d0 2024-05-27T09:32:18.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.463+0000 7ff102e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc073790 con 0x7ff0fc0741f0 2024-05-27T09:32:18.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.463+0000 7ff102e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7ff0fc06f680 con 0x7ff0fc074840 2024-05-27T09:32:18.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.464+0000 7ff102400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0fc1547d0 0x7ff0fc1582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:18.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.464+0000 7ff102400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0fc1547d0 0x7ff0fc1582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59490/0 (socket says 172.21.15.114:59490) 2024-05-27T09:32:18.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.464+0000 7ff102400640 1 -- 172.21.15.114:0/1104062360 learned_addr learned my addr 172.21.15.114:0/1104062360 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:18.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.464+0000 7ff101a00640 1 --2- 172.21.15.114:0/1104062360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0fc0741f0 0x7ff0fc0af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:18.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.464+0000 7ff101000640 1 --2- 172.21.15.114:0/1104062360 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0fc074840 0x7ff0fc15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:18.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.464+0000 7ff101a00640 1 -- 172.21.15.114:0/1104062360 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0fc074840 msgr2=0x7ff0fc15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:18.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.464+0000 7ff101a00640 1 --2- 172.21.15.114:0/1104062360 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0fc074840 0x7ff0fc15bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:18.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.464+0000 7ff101a00640 1 -- 172.21.15.114:0/1104062360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0fc1547d0 msgr2=0x7ff0fc1582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:18.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.464+0000 7ff101a00640 1 --2- 172.21.15.114:0/1104062360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0fc1547d0 0x7ff0fc1582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:18.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.464+0000 7ff101a00640 1 -- 172.21.15.114:0/1104062360 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0fc15b810 con 0x7ff0fc0741f0 2024-05-27T09:32:18.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.465+0000 7ff101a00640 1 --2- 172.21.15.114:0/1104062360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0fc0741f0 0x7ff0fc0af240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0031000 tx=0x7ff0f00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:18.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.465+0000 7ff0f9600640 1 -- 172.21.15.114:0/1104062360 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0041070 con 0x7ff0fc0741f0 2024-05-27T09:32:18.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.465+0000 7ff0f9600640 1 -- 172.21.15.114:0/1104062360 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f003c8b0 con 0x7ff0fc0741f0 2024-05-27T09:32:18.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.465+0000 7ff0f9600640 1 -- 172.21.15.114:0/1104062360 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0037040 con 0x7ff0fc0741f0 2024-05-27T09:32:18.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.465+0000 7ff102e00640 1 -- 172.21.15.114:0/1104062360 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0fc15d250 con 0x7ff0fc0741f0 2024-05-27T09:32:18.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.465+0000 7ff102e00640 1 -- 172.21.15.114:0/1104062360 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0fc15d810 con 0x7ff0fc0741f0 2024-05-27T09:32:18.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.466+0000 7ff102e00640 1 -- 172.21.15.114:0/1104062360 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0fc070680 con 0x7ff0fc0741f0 2024-05-27T09:32:18.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.468+0000 7ff0f9600640 1 -- 172.21.15.114:0/1104062360 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff0f0044070 con 0x7ff0fc0741f0 2024-05-27T09:32:18.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.469+0000 7ff0f9600640 1 --2- 172.21.15.114:0/1104062360 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff0dc0762d0 0x7ff0dc0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:18.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.469+0000 7ff0f9600640 1 -- 172.21.15.114:0/1104062360 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5982+0+0 (secure 0 0 0) 0x7ff0f00b8840 con 0x7ff0fc0741f0 2024-05-27T09:32:18.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.469+0000 7ff101000640 1 --2- 172.21.15.114:0/1104062360 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff0dc0762d0 0x7ff0dc0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:18.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.470+0000 7ff101000640 1 --2- 172.21.15.114:0/1104062360 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff0dc0762d0 0x7ff0dc0787e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8000f60 tx=0x7ff0e8007e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:18.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.473+0000 7ff0f9600640 1 -- 172.21.15.114:0/1104062360 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff0f0081a20 con 0x7ff0fc0741f0 2024-05-27T09:32:18.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.717+0000 7ff102e00640 1 -- 172.21.15.114:0/1104062360 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0fc15aee0 con 0x7ff0fc0741f0 2024-05-27T09:32:18.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.719+0000 7ff0f9600640 1 -- 172.21.15.114:0/1104062360 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+6883 (secure 0 0 0) 0x7ff0f0085d20 con 0x7ff0fc0741f0 2024-05-27T09:32:18.719 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:32:18.720 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4875,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:31:20.888182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4828,4300,4297]},"id":1}]} 2024-05-27T09:32:18.720 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 24 2024-05-27T09:32:18.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.721+0000 7ff102e00640 1 -- 172.21.15.114:0/1104062360 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff0dc0762d0 msgr2=0x7ff0dc0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:18.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.721+0000 7ff102e00640 1 --2- 172.21.15.114:0/1104062360 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff0dc0762d0 0x7ff0dc0787e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8000f60 tx=0x7ff0e8007e40 comp rx=0 tx=0).stop 2024-05-27T09:32:18.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.721+0000 7ff102e00640 1 -- 172.21.15.114:0/1104062360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0fc0741f0 msgr2=0x7ff0fc0af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:18.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.721+0000 7ff102e00640 1 --2- 172.21.15.114:0/1104062360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0fc0741f0 0x7ff0fc0af240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0031000 tx=0x7ff0f00046c0 comp rx=0 tx=0).stop 2024-05-27T09:32:18.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.722+0000 7ff102e00640 1 -- 172.21.15.114:0/1104062360 shutdown_connections 2024-05-27T09:32:18.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.722+0000 7ff102e00640 1 --2- 172.21.15.114:0/1104062360 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff0dc0762d0 0x7ff0dc0787e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:18.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.722+0000 7ff102e00640 1 --2- 172.21.15.114:0/1104062360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0fc1547d0 0x7ff0fc1582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:18.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.722+0000 7ff102e00640 1 --2- 172.21.15.114:0/1104062360 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0fc074840 0x7ff0fc15bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:18.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.722+0000 7ff102e00640 1 --2- 172.21.15.114:0/1104062360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0fc0741f0 0x7ff0fc0af240 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:18.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.722+0000 7ff102e00640 1 -- 172.21.15.114:0/1104062360 >> 172.21.15.114:0/1104062360 conn(0x7ff0fc0ca320 msgr2=0x7ff0fc159ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:18.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.722+0000 7ff102e00640 1 -- 172.21.15.114:0/1104062360 shutdown_connections 2024-05-27T09:32:18.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:18.722+0000 7ff102e00640 1 -- 172.21.15.114:0/1104062360 wait complete. 2024-05-27T09:32:18.746 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-05-27T09:32:18.811 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:32:18.819 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:32:20.747 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:32:20.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:20.997+0000 7f1c46c00640 1 Processor -- start 2024-05-27T09:32:20.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:20.997+0000 7f1c46c00640 1 -- start start 2024-05-27T09:32:21.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:20.999+0000 7f1c46c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c401534c0 0x7f1c400c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:21.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:20.999+0000 7f1c46c00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1c400c8220 0x7f1c40159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:21.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.000+0000 7f1c46c00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c400c96d0 con 0x7f1c400c8630 2024-05-27T09:32:21.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.000+0000 7f1c46c00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f1c400c2f40 con 0x7f1c400c8220 2024-05-27T09:32:21.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.000+0000 7f1c46c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f1c40159190 con 0x7f1c401534c0 2024-05-27T09:32:21.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.000+0000 7f1c44e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1c400c8220 0x7f1c40159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:21.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.000+0000 7f1c46200640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f1c400c8630 0x7f1c40074140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:42548/0 (socket says 172.21.15.114:42548) 2024-05-27T09:32:21.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.000+0000 7f1c46200640 1 -- 172.21.15.114:0/328580273 learned_addr learned my addr 172.21.15.114:0/328580273 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:32:21.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.000+0000 7f1c45800640 1 --2- 172.21.15.114:0/328580273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c401534c0 0x7f1c400c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:21.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.000+0000 7f1c44e00640 1 -- 172.21.15.114:0/328580273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c401534c0 msgr2=0x7f1c400c2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:32:21.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.000+0000 7f1c44e00640 1 --2- 172.21.15.114:0/328580273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c401534c0 0x7f1c400c2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.000+0000 7f1c44e00640 1 -- 172.21.15.114:0/328580273 >> v1:172.21.15.114:6789/0 conn(0x7f1c400c8630 legacy=0x7f1c40074140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:21.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.000+0000 7f1c3fe00640 1 -- 172.21.15.114:0/328580273 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3680367051 0 0) 0x7f1c400c96d0 con 0x7f1c400c8630 2024-05-27T09:32:21.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.001+0000 7f1c44e00640 1 -- 172.21.15.114:0/328580273 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c400c9970 con 0x7f1c400c8220 2024-05-27T09:32:21.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.002+0000 7f1c44e00640 1 --2- 172.21.15.114:0/328580273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c400c8220 0x7f1c40159900 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0097a0 tx=0x7f1c2c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedaa38dace81412 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:21.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.002+0000 7f1c3fe00640 1 -- 172.21.15.114:0/328580273 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c037070 con 0x7f1c400c8220 2024-05-27T09:32:21.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.002+0000 7f1c3fe00640 1 -- 172.21.15.114:0/328580273 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c2c032440 con 0x7f1c400c8220 2024-05-27T09:32:21.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.002+0000 7f1c3fe00640 1 -- 172.21.15.114:0/328580273 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c032810 con 0x7f1c400c8220 2024-05-27T09:32:21.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.003+0000 7f1c46c00640 1 -- 172.21.15.114:0/328580273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c400c8220 msgr2=0x7f1c40159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:21.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.003+0000 7f1c46c00640 1 --2- 172.21.15.114:0/328580273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c400c8220 0x7f1c40159900 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0097a0 tx=0x7f1c2c02a180 comp rx=0 tx=0).stop 2024-05-27T09:32:21.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.003+0000 7f1c46c00640 1 -- 172.21.15.114:0/328580273 shutdown_connections 2024-05-27T09:32:21.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.003+0000 7f1c46c00640 1 --2- 172.21.15.114:0/328580273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c400c8220 0x7f1c40159900 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.003+0000 7f1c46c00640 1 --2- 172.21.15.114:0/328580273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c401534c0 0x7f1c400c2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.003+0000 7f1c46c00640 1 -- 172.21.15.114:0/328580273 >> 172.21.15.114:0/328580273 conn(0x7f1c400ca320 msgr2=0x7f1c400ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:21.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.003+0000 7f1c46c00640 1 -- 172.21.15.114:0/328580273 shutdown_connections 2024-05-27T09:32:21.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.003+0000 7f1c46c00640 1 -- 172.21.15.114:0/328580273 wait complete. 2024-05-27T09:32:21.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.007+0000 7f1c46c00640 1 Processor -- start 2024-05-27T09:32:21.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.007+0000 7f1c46c00640 1 -- start start 2024-05-27T09:32:21.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.009+0000 7f1c46c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c400c8220 0x7f1c401f0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:21.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.009+0000 7f1c46c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c400c8630 0x7f1c401e1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:21.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.009+0000 7f1c46c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c401534c0 0x7f1c4014e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:21.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.009+0000 7f1c46c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1c400c9f30 con 0x7f1c400c8220 2024-05-27T09:32:21.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.009+0000 7f1c46c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1c401538d0 con 0x7f1c401534c0 2024-05-27T09:32:21.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.009+0000 7f1c46c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f1c401593e0 con 0x7f1c400c8630 2024-05-27T09:32:21.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.010+0000 7f1c45800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c400c8220 0x7f1c401f0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:21.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.010+0000 7f1c45800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c400c8220 0x7f1c401f0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59500/0 (socket says 172.21.15.114:59500) 2024-05-27T09:32:21.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.010+0000 7f1c45800640 1 -- 172.21.15.114:0/252851068 learned_addr learned my addr 172.21.15.114:0/252851068 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:21.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.010+0000 7f1c46200640 1 --2- 172.21.15.114:0/252851068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c401534c0 0x7f1c4014e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:21.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.010+0000 7f1c44e00640 1 --2- 172.21.15.114:0/252851068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c400c8630 0x7f1c401e1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:21.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.010+0000 7f1c45800640 1 -- 172.21.15.114:0/252851068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c400c8630 msgr2=0x7f1c401e1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:21.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.010+0000 7f1c45800640 1 --2- 172.21.15.114:0/252851068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c400c8630 0x7f1c401e1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.010+0000 7f1c45800640 1 -- 172.21.15.114:0/252851068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c401534c0 msgr2=0x7f1c4014e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:21.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.010+0000 7f1c45800640 1 --2- 172.21.15.114:0/252851068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c401534c0 0x7f1c4014e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.010+0000 7f1c45800640 1 -- 172.21.15.114:0/252851068 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c400c7560 con 0x7f1c400c8220 2024-05-27T09:32:21.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.010+0000 7f1c45800640 1 --2- 172.21.15.114:0/252851068 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c400c8220 0x7f1c401f0720 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1c3400e8a0 tx=0x7f1c3400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:21.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.011+0000 7f1c3d600640 1 -- 172.21.15.114:0/252851068 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c34017070 con 0x7f1c400c8220 2024-05-27T09:32:21.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.011+0000 7f1c3d600640 1 -- 172.21.15.114:0/252851068 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c3400c940 con 0x7f1c400c8220 2024-05-27T09:32:21.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.011+0000 7f1c3d600640 1 -- 172.21.15.114:0/252851068 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3400cd30 con 0x7f1c400c8220 2024-05-27T09:32:21.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.011+0000 7f1c46c00640 1 -- 172.21.15.114:0/252851068 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c401e2c80 con 0x7f1c400c8220 2024-05-27T09:32:21.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.011+0000 7f1c46c00640 1 -- 172.21.15.114:0/252851068 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c401f4190 con 0x7f1c400c8220 2024-05-27T09:32:21.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.012+0000 7f1c46c00640 1 -- 172.21.15.114:0/252851068 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c401f3cc0 con 0x7f1c400c8220 2024-05-27T09:32:21.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.013+0000 7f1c3d600640 1 -- 172.21.15.114:0/252851068 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1c3401a070 con 0x7f1c400c8220 2024-05-27T09:32:21.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.014+0000 7f1c3d600640 1 --2- 172.21.15.114:0/252851068 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1c20076300 0x7f1c20078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:21.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.014+0000 7f1c3d600640 1 -- 172.21.15.114:0/252851068 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5954+0+0 (secure 0 0 0) 0x7f1c340975a0 con 0x7f1c400c8220 2024-05-27T09:32:21.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.014+0000 7f1c44e00640 1 --2- 172.21.15.114:0/252851068 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1c20076300 0x7f1c20078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:21.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.015+0000 7f1c44e00640 1 --2- 172.21.15.114:0/252851068 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1c20076300 0x7f1c20078810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c034000 tx=0x7f1c2c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:21.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.019+0000 7f1c3d600640 1 -- 172.21.15.114:0/252851068 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1c34060680 con 0x7f1c400c8220 2024-05-27T09:32:21.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.267+0000 7f1c46c00640 1 -- 172.21.15.114:0/252851068 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c40050860 con 0x7f1c400c8220 2024-05-27T09:32:21.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.268+0000 7f1c3d600640 1 -- 172.21.15.114:0/252851068 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+6883 (secure 0 0 0) 0x7f1c34064980 con 0x7f1c400c8220 2024-05-27T09:32:21.268 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:32:21.268 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4875,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:31:20.888182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4828,4300,4297]},"id":1}]} 2024-05-27T09:32:21.269 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 24 2024-05-27T09:32:21.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.270+0000 7f1c46c00640 1 -- 172.21.15.114:0/252851068 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1c20076300 msgr2=0x7f1c20078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:21.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.270+0000 7f1c46c00640 1 --2- 172.21.15.114:0/252851068 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1c20076300 0x7f1c20078810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c034000 tx=0x7f1c2c02e000 comp rx=0 tx=0).stop 2024-05-27T09:32:21.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.270+0000 7f1c46c00640 1 -- 172.21.15.114:0/252851068 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c400c8220 msgr2=0x7f1c401f0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:21.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.270+0000 7f1c46c00640 1 --2- 172.21.15.114:0/252851068 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c400c8220 0x7f1c401f0720 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1c3400e8a0 tx=0x7f1c3400eca0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.271+0000 7f1c46c00640 1 -- 172.21.15.114:0/252851068 shutdown_connections 2024-05-27T09:32:21.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.271+0000 7f1c46c00640 1 --2- 172.21.15.114:0/252851068 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1c20076300 0x7f1c20078810 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.271+0000 7f1c46c00640 1 --2- 172.21.15.114:0/252851068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c401534c0 0x7f1c4014e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.271+0000 7f1c46c00640 1 --2- 172.21.15.114:0/252851068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c400c8630 0x7f1c401e1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.271+0000 7f1c46c00640 1 --2- 172.21.15.114:0/252851068 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c400c8220 0x7f1c401f0720 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.271+0000 7f1c46c00640 1 -- 172.21.15.114:0/252851068 >> 172.21.15.114:0/252851068 conn(0x7f1c400ca320 msgr2=0x7f1c400761b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:21.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.271+0000 7f1c46c00640 1 -- 172.21.15.114:0/252851068 shutdown_connections 2024-05-27T09:32:21.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:21.271+0000 7f1c46c00640 1 -- 172.21.15.114:0/252851068 wait complete. 2024-05-27T09:32:21.297 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-05-27T09:32:23.297 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:32:23.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.581+0000 7f5c50a00640 1 Processor -- start 2024-05-27T09:32:23.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.582+0000 7f5c50a00640 1 -- start start 2024-05-27T09:32:23.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.584+0000 7f5c50a00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5c4c164db0 0x7f5c4c0c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:23.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.584+0000 7f5c50a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5c4c160b40 0x7f5c4c160f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:23.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.584+0000 7f5c50a00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c4c1615d0 con 0x7f5c4c074860 2024-05-27T09:32:23.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.584+0000 7f5c50a00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f5c4c0cade0 con 0x7f5c4c160b40 2024-05-27T09:32:23.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.584+0000 7f5c50a00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f5c4c0c3d60 con 0x7f5c4c164db0 2024-05-27T09:32:23.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.584+0000 7f5c49e00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5c4c164db0 0x7f5c4c0c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:23.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.584+0000 7f5c4a800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5c4c160b40 0x7f5c4c160f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:23.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.584+0000 7f5c49400640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5c4c074860 0x7f5c4c15aac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:48980/0 (socket says 172.21.15.114:48980) 2024-05-27T09:32:23.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.584+0000 7f5c49400640 1 -- 172.21.15.114:0/3873854118 learned_addr learned my addr 172.21.15.114:0/3873854118 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:32:23.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.585+0000 7f5c49e00640 1 -- 172.21.15.114:0/3873854118 >> v2:172.21.15.114:3300/0 conn(0x7f5c4c160b40 msgr2=0x7f5c4c160f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:32:23.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.585+0000 7f5c49e00640 1 --2- 172.21.15.114:0/3873854118 >> v2:172.21.15.114:3300/0 conn(0x7f5c4c160b40 0x7f5c4c160f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:23.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.585+0000 7f5c49e00640 1 -- 172.21.15.114:0/3873854118 >> v1:172.21.15.163:6789/0 conn(0x7f5c4c074860 legacy=0x7f5c4c15aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:23.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.585+0000 7f5c48a00640 1 -- 172.21.15.114:0/3873854118 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3915554371 0 0) 0x7f5c4c1615d0 con 0x7f5c4c074860 2024-05-27T09:32:23.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.585+0000 7f5c49e00640 1 -- 172.21.15.114:0/3873854118 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4c161890 con 0x7f5c4c164db0 2024-05-27T09:32:23.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.585+0000 7f5c4a800640 1 --2- 172.21.15.114:0/3873854118 >> v2:172.21.15.114:3300/0 conn(0x7f5c4c160b40 0x7f5c4c160f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:32:23.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.586+0000 7f5c49e00640 1 --2- 172.21.15.114:0/3873854118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c4c164db0 0x7f5c4c0c3820 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0c3ee0 tx=0x7f5c3c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbe844832c36ed0b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:23.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.586+0000 7f5c48a00640 1 -- 172.21.15.114:0/3873854118 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c037070 con 0x7f5c4c164db0 2024-05-27T09:32:23.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.586+0000 7f5c48a00640 1 -- 172.21.15.114:0/3873854118 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c3c032440 con 0x7f5c4c164db0 2024-05-27T09:32:23.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.586+0000 7f5c48a00640 1 -- 172.21.15.114:0/3873854118 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c032810 con 0x7f5c4c164db0 2024-05-27T09:32:23.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.587+0000 7f5c50a00640 1 -- 172.21.15.114:0/3873854118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c4c164db0 msgr2=0x7f5c4c0c3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:23.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.587+0000 7f5c50a00640 1 --2- 172.21.15.114:0/3873854118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c4c164db0 0x7f5c4c0c3820 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0c3ee0 tx=0x7f5c3c02a250 comp rx=0 tx=0).stop 2024-05-27T09:32:23.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.587+0000 7f5c50a00640 1 -- 172.21.15.114:0/3873854118 shutdown_connections 2024-05-27T09:32:23.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.587+0000 7f5c50a00640 1 --2- 172.21.15.114:0/3873854118 >> v2:172.21.15.114:3300/0 conn(0x7f5c4c160b40 0x7f5c4c160f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:23.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.587+0000 7f5c50a00640 1 --2- 172.21.15.114:0/3873854118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c4c164db0 0x7f5c4c0c3820 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:23.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.587+0000 7f5c50a00640 1 -- 172.21.15.114:0/3873854118 >> 172.21.15.114:0/3873854118 conn(0x7f5c4c06f6e0 msgr2=0x7f5c4c070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:23.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.587+0000 7f5c50a00640 1 -- 172.21.15.114:0/3873854118 shutdown_connections 2024-05-27T09:32:23.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.587+0000 7f5c50a00640 1 -- 172.21.15.114:0/3873854118 wait complete. 2024-05-27T09:32:23.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.589+0000 7f5c50a00640 1 Processor -- start 2024-05-27T09:32:23.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.590+0000 7f5c50a00640 1 -- start start 2024-05-27T09:32:23.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.591+0000 7f5c50a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c4c074860 0x7f5c4c158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:23.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.591+0000 7f5c50a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c4c160b40 0x7f5c4c1ed2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:23.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.591+0000 7f5c50a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c4c164db0 0x7f5c4c1ed7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:23.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.591+0000 7f5c50a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c164890 con 0x7f5c4c164db0 2024-05-27T09:32:23.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.591+0000 7f5c50a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c1645b0 con 0x7f5c4c160b40 2024-05-27T09:32:23.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.591+0000 7f5c50a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f5c4c073310 con 0x7f5c4c074860 2024-05-27T09:32:23.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.591+0000 7f5c49400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c4c160b40 0x7f5c4c1ed2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:23.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.591+0000 7f5c4a800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c4c164db0 0x7f5c4c1ed7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:23.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.591+0000 7f5c49e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c4c074860 0x7f5c4c158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:23.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.591+0000 7f5c49400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c4c160b40 0x7f5c4c1ed2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:52364/0 (socket says 172.21.15.114:52364) 2024-05-27T09:32:23.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.591+0000 7f5c49400640 1 -- 172.21.15.114:0/1505665864 learned_addr learned my addr 172.21.15.114:0/1505665864 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:23.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.592+0000 7f5c49400640 1 -- 172.21.15.114:0/1505665864 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c4c074860 msgr2=0x7f5c4c158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:23.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.592+0000 7f5c49400640 1 --2- 172.21.15.114:0/1505665864 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c4c074860 0x7f5c4c158290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:23.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.592+0000 7f5c49400640 1 -- 172.21.15.114:0/1505665864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c4c164db0 msgr2=0x7f5c4c1ed7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:23.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.592+0000 7f5c49400640 1 --2- 172.21.15.114:0/1505665864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c4c164db0 0x7f5c4c1ed7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:23.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.592+0000 7f5c49400640 1 -- 172.21.15.114:0/1505665864 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4c151830 con 0x7f5c4c160b40 2024-05-27T09:32:23.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.592+0000 7f5c4a800640 1 --2- 172.21.15.114:0/1505665864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c4c164db0 0x7f5c4c1ed7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:32:23.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.592+0000 7f5c49400640 1 --2- 172.21.15.114:0/1505665864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c4c160b40 0x7f5c4c1ed2a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400c730 tx=0x7f5c3400cb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:23.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.593+0000 7f5c42000640 1 -- 172.21.15.114:0/1505665864 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c34018070 con 0x7f5c4c160b40 2024-05-27T09:32:23.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.593+0000 7f5c42000640 1 -- 172.21.15.114:0/1505665864 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c34004d80 con 0x7f5c4c160b40 2024-05-27T09:32:23.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.593+0000 7f5c42000640 1 -- 172.21.15.114:0/1505665864 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3400a820 con 0x7f5c4c160b40 2024-05-27T09:32:23.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.593+0000 7f5c50a00640 1 -- 172.21.15.114:0/1505665864 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c4c1f7890 con 0x7f5c4c160b40 2024-05-27T09:32:23.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.593+0000 7f5c50a00640 1 -- 172.21.15.114:0/1505665864 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c4c151310 con 0x7f5c4c160b40 2024-05-27T09:32:23.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.594+0000 7f5c50a00640 1 -- 172.21.15.114:0/1505665864 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c4c06a380 con 0x7f5c4c160b40 2024-05-27T09:32:23.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.595+0000 7f5c42000640 1 -- 172.21.15.114:0/1505665864 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5c3400a9c0 con 0x7f5c4c160b40 2024-05-27T09:32:23.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.596+0000 7f5c42000640 1 --2- 172.21.15.114:0/1505665864 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5c28076300 0x7f5c28078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:23.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.596+0000 7f5c42000640 1 -- 172.21.15.114:0/1505665864 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5998+0+0 (secure 0 0 0) 0x7f5c34096b20 con 0x7f5c4c160b40 2024-05-27T09:32:23.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.596+0000 7f5c49e00640 1 --2- 172.21.15.114:0/1505665864 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5c28076300 0x7f5c28078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:23.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.597+0000 7f5c49e00640 1 --2- 172.21.15.114:0/1505665864 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5c28076300 0x7f5c28078810 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c034000 tx=0x7f5c3c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:23.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.601+0000 7f5c42000640 1 -- 172.21.15.114:0/1505665864 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5c3405fc70 con 0x7f5c4c160b40 2024-05-27T09:32:23.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.849+0000 7f5c50a00640 1 -- 172.21.15.114:0/1505665864 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c4c073e30 con 0x7f5c4c160b40 2024-05-27T09:32:23.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.850+0000 7f5c42000640 1 -- 172.21.15.114:0/1505665864 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+6883 (secure 0 0 0) 0x7f5c34063f70 con 0x7f5c4c160b40 2024-05-27T09:32:23.850 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:32:23.850 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4875,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:31:20.888182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4828,4300,4297]},"id":1}]} 2024-05-27T09:32:23.851 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 24 2024-05-27T09:32:23.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.852+0000 7f5c50a00640 1 -- 172.21.15.114:0/1505665864 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5c28076300 msgr2=0x7f5c28078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:23.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.852+0000 7f5c50a00640 1 --2- 172.21.15.114:0/1505665864 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5c28076300 0x7f5c28078810 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c034000 tx=0x7f5c3c02e000 comp rx=0 tx=0).stop 2024-05-27T09:32:23.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.852+0000 7f5c50a00640 1 -- 172.21.15.114:0/1505665864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c4c160b40 msgr2=0x7f5c4c1ed2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:23.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.852+0000 7f5c50a00640 1 --2- 172.21.15.114:0/1505665864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c4c160b40 0x7f5c4c1ed2a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400c730 tx=0x7f5c3400cb30 comp rx=0 tx=0).stop 2024-05-27T09:32:23.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.853+0000 7f5c50a00640 1 -- 172.21.15.114:0/1505665864 shutdown_connections 2024-05-27T09:32:23.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.853+0000 7f5c50a00640 1 --2- 172.21.15.114:0/1505665864 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5c28076300 0x7f5c28078810 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:23.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.853+0000 7f5c50a00640 1 --2- 172.21.15.114:0/1505665864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c4c164db0 0x7f5c4c1ed7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:23.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.853+0000 7f5c50a00640 1 --2- 172.21.15.114:0/1505665864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c4c160b40 0x7f5c4c1ed2a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:23.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.853+0000 7f5c50a00640 1 --2- 172.21.15.114:0/1505665864 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c4c074860 0x7f5c4c158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:23.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.853+0000 7f5c50a00640 1 -- 172.21.15.114:0/1505665864 >> 172.21.15.114:0/1505665864 conn(0x7f5c4c06f6e0 msgr2=0x7f5c4c15aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:23.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.853+0000 7f5c50a00640 1 -- 172.21.15.114:0/1505665864 shutdown_connections 2024-05-27T09:32:23.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:23.853+0000 7f5c50a00640 1 -- 172.21.15.114:0/1505665864 wait complete. 2024-05-27T09:32:23.878 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-05-27T09:32:25.879 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:32:26.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.152+0000 7fc913200640 1 Processor -- start 2024-05-27T09:32:26.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.153+0000 7fc913200640 1 -- start start 2024-05-27T09:32:26.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.155+0000 7fc913200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc914156880 0x7fc91406edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:26.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.155+0000 7fc913200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc914070060 0x7fc91406f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:26.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.155+0000 7fc913200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9140c8470 con 0x7fc914074a80 2024-05-27T09:32:26.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.155+0000 7fc913200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc91415d480 con 0x7fc914156880 2024-05-27T09:32:26.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.155+0000 7fc913200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fc9140c8220 con 0x7fc914070060 2024-05-27T09:32:26.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.155+0000 7fc911400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc914156880 0x7fc91406edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:26.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.155+0000 7fc912800640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc914070060 0x7fc91406f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:26.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.155+0000 7fc911e00640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc914074a80 0x7fc91415cd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:40488/0 (socket says 172.21.15.114:40488) 2024-05-27T09:32:26.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.155+0000 7fc911e00640 1 -- 172.21.15.114:0/2298136381 learned_addr learned my addr 172.21.15.114:0/2298136381 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:32:26.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.156+0000 7fc912800640 1 -- 172.21.15.114:0/2298136381 >> v2:172.21.15.114:3300/0 conn(0x7fc914156880 msgr2=0x7fc91406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:32:26.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.156+0000 7fc912800640 1 --2- 172.21.15.114:0/2298136381 >> v2:172.21.15.114:3300/0 conn(0x7fc914156880 0x7fc91406edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:26.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.156+0000 7fc912800640 1 -- 172.21.15.114:0/2298136381 >> v1:172.21.15.163:6789/0 conn(0x7fc914074a80 legacy=0x7fc91415cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:26.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.156+0000 7fc912800640 1 -- 172.21.15.114:0/2298136381 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9140c8dd0 con 0x7fc914070060 2024-05-27T09:32:26.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.156+0000 7fc910a00640 1 -- 172.21.15.114:0/2298136381 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1788301948 0 0) 0x7fc9140c8470 con 0x7fc914074a80 2024-05-27T09:32:26.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.157+0000 7fc912800640 1 --2- 172.21.15.114:0/2298136381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc914070060 0x7fc91406f330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc9140c83a0 tx=0x7fc90c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89d57f1c17c46b05 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:26.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.157+0000 7fc910a00640 1 -- 172.21.15.114:0/2298136381 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90c037070 con 0x7fc914070060 2024-05-27T09:32:26.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.157+0000 7fc910a00640 1 -- 172.21.15.114:0/2298136381 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc90c032440 con 0x7fc914070060 2024-05-27T09:32:26.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.157+0000 7fc910a00640 1 -- 172.21.15.114:0/2298136381 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90c032810 con 0x7fc914070060 2024-05-27T09:32:26.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.158+0000 7fc913200640 1 -- 172.21.15.114:0/2298136381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc914070060 msgr2=0x7fc91406f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:26.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.158+0000 7fc913200640 1 --2- 172.21.15.114:0/2298136381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc914070060 0x7fc91406f330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc9140c83a0 tx=0x7fc90c02a1a0 comp rx=0 tx=0).stop 2024-05-27T09:32:26.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.158+0000 7fc913200640 1 -- 172.21.15.114:0/2298136381 shutdown_connections 2024-05-27T09:32:26.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.158+0000 7fc913200640 1 --2- 172.21.15.114:0/2298136381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc914070060 0x7fc91406f330 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:26.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.158+0000 7fc913200640 1 --2- 172.21.15.114:0/2298136381 >> v2:172.21.15.114:3300/0 conn(0x7fc914156880 0x7fc91406edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:26.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.158+0000 7fc913200640 1 -- 172.21.15.114:0/2298136381 >> 172.21.15.114:0/2298136381 conn(0x7fc9140ca320 msgr2=0x7fc9140ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:26.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.158+0000 7fc913200640 1 -- 172.21.15.114:0/2298136381 shutdown_connections 2024-05-27T09:32:26.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.158+0000 7fc913200640 1 -- 172.21.15.114:0/2298136381 wait complete. 2024-05-27T09:32:26.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.160+0000 7fc913200640 1 Processor -- start 2024-05-27T09:32:26.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.160+0000 7fc913200640 1 -- start start 2024-05-27T09:32:26.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.162+0000 7fc913200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc914070060 0x7fc9141f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:26.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.162+0000 7fc913200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc914074a80 0x7fc9140c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:26.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.162+0000 7fc913200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc914156880 0x7fc9140c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:26.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.162+0000 7fc913200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc91415c8e0 con 0x7fc914156880 2024-05-27T09:32:26.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.162+0000 7fc913200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc9140c8750 con 0x7fc914074a80 2024-05-27T09:32:26.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.162+0000 7fc913200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fc9140ca0c0 con 0x7fc914070060 2024-05-27T09:32:26.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.162+0000 7fc912800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc914156880 0x7fc9140c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:26.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.162+0000 7fc911400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc914074a80 0x7fc9140c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:26.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.162+0000 7fc911e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc914070060 0x7fc9141f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:26.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.162+0000 7fc912800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc914156880 0x7fc9140c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42318/0 (socket says 172.21.15.114:42318) 2024-05-27T09:32:26.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.162+0000 7fc912800640 1 -- 172.21.15.114:0/2222985151 learned_addr learned my addr 172.21.15.114:0/2222985151 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:26.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.163+0000 7fc912800640 1 -- 172.21.15.114:0/2222985151 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc914070060 msgr2=0x7fc9141f3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:26.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.163+0000 7fc912800640 1 --2- 172.21.15.114:0/2222985151 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc914070060 0x7fc9141f3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:26.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.163+0000 7fc912800640 1 -- 172.21.15.114:0/2222985151 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc914074a80 msgr2=0x7fc9140c4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:26.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.163+0000 7fc912800640 1 --2- 172.21.15.114:0/2222985151 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc914074a80 0x7fc9140c4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:26.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.163+0000 7fc912800640 1 -- 172.21.15.114:0/2222985151 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9141f4060 con 0x7fc914156880 2024-05-27T09:32:26.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.163+0000 7fc912800640 1 --2- 172.21.15.114:0/2222985151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc914156880 0x7fc9140c6af0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc90c009860 tx=0x7fc90c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:26.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.163+0000 7fc90a000640 1 -- 172.21.15.114:0/2222985151 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90c037040 con 0x7fc914156880 2024-05-27T09:32:26.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.163+0000 7fc90a000640 1 -- 172.21.15.114:0/2222985151 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc90c042070 con 0x7fc914156880 2024-05-27T09:32:26.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.163+0000 7fc913200640 1 -- 172.21.15.114:0/2222985151 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc914154070 con 0x7fc914156880 2024-05-27T09:32:26.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.164+0000 7fc90a000640 1 -- 172.21.15.114:0/2222985151 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90c030880 con 0x7fc914156880 2024-05-27T09:32:26.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.164+0000 7fc913200640 1 -- 172.21.15.114:0/2222985151 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc914154640 con 0x7fc914156880 2024-05-27T09:32:26.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.165+0000 7fc913200640 1 -- 172.21.15.114:0/2222985151 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc914050860 con 0x7fc914156880 2024-05-27T09:32:26.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.166+0000 7fc90a000640 1 -- 172.21.15.114:0/2222985151 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc90c030a20 con 0x7fc914156880 2024-05-27T09:32:26.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.167+0000 7fc90a000640 1 --2- 172.21.15.114:0/2222985151 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc8f00762d0 0x7fc8f00787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:26.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.167+0000 7fc90a000640 1 -- 172.21.15.114:0/2222985151 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5998+0+0 (secure 0 0 0) 0x7fc90c0b7d50 con 0x7fc914156880 2024-05-27T09:32:26.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.167+0000 7fc911e00640 1 --2- 172.21.15.114:0/2222985151 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc8f00762d0 0x7fc8f00787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:26.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.168+0000 7fc911e00640 1 --2- 172.21.15.114:0/2222985151 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc8f00762d0 0x7fc8f00787e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc007fd0 tx=0x7fc8fc007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:26.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.172+0000 7fc90a000640 1 -- 172.21.15.114:0/2222985151 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc90c0bb030 con 0x7fc914156880 2024-05-27T09:32:26.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.412+0000 7fc913200640 1 -- 172.21.15.114:0/2222985151 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9140c8ab0 con 0x7fc914156880 2024-05-27T09:32:26.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.413+0000 7fc90a000640 1 -- 172.21.15.114:0/2222985151 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+6883 (secure 0 0 0) 0x7fc90c080f40 con 0x7fc914156880 2024-05-27T09:32:26.414 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:32:26.414 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4875,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:31:20.888182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4828,4300,4297]},"id":1}]} 2024-05-27T09:32:26.414 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 24 2024-05-27T09:32:26.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.415+0000 7fc913200640 1 -- 172.21.15.114:0/2222985151 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc8f00762d0 msgr2=0x7fc8f00787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:26.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.416+0000 7fc913200640 1 --2- 172.21.15.114:0/2222985151 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc8f00762d0 0x7fc8f00787e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc007fd0 tx=0x7fc8fc007de0 comp rx=0 tx=0).stop 2024-05-27T09:32:26.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.416+0000 7fc913200640 1 -- 172.21.15.114:0/2222985151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc914156880 msgr2=0x7fc9140c6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:26.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.416+0000 7fc913200640 1 --2- 172.21.15.114:0/2222985151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc914156880 0x7fc9140c6af0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc90c009860 tx=0x7fc90c0046c0 comp rx=0 tx=0).stop 2024-05-27T09:32:26.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.416+0000 7fc913200640 1 -- 172.21.15.114:0/2222985151 shutdown_connections 2024-05-27T09:32:26.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.416+0000 7fc913200640 1 --2- 172.21.15.114:0/2222985151 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc8f00762d0 0x7fc8f00787e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:26.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.416+0000 7fc913200640 1 --2- 172.21.15.114:0/2222985151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc914156880 0x7fc9140c6af0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:26.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.416+0000 7fc913200640 1 --2- 172.21.15.114:0/2222985151 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc914074a80 0x7fc9140c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:26.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.416+0000 7fc913200640 1 --2- 172.21.15.114:0/2222985151 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc914070060 0x7fc9141f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:26.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.416+0000 7fc913200640 1 -- 172.21.15.114:0/2222985151 >> 172.21.15.114:0/2222985151 conn(0x7fc9140ca320 msgr2=0x7fc914070990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:26.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.416+0000 7fc913200640 1 -- 172.21.15.114:0/2222985151 shutdown_connections 2024-05-27T09:32:26.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:26.416+0000 7fc913200640 1 -- 172.21.15.114:0/2222985151 wait complete. 2024-05-27T09:32:26.441 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-05-27T09:32:28.442 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:32:28.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.723+0000 7f8753e00640 1 Processor -- start 2024-05-27T09:32:28.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.723+0000 7f8753e00640 1 -- start start 2024-05-27T09:32:28.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.725+0000 7f8753e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f87540741f0 0x7f87540c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:28.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.725+0000 7f8753e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87541547d0 0x7f875415ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:28.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.725+0000 7f8753e00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f875406fe50 con 0x7f8754074840 2024-05-27T09:32:28.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.725+0000 7f8753e00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f87540703d0 con 0x7f87540741f0 2024-05-27T09:32:28.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.725+0000 7f8753e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f875415b260 con 0x7f87541547d0 2024-05-27T09:32:28.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.725+0000 7f8752a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f87540741f0 0x7f87540c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:28.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.725+0000 7f8752a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f87540741f0 0x7f87540c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42324/0 (socket says 172.21.15.114:42324) 2024-05-27T09:32:28.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.726+0000 7f8752a00640 1 -- 172.21.15.114:0/3737278143 learned_addr learned my addr 172.21.15.114:0/3737278143 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:28.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.726+0000 7f8752000640 1 --2- 172.21.15.114:0/3737278143 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87541547d0 0x7f875415ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:28.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.726+0000 7f8752a00640 1 -- 172.21.15.114:0/3737278143 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87541547d0 msgr2=0x7f875415ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:32:28.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.726+0000 7f8752a00640 1 --2- 172.21.15.114:0/3737278143 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87541547d0 0x7f875415ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:28.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.726+0000 7f8752a00640 1 -- 172.21.15.114:0/3737278143 >> v1:172.21.15.114:6789/0 conn(0x7f8754074840 legacy=0x7f875406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:28.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.726+0000 7f8752a00640 1 -- 172.21.15.114:0/3737278143 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87540700f0 con 0x7f87540741f0 2024-05-27T09:32:28.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.727+0000 7f8752a00640 1 --2- 172.21.15.114:0/3737278143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87540741f0 0x7f87540c63a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8744009870 tx=0x7f874402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80edfca90d4be25 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:28.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.727+0000 7f8751600640 1 -- 172.21.15.114:0/3737278143 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8744037070 con 0x7f87540741f0 2024-05-27T09:32:28.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.727+0000 7f8751600640 1 -- 172.21.15.114:0/3737278143 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f874402ae50 con 0x7f87540741f0 2024-05-27T09:32:28.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.728+0000 7f8751600640 1 -- 172.21.15.114:0/3737278143 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8744032860 con 0x7f87540741f0 2024-05-27T09:32:28.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.728+0000 7f8753e00640 1 -- 172.21.15.114:0/3737278143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87540741f0 msgr2=0x7f87540c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:28.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.728+0000 7f8753e00640 1 --2- 172.21.15.114:0/3737278143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87540741f0 0x7f87540c63a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8744009870 tx=0x7f874402a300 comp rx=0 tx=0).stop 2024-05-27T09:32:28.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.728+0000 7f8753e00640 1 -- 172.21.15.114:0/3737278143 shutdown_connections 2024-05-27T09:32:28.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.728+0000 7f8753e00640 1 --2- 172.21.15.114:0/3737278143 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87541547d0 0x7f875415ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:28.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.728+0000 7f8753e00640 1 --2- 172.21.15.114:0/3737278143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87540741f0 0x7f87540c63a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:28.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.728+0000 7f8753e00640 1 -- 172.21.15.114:0/3737278143 >> 172.21.15.114:0/3737278143 conn(0x7f87540ca320 msgr2=0x7f87540ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:28.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.728+0000 7f8753e00640 1 -- 172.21.15.114:0/3737278143 shutdown_connections 2024-05-27T09:32:28.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.728+0000 7f8753e00640 1 -- 172.21.15.114:0/3737278143 wait complete. 2024-05-27T09:32:28.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.731+0000 7f8753e00640 1 Processor -- start 2024-05-27T09:32:28.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.731+0000 7f8753e00640 1 -- start start 2024-05-27T09:32:28.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.732+0000 7f8753e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87540741f0 0x7f87540c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:28.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.732+0000 7f8753e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8754074840 0x7f8754154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:28.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.732+0000 7f8753e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87541547d0 0x7f87541f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:28.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.732+0000 7f8753e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8754073790 con 0x7f87541547d0 2024-05-27T09:32:28.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.732+0000 7f8753e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f87540734b0 con 0x7f87540741f0 2024-05-27T09:32:28.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.732+0000 7f8753e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f875406f500 con 0x7f8754074840 2024-05-27T09:32:28.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.733+0000 7f8752000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8754074840 0x7f8754154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:28.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.733+0000 7f8753400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87541547d0 0x7f87541f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:28.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.733+0000 7f8752a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87540741f0 0x7f87540c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:28.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.733+0000 7f8752000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8754074840 0x7f8754154cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:37536/0 (socket says 172.21.15.114:37536) 2024-05-27T09:32:28.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.733+0000 7f8752000640 1 -- 172.21.15.114:0/2281297477 learned_addr learned my addr 172.21.15.114:0/2281297477 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:28.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.733+0000 7f8753400640 1 -- 172.21.15.114:0/2281297477 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8754074840 msgr2=0x7f8754154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:28.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.733+0000 7f8753400640 1 --2- 172.21.15.114:0/2281297477 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8754074840 0x7f8754154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:28.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.733+0000 7f8753400640 1 -- 172.21.15.114:0/2281297477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87540741f0 msgr2=0x7f87540c44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:28.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.733+0000 7f8753400640 1 --2- 172.21.15.114:0/2281297477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87540741f0 0x7f87540c44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:28.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.733+0000 7f8753400640 1 -- 172.21.15.114:0/2281297477 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8754155850 con 0x7f87541547d0 2024-05-27T09:32:28.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.733+0000 7f8752000640 1 --2- 172.21.15.114:0/2281297477 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8754074840 0x7f8754154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:32:28.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.733+0000 7f8753400640 1 --2- 172.21.15.114:0/2281297477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87541547d0 0x7f87541f6ba0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8754070020 tx=0x7f874c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:28.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.734+0000 7f874aa00640 1 -- 172.21.15.114:0/2281297477 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c003b90 con 0x7f87541547d0 2024-05-27T09:32:28.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.734+0000 7f874aa00640 1 -- 172.21.15.114:0/2281297477 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f874c003d50 con 0x7f87541547d0 2024-05-27T09:32:28.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.734+0000 7f874aa00640 1 -- 172.21.15.114:0/2281297477 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c005310 con 0x7f87541547d0 2024-05-27T09:32:28.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.734+0000 7f8753e00640 1 -- 172.21.15.114:0/2281297477 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87541f70e0 con 0x7f87541547d0 2024-05-27T09:32:28.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.734+0000 7f8753e00640 1 -- 172.21.15.114:0/2281297477 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87540c3710 con 0x7f87541547d0 2024-05-27T09:32:28.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.735+0000 7f8753e00640 1 -- 172.21.15.114:0/2281297477 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8754157cb0 con 0x7f87541547d0 2024-05-27T09:32:28.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.736+0000 7f874aa00640 1 -- 172.21.15.114:0/2281297477 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f874c0054b0 con 0x7f87541547d0 2024-05-27T09:32:28.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.737+0000 7f874aa00640 1 --2- 172.21.15.114:0/2281297477 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f87300762d0 0x7f87300787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:28.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.737+0000 7f874aa00640 1 -- 172.21.15.114:0/2281297477 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5998+0+0 (secure 0 0 0) 0x7f874c097360 con 0x7f87541547d0 2024-05-27T09:32:28.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.738+0000 7f8752a00640 1 --2- 172.21.15.114:0/2281297477 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f87300762d0 0x7f87300787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:28.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.738+0000 7f8752a00640 1 --2- 172.21.15.114:0/2281297477 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f87300762d0 0x7f87300787e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8744034000 tx=0x7f874402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:28.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.742+0000 7f874aa00640 1 -- 172.21.15.114:0/2281297477 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f874c0604b0 con 0x7f87541547d0 2024-05-27T09:32:28.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.991+0000 7f8753e00640 1 -- 172.21.15.114:0/2281297477 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f875406ef80 con 0x7f87541547d0 2024-05-27T09:32:28.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.993+0000 7f874aa00640 1 -- 172.21.15.114:0/2281297477 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+6883 (secure 0 0 0) 0x7f874c0647b0 con 0x7f87541547d0 2024-05-27T09:32:28.993 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:32:28.993 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4875,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:31:20.888182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4297,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.114:6834/657027519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":657027519},{"type":"v1","addr":"172.21.15.114:6834","nonce":657027519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4828,4300,4297]},"id":1}]} 2024-05-27T09:32:28.993 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 24 2024-05-27T09:32:28.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.996+0000 7f8753e00640 1 -- 172.21.15.114:0/2281297477 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f87300762d0 msgr2=0x7f87300787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:28.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.996+0000 7f8753e00640 1 --2- 172.21.15.114:0/2281297477 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f87300762d0 0x7f87300787e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8744034000 tx=0x7f874402e000 comp rx=0 tx=0).stop 2024-05-27T09:32:28.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.996+0000 7f8753e00640 1 -- 172.21.15.114:0/2281297477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87541547d0 msgr2=0x7f87541f6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:28.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.996+0000 7f8753e00640 1 --2- 172.21.15.114:0/2281297477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87541547d0 0x7f87541f6ba0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8754070020 tx=0x7f874c00cb90 comp rx=0 tx=0).stop 2024-05-27T09:32:28.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.996+0000 7f8753e00640 1 -- 172.21.15.114:0/2281297477 shutdown_connections 2024-05-27T09:32:28.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.996+0000 7f8753e00640 1 --2- 172.21.15.114:0/2281297477 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f87300762d0 0x7f87300787e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:28.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.996+0000 7f8753e00640 1 --2- 172.21.15.114:0/2281297477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87541547d0 0x7f87541f6ba0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:28.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.996+0000 7f8753e00640 1 --2- 172.21.15.114:0/2281297477 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8754074840 0x7f8754154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:28.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.996+0000 7f8753e00640 1 --2- 172.21.15.114:0/2281297477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87540741f0 0x7f87540c44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:28.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.996+0000 7f8753e00640 1 -- 172.21.15.114:0/2281297477 >> 172.21.15.114:0/2281297477 conn(0x7f87540ca320 msgr2=0x7f8754070990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:28.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.996+0000 7f8753e00640 1 -- 172.21.15.114:0/2281297477 shutdown_connections 2024-05-27T09:32:28.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:28.996+0000 7f8753e00640 1 -- 172.21.15.114:0/2281297477 wait complete. 2024-05-27T09:32:29.022 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-05-27T09:32:31.023 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:32:31.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.303+0000 7f475ba00640 1 Processor -- start 2024-05-27T09:32:31.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.304+0000 7f475ba00640 1 -- start start 2024-05-27T09:32:31.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.306+0000 7f475ba00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f47540b2c60 0x7f475415d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:31.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.306+0000 7f475ba00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4754156f80 con 0x7f47540ca550 2024-05-27T09:32:31.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.306+0000 7f475ba00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47540cac50 con 0x7f47540ca140 2024-05-27T09:32:31.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.306+0000 7f475ba00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f475415d7c0 con 0x7f47540b2c60 2024-05-27T09:32:31.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.306+0000 7f4759a00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f47540ca550 0x7f47540c3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54302/0 (socket says 172.21.15.114:54302) 2024-05-27T09:32:31.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.306+0000 7f4759a00640 1 -- 172.21.15.114:0/516779596 learned_addr learned my addr 172.21.15.114:0/516779596 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:32:31.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.306+0000 7f4759000640 1 --2- 172.21.15.114:0/516779596 >> v2:172.21.15.163:3300/0 conn(0x7f47540b2c60 0x7f475415d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:31.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.307+0000 7f4753400640 1 -- 172.21.15.114:0/516779596 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4276621491 0 0) 0x7f4754156f80 con 0x7f47540ca550 2024-05-27T09:32:31.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.307+0000 7f4753400640 1 -- 172.21.15.114:0/516779596 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4738003520 con 0x7f47540ca550 2024-05-27T09:32:31.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.307+0000 7f4753400640 1 -- 172.21.15.114:0/516779596 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3351472371 0 0) 0x7f47540cac50 con 0x7f47540ca140 2024-05-27T09:32:31.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.307+0000 7f4753400640 1 -- 172.21.15.114:0/516779596 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4754156f80 con 0x7f47540ca140 2024-05-27T09:32:31.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.307+0000 7f4759000640 1 -- 172.21.15.114:0/516779596 >> v1:172.21.15.163:6789/0 conn(0x7f47540ca140 legacy=0x7f4754073310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:31.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.307+0000 7f4759000640 1 -- 172.21.15.114:0/516779596 >> v1:172.21.15.114:6789/0 conn(0x7f47540ca550 legacy=0x7f47540c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:31.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.307+0000 7f4759000640 1 -- 172.21.15.114:0/516779596 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475415ffa0 con 0x7f47540b2c60 2024-05-27T09:32:31.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.308+0000 7f4759000640 1 --2- 172.21.15.114:0/516779596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47540b2c60 0x7f475415d280 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f47440097a0 tx=0x7f474402a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5df6572f37d3e936 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:31.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.308+0000 7f4753400640 1 -- 172.21.15.114:0/516779596 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744037070 con 0x7f47540b2c60 2024-05-27T09:32:31.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.308+0000 7f4753400640 1 -- 172.21.15.114:0/516779596 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4744032640 con 0x7f47540b2c60 2024-05-27T09:32:31.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.308+0000 7f4753400640 1 -- 172.21.15.114:0/516779596 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47440329f0 con 0x7f47540b2c60 2024-05-27T09:32:31.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.309+0000 7f475ba00640 1 -- 172.21.15.114:0/516779596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47540b2c60 msgr2=0x7f475415d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:31.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.309+0000 7f475ba00640 1 --2- 172.21.15.114:0/516779596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47540b2c60 0x7f475415d280 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f47440097a0 tx=0x7f474402a2c0 comp rx=0 tx=0).stop 2024-05-27T09:32:31.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.309+0000 7f475ba00640 1 -- 172.21.15.114:0/516779596 shutdown_connections 2024-05-27T09:32:31.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.309+0000 7f475ba00640 1 --2- 172.21.15.114:0/516779596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47540b2c60 0x7f475415d280 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:31.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.309+0000 7f475ba00640 1 -- 172.21.15.114:0/516779596 >> 172.21.15.114:0/516779596 conn(0x7f475406f6e0 msgr2=0x7f4754070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:31.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.309+0000 7f475ba00640 1 -- 172.21.15.114:0/516779596 shutdown_connections 2024-05-27T09:32:31.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.309+0000 7f475ba00640 1 -- 172.21.15.114:0/516779596 wait complete. 2024-05-27T09:32:31.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.312+0000 7f475ba00640 1 Processor -- start 2024-05-27T09:32:31.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.312+0000 7f475ba00640 1 -- start start 2024-05-27T09:32:31.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.313+0000 7f475ba00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47540b2c60 0x7f4754151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:31.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.313+0000 7f475ba00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47540ca140 0x7f4754158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:31.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.313+0000 7f475ba00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47540ca550 0x7f475415e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:31.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.313+0000 7f475ba00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4754160560 con 0x7f47540ca550 2024-05-27T09:32:31.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.313+0000 7f475ba00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f475415fd50 con 0x7f47540ca140 2024-05-27T09:32:31.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.313+0000 7f475ba00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f4754160840 con 0x7f47540b2c60 2024-05-27T09:32:31.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.313+0000 7f4759a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47540ca550 0x7f475415e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:31.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.314+0000 7f4759a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47540ca550 0x7f475415e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42342/0 (socket says 172.21.15.114:42342) 2024-05-27T09:32:31.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.314+0000 7f4759a00640 1 -- 172.21.15.114:0/1204576399 learned_addr learned my addr 172.21.15.114:0/1204576399 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:31.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.314+0000 7f4753e00640 1 --2- 172.21.15.114:0/1204576399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47540ca140 0x7f4754158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:31.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.314+0000 7f4759000640 1 --2- 172.21.15.114:0/1204576399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47540b2c60 0x7f4754151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:31.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.314+0000 7f4759a00640 1 -- 172.21.15.114:0/1204576399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47540b2c60 msgr2=0x7f4754151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:31.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.314+0000 7f4759a00640 1 --2- 172.21.15.114:0/1204576399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47540b2c60 0x7f4754151780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:31.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.314+0000 7f4759a00640 1 -- 172.21.15.114:0/1204576399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47540ca140 msgr2=0x7f4754158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:31.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.314+0000 7f4759a00640 1 --2- 172.21.15.114:0/1204576399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47540ca140 0x7f4754158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:31.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.314+0000 7f4759a00640 1 -- 172.21.15.114:0/1204576399 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4754151cc0 con 0x7f47540ca550 2024-05-27T09:32:31.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.314+0000 7f4759000640 1 --2- 172.21.15.114:0/1204576399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47540b2c60 0x7f4754151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:32:31.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.314+0000 7f4759a00640 1 --2- 172.21.15.114:0/1204576399 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47540ca550 0x7f475415e2a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f474c00ca70 tx=0x7f474c00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:31.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.315+0000 7f4750c00640 1 -- 172.21.15.114:0/1204576399 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c00abd0 con 0x7f47540ca550 2024-05-27T09:32:31.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.315+0000 7f4750c00640 1 -- 172.21.15.114:0/1204576399 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f474c00ad70 con 0x7f47540ca550 2024-05-27T09:32:31.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.315+0000 7f4750c00640 1 -- 172.21.15.114:0/1204576399 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c012640 con 0x7f47540ca550 2024-05-27T09:32:31.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.315+0000 7f475ba00640 1 -- 172.21.15.114:0/1204576399 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47541f3b70 con 0x7f47540ca550 2024-05-27T09:32:31.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.315+0000 7f475ba00640 1 -- 172.21.15.114:0/1204576399 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4754150f80 con 0x7f47540ca550 2024-05-27T09:32:31.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.316+0000 7f475ba00640 1 -- 172.21.15.114:0/1204576399 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47540cbec0 con 0x7f47540ca550 2024-05-27T09:32:31.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.317+0000 7f4750c00640 1 -- 172.21.15.114:0/1204576399 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f474c0127e0 con 0x7f47540ca550 2024-05-27T09:32:31.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.318+0000 7f4750c00640 1 --2- 172.21.15.114:0/1204576399 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f47300762d0 0x7f47300787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:31.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.318+0000 7f4750c00640 1 -- 172.21.15.114:0/1204576399 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6040+0+0 (secure 0 0 0) 0x7f474c097b90 con 0x7f47540ca550 2024-05-27T09:32:31.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.319+0000 7f4759000640 1 --2- 172.21.15.114:0/1204576399 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f47300762d0 0x7f47300787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:31.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.319+0000 7f4759000640 1 --2- 172.21.15.114:0/1204576399 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f47300762d0 0x7f47300787e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4744034000 tx=0x7f474402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:31.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.323+0000 7f4750c00640 1 -- 172.21.15.114:0/1204576399 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f47540cbec0 con 0x7f47540ca550 2024-05-27T09:32:31.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.579+0000 7f475ba00640 1 -- 172.21.15.114:0/1204576399 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47540cc0c0 con 0x7f47540ca550 2024-05-27T09:32:31.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.580+0000 7f4750c00640 1 -- 172.21.15.114:0/1204576399 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+6054 (secure 0 0 0) 0x7f47540cc0c0 con 0x7f47540ca550 2024-05-27T09:32:31.582 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:32:31.582 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:32:30.374582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:replay","state_seq":1,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4828]},"id":1}]} 2024-05-27T09:32:31.582 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 25 2024-05-27T09:32:31.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.583+0000 7f475ba00640 1 -- 172.21.15.114:0/1204576399 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f47300762d0 msgr2=0x7f47300787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:31.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.583+0000 7f475ba00640 1 --2- 172.21.15.114:0/1204576399 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f47300762d0 0x7f47300787e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4744034000 tx=0x7f474402e000 comp rx=0 tx=0).stop 2024-05-27T09:32:31.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.583+0000 7f475ba00640 1 -- 172.21.15.114:0/1204576399 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47540ca550 msgr2=0x7f475415e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:31.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.583+0000 7f475ba00640 1 --2- 172.21.15.114:0/1204576399 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47540ca550 0x7f475415e2a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f474c00ca70 tx=0x7f474c00cf30 comp rx=0 tx=0).stop 2024-05-27T09:32:31.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.583+0000 7f475ba00640 1 -- 172.21.15.114:0/1204576399 shutdown_connections 2024-05-27T09:32:31.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.583+0000 7f475ba00640 1 --2- 172.21.15.114:0/1204576399 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f47300762d0 0x7f47300787e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:31.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.583+0000 7f475ba00640 1 --2- 172.21.15.114:0/1204576399 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f47540ca550 0x7f475415e2a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:31.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.583+0000 7f475ba00640 1 --2- 172.21.15.114:0/1204576399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47540ca140 0x7f4754158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:31.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.583+0000 7f475ba00640 1 --2- 172.21.15.114:0/1204576399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47540b2c60 0x7f4754151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:31.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.583+0000 7f475ba00640 1 -- 172.21.15.114:0/1204576399 >> 172.21.15.114:0/1204576399 conn(0x7f475406f6e0 msgr2=0x7f47540cb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:31.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.584+0000 7f475ba00640 1 -- 172.21.15.114:0/1204576399 shutdown_connections 2024-05-27T09:32:31.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.584+0000 7f475ba00640 1 -- 172.21.15.114:0/1204576399 wait complete. 2024-05-27T09:32:31.610 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2024-05-27T09:32:31.610 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-05-27T09:32:31.610 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:32:31.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.871+0000 7f1ba0c00640 1 Processor -- start 2024-05-27T09:32:31.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.872+0000 7f1ba0c00640 1 -- start start 2024-05-27T09:32:31.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.874+0000 7f1ba0c00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1b9c0741f0 0x7f1b9c0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:31.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.874+0000 7f1ba0c00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1b9c1547d0 0x7f1b9c15ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:31.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.874+0000 7f1ba0c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b9c074840 0x7f1b9c06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:31.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.874+0000 7f1ba0c00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f1b9c0703d0 con 0x7f1b9c1547d0 2024-05-27T09:32:31.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.874+0000 7f1ba0c00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f1b9c15b260 con 0x7f1b9c0741f0 2024-05-27T09:32:31.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.874+0000 7f1ba0c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f1b9c06fe50 con 0x7f1b9c074840 2024-05-27T09:32:31.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.874+0000 7f1b93e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1b9c1547d0 0x7f1b9c15ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:31.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.874+0000 7f1b9b400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1b9c0741f0 0x7f1b9c0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:31.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.874+0000 7f1b9be00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b9c074840 0x7f1b9c06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:31.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.874+0000 7f1b93e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1b9c1547d0 0x7f1b9c15ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42352/0 (socket says 172.21.15.114:42352) 2024-05-27T09:32:31.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.874+0000 7f1b93e00640 1 -- 172.21.15.114:0/2407474166 learned_addr learned my addr 172.21.15.114:0/2407474166 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:31.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.875+0000 7f1b93e00640 1 -- 172.21.15.114:0/2407474166 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b9c074840 msgr2=0x7f1b9c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:32:31.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.875+0000 7f1b93e00640 1 --2- 172.21.15.114:0/2407474166 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b9c074840 0x7f1b9c06edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:31.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.875+0000 7f1b93e00640 1 -- 172.21.15.114:0/2407474166 >> v2:172.21.15.163:3300/0 conn(0x7f1b9c0741f0 msgr2=0x7f1b9c0c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:32:31.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.875+0000 7f1b93e00640 1 --2- 172.21.15.114:0/2407474166 >> v2:172.21.15.163:3300/0 conn(0x7f1b9c0741f0 0x7f1b9c0c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:31.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.875+0000 7f1b93e00640 1 -- 172.21.15.114:0/2407474166 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9c0700f0 con 0x7f1b9c1547d0 2024-05-27T09:32:31.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.876+0000 7f1b93e00640 1 --2- 172.21.15.114:0/2407474166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9c1547d0 0x7f1b9c15ad20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0caf30 tx=0x7f1b8402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5906d256399c726c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:31.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.876+0000 7f1b9aa00640 1 -- 172.21.15.114:0/2407474166 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b84037070 con 0x7f1b9c1547d0 2024-05-27T09:32:31.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.876+0000 7f1b9aa00640 1 -- 172.21.15.114:0/2407474166 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b84032440 con 0x7f1b9c1547d0 2024-05-27T09:32:31.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.876+0000 7f1b9aa00640 1 -- 172.21.15.114:0/2407474166 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b84032810 con 0x7f1b9c1547d0 2024-05-27T09:32:31.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.876+0000 7f1ba0c00640 1 -- 172.21.15.114:0/2407474166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9c1547d0 msgr2=0x7f1b9c15ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:31.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.876+0000 7f1ba0c00640 1 --2- 172.21.15.114:0/2407474166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9c1547d0 0x7f1b9c15ad20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0caf30 tx=0x7f1b8402a2a0 comp rx=0 tx=0).stop 2024-05-27T09:32:31.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.877+0000 7f1ba0c00640 1 -- 172.21.15.114:0/2407474166 shutdown_connections 2024-05-27T09:32:31.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.877+0000 7f1ba0c00640 1 --2- 172.21.15.114:0/2407474166 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b9c074840 0x7f1b9c06edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:31.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.877+0000 7f1ba0c00640 1 --2- 172.21.15.114:0/2407474166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9c1547d0 0x7f1b9c15ad20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:31.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.877+0000 7f1ba0c00640 1 --2- 172.21.15.114:0/2407474166 >> v2:172.21.15.163:3300/0 conn(0x7f1b9c0741f0 0x7f1b9c0c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:31.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.877+0000 7f1ba0c00640 1 -- 172.21.15.114:0/2407474166 >> 172.21.15.114:0/2407474166 conn(0x7f1b9c0ca320 msgr2=0x7f1b9c0ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:31.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.877+0000 7f1ba0c00640 1 -- 172.21.15.114:0/2407474166 shutdown_connections 2024-05-27T09:32:31.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.877+0000 7f1ba0c00640 1 -- 172.21.15.114:0/2407474166 wait complete. 2024-05-27T09:32:31.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.879+0000 7f1ba0c00640 1 Processor -- start 2024-05-27T09:32:31.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.879+0000 7f1ba0c00640 1 -- start start 2024-05-27T09:32:31.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.881+0000 7f1ba0c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9c0741f0 0x7f1b9c1f0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:31.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.881+0000 7f1ba0c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b9c074840 0x7f1b9c14f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:31.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.881+0000 7f1ba0c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b9c1547d0 0x7f1b9c147290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:31.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.881+0000 7f1ba0c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c073790 con 0x7f1b9c0741f0 2024-05-27T09:32:31.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.881+0000 7f1ba0c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c0734b0 con 0x7f1b9c074840 2024-05-27T09:32:31.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.881+0000 7f1ba0c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f1b9c073cb0 con 0x7f1b9c1547d0 2024-05-27T09:32:31.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.881+0000 7f1b9b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9c0741f0 0x7f1b9c1f0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:31.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.881+0000 7f1b9b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9c0741f0 0x7f1b9c1f0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42354/0 (socket says 172.21.15.114:42354) 2024-05-27T09:32:31.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.881+0000 7f1b9b400640 1 -- 172.21.15.114:0/3457089919 learned_addr learned my addr 172.21.15.114:0/3457089919 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:31.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.881+0000 7f1b93e00640 1 --2- 172.21.15.114:0/3457089919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b9c074840 0x7f1b9c14f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:31.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.881+0000 7f1b9be00640 1 --2- 172.21.15.114:0/3457089919 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b9c1547d0 0x7f1b9c147290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:31.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.881+0000 7f1b9b400640 1 -- 172.21.15.114:0/3457089919 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b9c1547d0 msgr2=0x7f1b9c147290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:31.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.881+0000 7f1b9b400640 1 --2- 172.21.15.114:0/3457089919 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b9c1547d0 0x7f1b9c147290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:31.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.881+0000 7f1b9b400640 1 -- 172.21.15.114:0/3457089919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b9c074840 msgr2=0x7f1b9c14f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:31.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.881+0000 7f1b9b400640 1 --2- 172.21.15.114:0/3457089919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b9c074840 0x7f1b9c14f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:31.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.881+0000 7f1b9b400640 1 -- 172.21.15.114:0/3457089919 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9c15ca20 con 0x7f1b9c0741f0 2024-05-27T09:32:31.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.882+0000 7f1b9b400640 1 --2- 172.21.15.114:0/3457089919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9c0741f0 0x7f1b9c1f0220 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00e870 tx=0x7f1b8c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:31.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.882+0000 7f1b93400640 1 -- 172.21.15.114:0/3457089919 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c017070 con 0x7f1b9c0741f0 2024-05-27T09:32:31.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.882+0000 7f1b93400640 1 -- 172.21.15.114:0/3457089919 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8c00ee60 con 0x7f1b9c0741f0 2024-05-27T09:32:31.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.882+0000 7f1b93400640 1 -- 172.21.15.114:0/3457089919 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c00cb80 con 0x7f1b9c0741f0 2024-05-27T09:32:31.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.882+0000 7f1ba0c00640 1 -- 172.21.15.114:0/3457089919 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b9c157020 con 0x7f1b9c0741f0 2024-05-27T09:32:31.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.882+0000 7f1ba0c00640 1 -- 172.21.15.114:0/3457089919 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b9c1575e0 con 0x7f1b9c0741f0 2024-05-27T09:32:31.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.883+0000 7f1ba0c00640 1 -- 172.21.15.114:0/3457089919 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b9c14fbd0 con 0x7f1b9c0741f0 2024-05-27T09:32:31.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.884+0000 7f1b93400640 1 -- 172.21.15.114:0/3457089919 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1b8c01a070 con 0x7f1b9c0741f0 2024-05-27T09:32:31.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.885+0000 7f1b93400640 1 --2- 172.21.15.114:0/3457089919 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1b780762d0 0x7f1b780787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:31.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.885+0000 7f1b93400640 1 -- 172.21.15.114:0/3457089919 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6040+0+0 (secure 0 0 0) 0x7f1b8c0979d0 con 0x7f1b9c0741f0 2024-05-27T09:32:31.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.886+0000 7f1b93e00640 1 --2- 172.21.15.114:0/3457089919 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1b780762d0 0x7f1b780787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:31.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.887+0000 7f1b93e00640 1 --2- 172.21.15.114:0/3457089919 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1b780762d0 0x7f1b780787e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1b84034000 tx=0x7f1b8402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:31.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:31.890+0000 7f1b93400640 1 -- 172.21.15.114:0/3457089919 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1b8c060a60 con 0x7f1b9c0741f0 2024-05-27T09:32:32.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:32.136+0000 7f1ba0c00640 1 -- 172.21.15.114:0/3457089919 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b9c074600 con 0x7f1b9c0741f0 2024-05-27T09:32:32.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:32.137+0000 7f1b93400640 1 -- 172.21.15.114:0/3457089919 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+6054 (secure 0 0 0) 0x7f1b8c064d60 con 0x7f1b9c0741f0 2024-05-27T09:32:32.138 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:32:32.138 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:32:30.374582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:replay","state_seq":1,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4828]},"id":1}]} 2024-05-27T09:32:32.139 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 25 2024-05-27T09:32:32.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:32.139+0000 7f1ba0c00640 1 -- 172.21.15.114:0/3457089919 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1b780762d0 msgr2=0x7f1b780787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:32.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:32.139+0000 7f1ba0c00640 1 --2- 172.21.15.114:0/3457089919 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1b780762d0 0x7f1b780787e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1b84034000 tx=0x7f1b8402e000 comp rx=0 tx=0).stop 2024-05-27T09:32:32.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:32.139+0000 7f1ba0c00640 1 -- 172.21.15.114:0/3457089919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9c0741f0 msgr2=0x7f1b9c1f0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:32.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:32.139+0000 7f1ba0c00640 1 --2- 172.21.15.114:0/3457089919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9c0741f0 0x7f1b9c1f0220 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00e870 tx=0x7f1b8c00ed30 comp rx=0 tx=0).stop 2024-05-27T09:32:32.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:32.139+0000 7f1ba0c00640 1 -- 172.21.15.114:0/3457089919 shutdown_connections 2024-05-27T09:32:32.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:32.139+0000 7f1ba0c00640 1 --2- 172.21.15.114:0/3457089919 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1b780762d0 0x7f1b780787e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:32.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:32.139+0000 7f1ba0c00640 1 --2- 172.21.15.114:0/3457089919 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b9c1547d0 0x7f1b9c147290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:32.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:32.139+0000 7f1ba0c00640 1 --2- 172.21.15.114:0/3457089919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b9c074840 0x7f1b9c14f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:32.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:32.139+0000 7f1ba0c00640 1 --2- 172.21.15.114:0/3457089919 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b9c0741f0 0x7f1b9c1f0220 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:32.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:32.139+0000 7f1ba0c00640 1 -- 172.21.15.114:0/3457089919 >> 172.21.15.114:0/3457089919 conn(0x7f1b9c0ca320 msgr2=0x7f1b9c153310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:32.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:32.140+0000 7f1ba0c00640 1 -- 172.21.15.114:0/3457089919 shutdown_connections 2024-05-27T09:32:32.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:32.140+0000 7f1ba0c00640 1 -- 172.21.15.114:0/3457089919 wait complete. 2024-05-27T09:32:32.168 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 25, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-05-27T09:27:18.079214+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 25, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4300": { "addr": "172.21.15.114:6837/1448950017", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6836", "nonce": 1448950017, "type": "v2" }, { "addr": "172.21.15.114:6837", "nonce": 1448950017, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4300, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 4 }, "gid_4828": { "addr": "172.21.15.163:6839/3371890103", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6837", "nonce": 3371890103, "type": "v2" }, { "addr": "172.21.15.163:6839", "nonce": 3371890103, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4828, "incarnation": 19, "join_fscid": -1, "name": "f", "rank": 2, "state": "up:active", "state_seq": 20 }, "gid_4875": { "addr": "172.21.15.114:6835/2747052034", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6833", "nonce": 2747052034, "type": "v2" }, { "addr": "172.21.15.114:6835", "nonce": 2747052034, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4875, "incarnation": 25, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 36, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-05-27T09:32:30.374582+0000", "qdb_cluster": [ 4300, 4828 ], "qdb_leader": 4300, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4300, "mds_1": 4875, "mds_2": 4828 } } } ], "standbys": [ { "addr": "172.21.15.163:6835/3513522318", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 3513522318, "type": "v2" }, { "addr": "172.21.15.163:6835", "nonce": 3513522318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6838/1753596112", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6836", "nonce": 1753596112, "type": "v2" }, { "addr": "172.21.15.163:6838", "nonce": 1753596112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-05-27T09:32:34.169 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:32:34.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.451+0000 7f5d01000640 1 Processor -- start 2024-05-27T09:32:34.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.452+0000 7f5d01000640 1 -- start start 2024-05-27T09:32:34.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.454+0000 7f5d01000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5cfc0ca9e0 0x7f5cfc15dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:34.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.454+0000 7f5d01000640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cfc15a810 con 0x7f5cfc154820 2024-05-27T09:32:34.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.454+0000 7f5d01000640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cfc15aaf0 con 0x7f5cfc074a60 2024-05-27T09:32:34.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.454+0000 7f5d01000640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f5cfc154c30 con 0x7f5cfc0ca9e0 2024-05-27T09:32:34.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.454+0000 7f5cf1400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f5cfc154820 0x7f5cfc0ca2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:54304/0 (socket says 172.21.15.114:54304) 2024-05-27T09:32:34.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.454+0000 7f5cf1400640 1 -- 172.21.15.114:0/3946960405 learned_addr learned my addr 172.21.15.114:0/3946960405 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:32:34.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.454+0000 7f5cfa800640 1 --2- 172.21.15.114:0/3946960405 >> v2:172.21.15.163:3300/0 conn(0x7f5cfc0ca9e0 0x7f5cfc15dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:34.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.455+0000 7f5cf9400640 1 -- 172.21.15.114:0/3946960405 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4151828311 0 0) 0x7f5cfc15a810 con 0x7f5cfc154820 2024-05-27T09:32:34.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.455+0000 7f5cf9400640 1 -- 172.21.15.114:0/3946960405 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ce0003520 con 0x7f5cfc154820 2024-05-27T09:32:34.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.455+0000 7f5cfa800640 1 -- 172.21.15.114:0/3946960405 >> v1:172.21.15.163:6789/0 conn(0x7f5cfc074a60 legacy=0x7f5cfc073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:34.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.455+0000 7f5cfa800640 1 -- 172.21.15.114:0/3946960405 >> v1:172.21.15.114:6789/0 conn(0x7f5cfc154820 legacy=0x7f5cfc0ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:34.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.455+0000 7f5cfa800640 1 -- 172.21.15.114:0/3946960405 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cfc15add0 con 0x7f5cfc0ca9e0 2024-05-27T09:32:34.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.456+0000 7f5cfa800640 1 --2- 172.21.15.114:0/3946960405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cfc0ca9e0 0x7f5cfc15dff0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40097a0 tx=0x7f5cf402a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c031a67ef699aeb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:34.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.456+0000 7f5cf9400640 1 -- 172.21.15.114:0/3946960405 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4037070 con 0x7f5cfc0ca9e0 2024-05-27T09:32:34.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.456+0000 7f5cf9400640 1 -- 172.21.15.114:0/3946960405 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cf4032640 con 0x7f5cfc0ca9e0 2024-05-27T09:32:34.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.456+0000 7f5cf9400640 1 -- 172.21.15.114:0/3946960405 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4032a10 con 0x7f5cfc0ca9e0 2024-05-27T09:32:34.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.457+0000 7f5d01000640 1 -- 172.21.15.114:0/3946960405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cfc0ca9e0 msgr2=0x7f5cfc15dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:34.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.457+0000 7f5d01000640 1 --2- 172.21.15.114:0/3946960405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cfc0ca9e0 0x7f5cfc15dff0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40097a0 tx=0x7f5cf402a2c0 comp rx=0 tx=0).stop 2024-05-27T09:32:34.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.457+0000 7f5d01000640 1 -- 172.21.15.114:0/3946960405 shutdown_connections 2024-05-27T09:32:34.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.457+0000 7f5d01000640 1 --2- 172.21.15.114:0/3946960405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cfc0ca9e0 0x7f5cfc15dff0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:34.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.457+0000 7f5d01000640 1 -- 172.21.15.114:0/3946960405 >> 172.21.15.114:0/3946960405 conn(0x7f5cfc06f6e0 msgr2=0x7f5cfc070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:34.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.457+0000 7f5d01000640 1 -- 172.21.15.114:0/3946960405 shutdown_connections 2024-05-27T09:32:34.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.457+0000 7f5d01000640 1 -- 172.21.15.114:0/3946960405 wait complete. 2024-05-27T09:32:34.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.460+0000 7f5d01000640 1 Processor -- start 2024-05-27T09:32:34.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.460+0000 7f5d01000640 1 -- start start 2024-05-27T09:32:34.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.461+0000 7f5d01000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cfc074a60 0x7f5cfc0b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:34.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.461+0000 7f5d01000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cfc0ca9e0 0x7f5cfc153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:34.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.461+0000 7f5d01000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cfc154820 0x7f5cfc146cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:34.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.461+0000 7f5d01000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5cfc15b390 con 0x7f5cfc074a60 2024-05-27T09:32:34.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.461+0000 7f5d01000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5cfc15db60 con 0x7f5cfc0ca9e0 2024-05-27T09:32:34.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.461+0000 7f5d01000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f5cfc073970 con 0x7f5cfc154820 2024-05-27T09:32:34.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.462+0000 7f5cf9e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cfc074a60 0x7f5cfc0b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:34.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.462+0000 7f5cf1400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cfc0ca9e0 0x7f5cfc153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:34.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.462+0000 7f5cf1400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cfc0ca9e0 0x7f5cfc153a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:58668/0 (socket says 172.21.15.114:58668) 2024-05-27T09:32:34.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.462+0000 7f5cf1400640 1 -- 172.21.15.114:0/2527553901 learned_addr learned my addr 172.21.15.114:0/2527553901 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:34.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.462+0000 7f5cfa800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cfc154820 0x7f5cfc146cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:34.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.462+0000 7f5cf9e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cfc074a60 0x7f5cfc0b1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42370/0 (socket says 172.21.15.114:42370) 2024-05-27T09:32:34.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.462+0000 7f5cf1400640 1 -- 172.21.15.114:0/2527553901 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cfc154820 msgr2=0x7f5cfc146cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:34.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.462+0000 7f5cf1400640 1 --2- 172.21.15.114:0/2527553901 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cfc154820 0x7f5cfc146cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:34.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.462+0000 7f5cf1400640 1 -- 172.21.15.114:0/2527553901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cfc074a60 msgr2=0x7f5cfc0b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:34.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.462+0000 7f5cf1400640 1 --2- 172.21.15.114:0/2527553901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cfc074a60 0x7f5cfc0b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:34.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.462+0000 7f5cf1400640 1 -- 172.21.15.114:0/2527553901 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cfc0cbcd0 con 0x7f5cfc0ca9e0 2024-05-27T09:32:34.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.462+0000 7f5cf9e00640 1 --2- 172.21.15.114:0/2527553901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cfc074a60 0x7f5cfc0b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:32:34.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.463+0000 7f5cf1400640 1 --2- 172.21.15.114:0/2527553901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cfc0ca9e0 0x7f5cfc153a70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5ce400c420 tx=0x7f5ce400c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:34.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.463+0000 7f5cf2a00640 1 -- 172.21.15.114:0/2527553901 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4017070 con 0x7f5cfc0ca9e0 2024-05-27T09:32:34.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.463+0000 7f5d01000640 1 -- 172.21.15.114:0/2527553901 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cfc153fb0 con 0x7f5cfc0ca9e0 2024-05-27T09:32:34.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.463+0000 7f5cf2a00640 1 -- 172.21.15.114:0/2527553901 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ce4005ce0 con 0x7f5cfc0ca9e0 2024-05-27T09:32:34.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.463+0000 7f5cf2a00640 1 -- 172.21.15.114:0/2527553901 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce40072d0 con 0x7f5cfc0ca9e0 2024-05-27T09:32:34.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.463+0000 7f5d01000640 1 -- 172.21.15.114:0/2527553901 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cfc1f2d30 con 0x7f5cfc0ca9e0 2024-05-27T09:32:34.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.464+0000 7f5d01000640 1 -- 172.21.15.114:0/2527553901 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cfc06a380 con 0x7f5cfc0ca9e0 2024-05-27T09:32:34.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.465+0000 7f5cf2a00640 1 -- 172.21.15.114:0/2527553901 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5ce401a070 con 0x7f5cfc0ca9e0 2024-05-27T09:32:34.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.466+0000 7f5cf2a00640 1 --2- 172.21.15.114:0/2527553901 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5cd80762d0 0x7f5cd80787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:34.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.467+0000 7f5cf2a00640 1 -- 172.21.15.114:0/2527553901 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6040+0+0 (secure 0 0 0) 0x7f5ce4096af0 con 0x7f5cfc0ca9e0 2024-05-27T09:32:34.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.467+0000 7f5cf9e00640 1 --2- 172.21.15.114:0/2527553901 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5cd80762d0 0x7f5cd80787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:34.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.467+0000 7f5cf9e00640 1 --2- 172.21.15.114:0/2527553901 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5cd80762d0 0x7f5cd80787e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5cec002210 tx=0x7f5cec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:34.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.471+0000 7f5cf2a00640 1 -- 172.21.15.114:0/2527553901 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5ce405fb00 con 0x7f5cfc0ca9e0 2024-05-27T09:32:34.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.712+0000 7f5d01000640 1 -- 172.21.15.114:0/2527553901 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cfc0c7c50 con 0x7f5cfc0ca9e0 2024-05-27T09:32:34.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.713+0000 7f5cf2a00640 1 -- 172.21.15.114:0/2527553901 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+6054 (secure 0 0 0) 0x7f5ce4063e00 con 0x7f5cfc0ca9e0 2024-05-27T09:32:34.714 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:32:34.715 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:32:30.374582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:replay","state_seq":1,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4828]},"id":1}]} 2024-05-27T09:32:34.715 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 25 2024-05-27T09:32:34.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.716+0000 7f5d01000640 1 -- 172.21.15.114:0/2527553901 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5cd80762d0 msgr2=0x7f5cd80787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:34.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.716+0000 7f5d01000640 1 --2- 172.21.15.114:0/2527553901 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5cd80762d0 0x7f5cd80787e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5cec002210 tx=0x7f5cec009040 comp rx=0 tx=0).stop 2024-05-27T09:32:34.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.716+0000 7f5d01000640 1 -- 172.21.15.114:0/2527553901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cfc0ca9e0 msgr2=0x7f5cfc153a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:34.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.716+0000 7f5d01000640 1 --2- 172.21.15.114:0/2527553901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cfc0ca9e0 0x7f5cfc153a70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5ce400c420 tx=0x7f5ce400c8e0 comp rx=0 tx=0).stop 2024-05-27T09:32:34.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.717+0000 7f5d01000640 1 -- 172.21.15.114:0/2527553901 shutdown_connections 2024-05-27T09:32:34.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.717+0000 7f5d01000640 1 --2- 172.21.15.114:0/2527553901 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5cd80762d0 0x7f5cd80787e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:34.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.717+0000 7f5d01000640 1 --2- 172.21.15.114:0/2527553901 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cfc154820 0x7f5cfc146cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:34.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.717+0000 7f5d01000640 1 --2- 172.21.15.114:0/2527553901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cfc0ca9e0 0x7f5cfc153a70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:34.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.717+0000 7f5d01000640 1 --2- 172.21.15.114:0/2527553901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cfc074a60 0x7f5cfc0b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:34.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.717+0000 7f5d01000640 1 -- 172.21.15.114:0/2527553901 >> 172.21.15.114:0/2527553901 conn(0x7f5cfc06f6e0 msgr2=0x7f5cfc0cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:34.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.717+0000 7f5d01000640 1 -- 172.21.15.114:0/2527553901 shutdown_connections 2024-05-27T09:32:34.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:34.717+0000 7f5d01000640 1 -- 172.21.15.114:0/2527553901 wait complete. 2024-05-27T09:32:34.741 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:32:36.741 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:32:37.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.019+0000 7f210ea00640 1 Processor -- start 2024-05-27T09:32:37.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.020+0000 7f210ea00640 1 -- start start 2024-05-27T09:32:37.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.022+0000 7f210ea00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f21081593b0 0x7f2108142720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:37.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.022+0000 7f210ea00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21080739a0 con 0x7f2108073260 2024-05-27T09:32:37.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.022+0000 7f210ea00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2108073c80 con 0x7f21080c2990 2024-05-27T09:32:37.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.022+0000 7f210ea00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2108074d20 con 0x7f21081593b0 2024-05-27T09:32:37.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.022+0000 7f2107e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f21081593b0 0x7f2108142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:37.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.022+0000 7f20ff400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f2108073260 0x7f210815a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:50034/0 (socket says 172.21.15.114:50034) 2024-05-27T09:32:37.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.022+0000 7f20ff400640 1 -- 172.21.15.114:0/677523455 learned_addr learned my addr 172.21.15.114:0/677523455 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:32:37.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.023+0000 7f2107e00640 1 -- 172.21.15.114:0/677523455 >> v1:172.21.15.163:6789/0 conn(0x7f21080c2990 legacy=0x7f2108163d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:37.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.023+0000 7f2107e00640 1 -- 172.21.15.114:0/677523455 >> v1:172.21.15.114:6789/0 conn(0x7f2108073260 legacy=0x7f210815a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:37.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.023+0000 7f2107e00640 1 -- 172.21.15.114:0/677523455 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2108073f60 con 0x7f21081593b0 2024-05-27T09:32:37.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.023+0000 7f2107400640 1 -- 172.21.15.114:0/677523455 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2248692870 0 0) 0x7f21080739a0 con 0x7f2108073260 2024-05-27T09:32:37.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.024+0000 7f2107e00640 1 --2- 172.21.15.114:0/677523455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21081593b0 0x7f2108142720 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009d90 tx=0x7f20f802c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4e7ae303947ae9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:37.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.024+0000 7f2107400640 1 -- 172.21.15.114:0/677523455 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8036020 con 0x7f21081593b0 2024-05-27T09:32:37.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.024+0000 7f2107400640 1 -- 172.21.15.114:0/677523455 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20f8032890 con 0x7f21081593b0 2024-05-27T09:32:37.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.024+0000 7f2107400640 1 -- 172.21.15.114:0/677523455 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8032c60 con 0x7f21081593b0 2024-05-27T09:32:37.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.025+0000 7f210ea00640 1 -- 172.21.15.114:0/677523455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21081593b0 msgr2=0x7f2108142720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:37.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.025+0000 7f210ea00640 1 --2- 172.21.15.114:0/677523455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21081593b0 0x7f2108142720 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009d90 tx=0x7f20f802c990 comp rx=0 tx=0).stop 2024-05-27T09:32:37.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.025+0000 7f210ea00640 1 -- 172.21.15.114:0/677523455 shutdown_connections 2024-05-27T09:32:37.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.025+0000 7f210ea00640 1 --2- 172.21.15.114:0/677523455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21081593b0 0x7f2108142720 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:37.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.025+0000 7f210ea00640 1 -- 172.21.15.114:0/677523455 >> 172.21.15.114:0/677523455 conn(0x7f210806f6e0 msgr2=0x7f2108070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:37.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.025+0000 7f210ea00640 1 -- 172.21.15.114:0/677523455 shutdown_connections 2024-05-27T09:32:37.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.025+0000 7f210ea00640 1 -- 172.21.15.114:0/677523455 wait complete. 2024-05-27T09:32:37.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.027+0000 7f210ea00640 1 Processor -- start 2024-05-27T09:32:37.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.028+0000 7f210ea00640 1 -- start start 2024-05-27T09:32:37.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.029+0000 7f210ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2108073260 0x7f21080c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:37.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.029+0000 7f210ea00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21080c2990 0x7f21080c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:37.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.029+0000 7f210ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21081593b0 0x7f210815cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:37.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.029+0000 7f210ea00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2108074800 con 0x7f2108073260 2024-05-27T09:32:37.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.029+0000 7f210ea00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2108074520 con 0x7f21081593b0 2024-05-27T09:32:37.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.029+0000 7f210ea00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f2108073670 con 0x7f21080c2990 2024-05-27T09:32:37.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.029+0000 7f2107e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2108073260 0x7f21080c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:37.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.029+0000 7f2107e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2108073260 0x7f21080c4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40768/0 (socket says 172.21.15.114:40768) 2024-05-27T09:32:37.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.029+0000 7f2107e00640 1 -- 172.21.15.114:0/1475531531 learned_addr learned my addr 172.21.15.114:0/1475531531 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:37.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.029+0000 7f20ff400640 1 --2- 172.21.15.114:0/1475531531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21080c2990 0x7f21080c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:37.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.029+0000 7f210ca00640 1 --2- 172.21.15.114:0/1475531531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21081593b0 0x7f210815cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:37.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.030+0000 7f2107e00640 1 -- 172.21.15.114:0/1475531531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21080c2990 msgr2=0x7f21080c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:37.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.030+0000 7f2107e00640 1 --2- 172.21.15.114:0/1475531531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21080c2990 0x7f21080c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:37.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.030+0000 7f2107e00640 1 -- 172.21.15.114:0/1475531531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21081593b0 msgr2=0x7f210815cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:37.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.030+0000 7f2107e00640 1 --2- 172.21.15.114:0/1475531531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21081593b0 0x7f210815cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:37.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.030+0000 7f2107e00640 1 -- 172.21.15.114:0/1475531531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21080c5430 con 0x7f2108073260 2024-05-27T09:32:37.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.030+0000 7f2107e00640 1 --2- 172.21.15.114:0/1475531531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2108073260 0x7f21080c4ef0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009d90 tx=0x7f20f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:37.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.030+0000 7f2104c00640 1 -- 172.21.15.114:0/1475531531 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8036040 con 0x7f2108073260 2024-05-27T09:32:37.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.030+0000 7f2104c00640 1 -- 172.21.15.114:0/1475531531 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20f8031ba0 con 0x7f2108073260 2024-05-27T09:32:37.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.030+0000 7f2104c00640 1 -- 172.21.15.114:0/1475531531 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8003990 con 0x7f2108073260 2024-05-27T09:32:37.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.031+0000 7f210ea00640 1 -- 172.21.15.114:0/1475531531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21081f2e50 con 0x7f2108073260 2024-05-27T09:32:37.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.031+0000 7f210ea00640 1 -- 172.21.15.114:0/1475531531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21081f3410 con 0x7f2108073260 2024-05-27T09:32:37.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.032+0000 7f210ea00640 1 -- 172.21.15.114:0/1475531531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f210806a380 con 0x7f2108073260 2024-05-27T09:32:37.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.033+0000 7f2104c00640 1 -- 172.21.15.114:0/1475531531 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f20f8043030 con 0x7f2108073260 2024-05-27T09:32:37.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.034+0000 7f2104c00640 1 --2- 172.21.15.114:0/1475531531 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f20e40762d0 0x7f20e40787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:37.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.034+0000 7f2104c00640 1 -- 172.21.15.114:0/1475531531 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6040+0+0 (secure 0 0 0) 0x7f20f80b86d0 con 0x7f2108073260 2024-05-27T09:32:37.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.034+0000 7f20ff400640 1 --2- 172.21.15.114:0/1475531531 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f20e40762d0 0x7f20e40787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:37.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.035+0000 7f20ff400640 1 --2- 172.21.15.114:0/1475531531 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f20e40762d0 0x7f20e40787e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f210815a4c0 tx=0x7f20f0007e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:37.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.039+0000 7f2104c00640 1 -- 172.21.15.114:0/1475531531 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f20f802e040 con 0x7f2108073260 2024-05-27T09:32:37.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.282+0000 7f210ea00640 1 -- 172.21.15.114:0/1475531531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21080c41f0 con 0x7f2108073260 2024-05-27T09:32:37.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.283+0000 7f2104c00640 1 -- 172.21.15.114:0/1475531531 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+6056 (secure 0 0 0) 0x7f20f8081760 con 0x7f2108073260 2024-05-27T09:32:37.285 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:32:37.285 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:32:35.412547+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:resolve","state_seq":19,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4828,4300]},"id":1}]} 2024-05-27T09:32:37.285 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 26 2024-05-27T09:32:37.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.286+0000 7f210ea00640 1 -- 172.21.15.114:0/1475531531 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f20e40762d0 msgr2=0x7f20e40787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:37.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.286+0000 7f210ea00640 1 --2- 172.21.15.114:0/1475531531 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f20e40762d0 0x7f20e40787e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f210815a4c0 tx=0x7f20f0007e40 comp rx=0 tx=0).stop 2024-05-27T09:32:37.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.286+0000 7f210ea00640 1 -- 172.21.15.114:0/1475531531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2108073260 msgr2=0x7f21080c4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:37.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.286+0000 7f210ea00640 1 --2- 172.21.15.114:0/1475531531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2108073260 0x7f21080c4ef0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009d90 tx=0x7f20f8004a40 comp rx=0 tx=0).stop 2024-05-27T09:32:37.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.286+0000 7f210ea00640 1 -- 172.21.15.114:0/1475531531 shutdown_connections 2024-05-27T09:32:37.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.286+0000 7f210ea00640 1 --2- 172.21.15.114:0/1475531531 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f20e40762d0 0x7f20e40787e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:37.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.286+0000 7f210ea00640 1 --2- 172.21.15.114:0/1475531531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21081593b0 0x7f210815cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:37.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.286+0000 7f210ea00640 1 --2- 172.21.15.114:0/1475531531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21080c2990 0x7f21080c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:37.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.286+0000 7f210ea00640 1 --2- 172.21.15.114:0/1475531531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2108073260 0x7f21080c4ef0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:37.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.286+0000 7f210ea00640 1 -- 172.21.15.114:0/1475531531 >> 172.21.15.114:0/1475531531 conn(0x7f210806f6e0 msgr2=0x7f21080b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:37.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.286+0000 7f210ea00640 1 -- 172.21.15.114:0/1475531531 shutdown_connections 2024-05-27T09:32:37.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:37.286+0000 7f210ea00640 1 -- 172.21.15.114:0/1475531531 wait complete. 2024-05-27T09:32:37.312 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:32:39.313 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:32:39.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.593+0000 7f9453200640 1 Processor -- start 2024-05-27T09:32:39.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.593+0000 7f9453200640 1 -- start start 2024-05-27T09:32:39.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.595+0000 7f9453200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f944c1534c0 0x7f944c0c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:39.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.595+0000 7f9453200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f944c0c8220 0x7f944c159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:39.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.595+0000 7f9453200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f944c0c8630 0x7f944c074140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:39.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.595+0000 7f9453200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f944c0c2f40 con 0x7f944c1534c0 2024-05-27T09:32:39.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.595+0000 7f9453200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f944c159190 con 0x7f944c0c8220 2024-05-27T09:32:39.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.595+0000 7f9453200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f944c1538d0 con 0x7f944c0c8630 2024-05-27T09:32:39.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.595+0000 7f9451e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f944c1534c0 0x7f944c0c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:39.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.595+0000 7f9451e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f944c1534c0 0x7f944c0c2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40778/0 (socket says 172.21.15.114:40778) 2024-05-27T09:32:39.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.595+0000 7f9451e00640 1 -- 172.21.15.114:0/1244229237 learned_addr learned my addr 172.21.15.114:0/1244229237 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:39.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.595+0000 7f9451400640 1 --2- 172.21.15.114:0/1244229237 >> v2:172.21.15.163:3300/0 conn(0x7f944c0c8220 0x7f944c159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:39.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.595+0000 7f9452800640 1 --2- 172.21.15.114:0/1244229237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f944c0c8630 0x7f944c074140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:39.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.596+0000 7f9451e00640 1 -- 172.21.15.114:0/1244229237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f944c0c8630 msgr2=0x7f944c074140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:32:39.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.596+0000 7f9451e00640 1 --2- 172.21.15.114:0/1244229237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f944c0c8630 0x7f944c074140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:39.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.596+0000 7f9451e00640 1 -- 172.21.15.114:0/1244229237 >> v2:172.21.15.163:3300/0 conn(0x7f944c0c8220 msgr2=0x7f944c159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:32:39.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.596+0000 7f9451e00640 1 --2- 172.21.15.114:0/1244229237 >> v2:172.21.15.163:3300/0 conn(0x7f944c0c8220 0x7f944c159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:39.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.596+0000 7f9451e00640 1 -- 172.21.15.114:0/1244229237 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944c0c96d0 con 0x7f944c1534c0 2024-05-27T09:32:39.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.597+0000 7f9451e00640 1 --2- 172.21.15.114:0/1244229237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f944c1534c0 0x7f944c0c2a00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f944c071be0 tx=0x7f944002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9adedaed4e753b8e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:39.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.597+0000 7f9450a00640 1 -- 172.21.15.114:0/1244229237 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440037070 con 0x7f944c1534c0 2024-05-27T09:32:39.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.597+0000 7f9450a00640 1 -- 172.21.15.114:0/1244229237 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9440032440 con 0x7f944c1534c0 2024-05-27T09:32:39.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.597+0000 7f9450a00640 1 -- 172.21.15.114:0/1244229237 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440032810 con 0x7f944c1534c0 2024-05-27T09:32:39.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.598+0000 7f9453200640 1 -- 172.21.15.114:0/1244229237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f944c1534c0 msgr2=0x7f944c0c2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:39.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.598+0000 7f9453200640 1 --2- 172.21.15.114:0/1244229237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f944c1534c0 0x7f944c0c2a00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f944c071be0 tx=0x7f944002a1e0 comp rx=0 tx=0).stop 2024-05-27T09:32:39.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.598+0000 7f9453200640 1 -- 172.21.15.114:0/1244229237 shutdown_connections 2024-05-27T09:32:39.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.598+0000 7f9453200640 1 --2- 172.21.15.114:0/1244229237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f944c0c8630 0x7f944c074140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:39.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.598+0000 7f9453200640 1 --2- 172.21.15.114:0/1244229237 >> v2:172.21.15.163:3300/0 conn(0x7f944c0c8220 0x7f944c159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:39.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.598+0000 7f9453200640 1 --2- 172.21.15.114:0/1244229237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f944c1534c0 0x7f944c0c2a00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:39.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.598+0000 7f9453200640 1 -- 172.21.15.114:0/1244229237 >> 172.21.15.114:0/1244229237 conn(0x7f944c0ca320 msgr2=0x7f944c0ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:39.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.598+0000 7f9453200640 1 -- 172.21.15.114:0/1244229237 shutdown_connections 2024-05-27T09:32:39.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.598+0000 7f9453200640 1 -- 172.21.15.114:0/1244229237 wait complete. 2024-05-27T09:32:39.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.600+0000 7f9453200640 1 Processor -- start 2024-05-27T09:32:39.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.600+0000 7f9453200640 1 -- start start 2024-05-27T09:32:39.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.602+0000 7f9453200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f944c0c8220 0x7f944c1f6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:39.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.602+0000 7f9453200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f944c0c8630 0x7f944c0c6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:39.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.602+0000 7f9453200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f944c1534c0 0x7f944c073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:39.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.602+0000 7f9453200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f944c0c9f30 con 0x7f944c0c8220 2024-05-27T09:32:39.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.602+0000 7f9453200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f944c0c9c50 con 0x7f944c1534c0 2024-05-27T09:32:39.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.602+0000 7f9453200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f944c1593e0 con 0x7f944c0c8630 2024-05-27T09:32:39.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.602+0000 7f9451e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f944c0c8220 0x7f944c1f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:39.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.602+0000 7f9451e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f944c0c8220 0x7f944c1f6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40794/0 (socket says 172.21.15.114:40794) 2024-05-27T09:32:39.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.602+0000 7f9451e00640 1 -- 172.21.15.114:0/2044001006 learned_addr learned my addr 172.21.15.114:0/2044001006 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:39.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.602+0000 7f9452800640 1 --2- 172.21.15.114:0/2044001006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f944c1534c0 0x7f944c073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:39.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.602+0000 7f9451400640 1 --2- 172.21.15.114:0/2044001006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f944c0c8630 0x7f944c0c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:39.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.603+0000 7f9451e00640 1 -- 172.21.15.114:0/2044001006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f944c0c8630 msgr2=0x7f944c0c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:39.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.603+0000 7f9451e00640 1 --2- 172.21.15.114:0/2044001006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f944c0c8630 0x7f944c0c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:39.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.603+0000 7f9451e00640 1 -- 172.21.15.114:0/2044001006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f944c1534c0 msgr2=0x7f944c073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:39.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.603+0000 7f9451e00640 1 --2- 172.21.15.114:0/2044001006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f944c1534c0 0x7f944c073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:39.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.603+0000 7f9451e00640 1 -- 172.21.15.114:0/2044001006 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944c153d60 con 0x7f944c0c8220 2024-05-27T09:32:39.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.603+0000 7f9452800640 1 --2- 172.21.15.114:0/2044001006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f944c1534c0 0x7f944c073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:32:39.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.603+0000 7f9451e00640 1 --2- 172.21.15.114:0/2044001006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f944c0c8220 0x7f944c1f6530 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9440000c00 tx=0x7f94400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:39.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.603+0000 7f9446000640 1 -- 172.21.15.114:0/2044001006 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440041020 con 0x7f944c0c8220 2024-05-27T09:32:39.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.603+0000 7f9446000640 1 -- 172.21.15.114:0/2044001006 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f944003d620 con 0x7f944c0c8220 2024-05-27T09:32:39.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.603+0000 7f9446000640 1 -- 172.21.15.114:0/2044001006 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440037040 con 0x7f944c0c8220 2024-05-27T09:32:39.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.603+0000 7f9453200640 1 -- 172.21.15.114:0/2044001006 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f944c1e4060 con 0x7f944c0c8220 2024-05-27T09:32:39.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.604+0000 7f9453200640 1 -- 172.21.15.114:0/2044001006 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f944c1e4620 con 0x7f944c0c8220 2024-05-27T09:32:39.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.605+0000 7f9453200640 1 -- 172.21.15.114:0/2044001006 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f944c050860 con 0x7f944c0c8220 2024-05-27T09:32:39.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.606+0000 7f9446000640 1 -- 172.21.15.114:0/2044001006 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9440044020 con 0x7f944c0c8220 2024-05-27T09:32:39.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.607+0000 7f9446000640 1 --2- 172.21.15.114:0/2044001006 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9430076300 0x7f9430078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:39.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.607+0000 7f9446000640 1 -- 172.21.15.114:0/2044001006 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6040+0+0 (secure 0 0 0) 0x7f94400b7d20 con 0x7f944c0c8220 2024-05-27T09:32:39.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.607+0000 7f9451400640 1 --2- 172.21.15.114:0/2044001006 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9430076300 0x7f9430078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:39.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.608+0000 7f9451400640 1 --2- 172.21.15.114:0/2044001006 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9430076300 0x7f9430078810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f944c14a080 tx=0x7f943c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:39.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.614+0000 7f9446000640 1 -- 172.21.15.114:0/2044001006 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f944004b030 con 0x7f944c0c8220 2024-05-27T09:32:39.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.862+0000 7f9453200640 1 -- 172.21.15.114:0/2044001006 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f944c0c40f0 con 0x7f944c0c8220 2024-05-27T09:32:39.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.864+0000 7f9446000640 1 -- 172.21.15.114:0/2044001006 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+6056 (secure 0 0 0) 0x7f9440080e40 con 0x7f944c0c8220 2024-05-27T09:32:39.865 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:32:39.865 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:32:35.412547+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:resolve","state_seq":19,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4828,4300]},"id":1}]} 2024-05-27T09:32:39.866 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 26 2024-05-27T09:32:39.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.867+0000 7f9453200640 1 -- 172.21.15.114:0/2044001006 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9430076300 msgr2=0x7f9430078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:39.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.867+0000 7f9453200640 1 --2- 172.21.15.114:0/2044001006 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9430076300 0x7f9430078810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f944c14a080 tx=0x7f943c009190 comp rx=0 tx=0).stop 2024-05-27T09:32:39.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.867+0000 7f9453200640 1 -- 172.21.15.114:0/2044001006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f944c0c8220 msgr2=0x7f944c1f6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:39.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.867+0000 7f9453200640 1 --2- 172.21.15.114:0/2044001006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f944c0c8220 0x7f944c1f6530 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9440000c00 tx=0x7f94400046c0 comp rx=0 tx=0).stop 2024-05-27T09:32:39.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.867+0000 7f9453200640 1 -- 172.21.15.114:0/2044001006 shutdown_connections 2024-05-27T09:32:39.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.867+0000 7f9453200640 1 --2- 172.21.15.114:0/2044001006 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9430076300 0x7f9430078810 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:39.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.867+0000 7f9453200640 1 --2- 172.21.15.114:0/2044001006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f944c1534c0 0x7f944c073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:39.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.867+0000 7f9453200640 1 --2- 172.21.15.114:0/2044001006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f944c0c8630 0x7f944c0c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:39.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.867+0000 7f9453200640 1 --2- 172.21.15.114:0/2044001006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f944c0c8220 0x7f944c1f6530 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:39.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.867+0000 7f9453200640 1 -- 172.21.15.114:0/2044001006 >> 172.21.15.114:0/2044001006 conn(0x7f944c0ca320 msgr2=0x7f944c152520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:39.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.867+0000 7f9453200640 1 -- 172.21.15.114:0/2044001006 shutdown_connections 2024-05-27T09:32:39.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:39.867+0000 7f9453200640 1 -- 172.21.15.114:0/2044001006 wait complete. 2024-05-27T09:32:39.896 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:32:41.897 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:32:42.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.174+0000 7f0a51600640 1 Processor -- start 2024-05-27T09:32:42.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.174+0000 7f0a51600640 1 -- start start 2024-05-27T09:32:42.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.176+0000 7f0a51600640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0a4c156790 0x7f0a4c0c9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:42.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.176+0000 7f0a51600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a4c0c9960 0x7f0a4c166dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:42.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.176+0000 7f0a51600640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a4c0c54f0 con 0x7f0a4c068990 2024-05-27T09:32:42.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.176+0000 7f0a51600640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f0a4c0c52a0 con 0x7f0a4c156790 2024-05-27T09:32:42.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.176+0000 7f0a51600640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f0a4c156ba0 con 0x7f0a4c0c9960 2024-05-27T09:32:42.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.176+0000 7f0a4be00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f0a4c068990 0x7f0a4c074360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:50044/0 (socket says 172.21.15.114:50044) 2024-05-27T09:32:42.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.176+0000 7f0a4be00640 1 -- 172.21.15.114:0/436037964 learned_addr learned my addr 172.21.15.114:0/436037964 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:32:42.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.176+0000 7f0a4b400640 1 --2- 172.21.15.114:0/436037964 >> v2:172.21.15.114:3300/0 conn(0x7f0a4c156790 0x7f0a4c0c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:42.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.176+0000 7f0a50c00640 1 --2- 172.21.15.114:0/436037964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a4c0c9960 0x7f0a4c166dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:42.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.177+0000 7f0a4aa00640 1 -- 172.21.15.114:0/436037964 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2592359014 0 0) 0x7f0a4c0c54f0 con 0x7f0a4c068990 2024-05-27T09:32:42.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.177+0000 7f0a4b400640 1 -- 172.21.15.114:0/436037964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a4c0c9960 msgr2=0x7f0a4c166dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:32:42.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.177+0000 7f0a4b400640 1 --2- 172.21.15.114:0/436037964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a4c0c9960 0x7f0a4c166dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:42.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.177+0000 7f0a4b400640 1 -- 172.21.15.114:0/436037964 >> v1:172.21.15.114:6789/0 conn(0x7f0a4c068990 legacy=0x7f0a4c074360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:42.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.177+0000 7f0a4b400640 1 -- 172.21.15.114:0/436037964 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4c0ca0d0 con 0x7f0a4c156790 2024-05-27T09:32:42.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.178+0000 7f0a4b400640 1 --2- 172.21.15.114:0/436037964 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a4c156790 0x7f0a4c0c9420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c009d90 tx=0x7f0a3c02a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bbc208a6941e5b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:42.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.178+0000 7f0a4aa00640 1 -- 172.21.15.114:0/436037964 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c007740 con 0x7f0a4c156790 2024-05-27T09:32:42.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.178+0000 7f0a4aa00640 1 -- 172.21.15.114:0/436037964 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a3c0078e0 con 0x7f0a4c156790 2024-05-27T09:32:42.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.178+0000 7f0a4aa00640 1 -- 172.21.15.114:0/436037964 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c007cb0 con 0x7f0a4c156790 2024-05-27T09:32:42.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.179+0000 7f0a51600640 1 -- 172.21.15.114:0/436037964 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a4c156790 msgr2=0x7f0a4c0c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:42.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.179+0000 7f0a51600640 1 --2- 172.21.15.114:0/436037964 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a4c156790 0x7f0a4c0c9420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c009d90 tx=0x7f0a3c02a530 comp rx=0 tx=0).stop 2024-05-27T09:32:42.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.179+0000 7f0a51600640 1 -- 172.21.15.114:0/436037964 shutdown_connections 2024-05-27T09:32:42.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.179+0000 7f0a51600640 1 --2- 172.21.15.114:0/436037964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a4c0c9960 0x7f0a4c166dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:42.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.179+0000 7f0a51600640 1 --2- 172.21.15.114:0/436037964 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a4c156790 0x7f0a4c0c9420 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:42.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.179+0000 7f0a51600640 1 -- 172.21.15.114:0/436037964 >> 172.21.15.114:0/436037964 conn(0x7f0a4c073840 msgr2=0x7f0a4c073c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:42.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.179+0000 7f0a51600640 1 -- 172.21.15.114:0/436037964 shutdown_connections 2024-05-27T09:32:42.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.179+0000 7f0a51600640 1 -- 172.21.15.114:0/436037964 wait complete. 2024-05-27T09:32:42.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.181+0000 7f0a51600640 1 Processor -- start 2024-05-27T09:32:42.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.181+0000 7f0a51600640 1 -- start start 2024-05-27T09:32:42.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.183+0000 7f0a51600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a4c068990 0x7f0a4c1553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:42.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.183+0000 7f0a51600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a4c0c9960 0x7f0a4c1f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:42.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.183+0000 7f0a51600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a4c156790 0x7f0a4c148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:42.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.183+0000 7f0a51600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c0c6900 con 0x7f0a4c068990 2024-05-27T09:32:42.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.183+0000 7f0a51600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c06df10 con 0x7f0a4c156790 2024-05-27T09:32:42.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.183+0000 7f0a51600640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f0a4c0c8140 con 0x7f0a4c0c9960 2024-05-27T09:32:42.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.183+0000 7f0a4be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a4c068990 0x7f0a4c1553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:42.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.183+0000 7f0a4be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a4c068990 0x7f0a4c1553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40812/0 (socket says 172.21.15.114:40812) 2024-05-27T09:32:42.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.183+0000 7f0a4be00640 1 -- 172.21.15.114:0/1120730873 learned_addr learned my addr 172.21.15.114:0/1120730873 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:42.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.183+0000 7f0a50c00640 1 --2- 172.21.15.114:0/1120730873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a4c156790 0x7f0a4c148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:42.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.183+0000 7f0a4b400640 1 --2- 172.21.15.114:0/1120730873 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a4c0c9960 0x7f0a4c1f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:42.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.184+0000 7f0a4be00640 1 -- 172.21.15.114:0/1120730873 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a4c0c9960 msgr2=0x7f0a4c1f31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:42.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.184+0000 7f0a4be00640 1 --2- 172.21.15.114:0/1120730873 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a4c0c9960 0x7f0a4c1f31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:42.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.184+0000 7f0a4be00640 1 -- 172.21.15.114:0/1120730873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a4c156790 msgr2=0x7f0a4c148990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:42.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.184+0000 7f0a4be00640 1 --2- 172.21.15.114:0/1120730873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a4c156790 0x7f0a4c148990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:42.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.184+0000 7f0a4be00640 1 -- 172.21.15.114:0/1120730873 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4c1f3720 con 0x7f0a4c068990 2024-05-27T09:32:42.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.184+0000 7f0a4be00640 1 --2- 172.21.15.114:0/1120730873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a4c068990 0x7f0a4c1553f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400c3e0 tx=0x7f0a3400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:42.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.184+0000 7f0a43e00640 1 -- 172.21.15.114:0/1120730873 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a340074e0 con 0x7f0a4c068990 2024-05-27T09:32:42.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.184+0000 7f0a43e00640 1 -- 172.21.15.114:0/1120730873 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a340047e0 con 0x7f0a4c068990 2024-05-27T09:32:42.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.184+0000 7f0a43e00640 1 -- 172.21.15.114:0/1120730873 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34004bd0 con 0x7f0a4c068990 2024-05-27T09:32:42.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.184+0000 7f0a51600640 1 -- 172.21.15.114:0/1120730873 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a4c1f2a20 con 0x7f0a4c068990 2024-05-27T09:32:42.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.184+0000 7f0a51600640 1 -- 172.21.15.114:0/1120730873 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a4c1f6a10 con 0x7f0a4c068990 2024-05-27T09:32:42.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.186+0000 7f0a51600640 1 -- 172.21.15.114:0/1120730873 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a4c050860 con 0x7f0a4c068990 2024-05-27T09:32:42.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.187+0000 7f0a43e00640 1 -- 172.21.15.114:0/1120730873 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0a34011070 con 0x7f0a4c068990 2024-05-27T09:32:42.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.188+0000 7f0a43e00640 1 --2- 172.21.15.114:0/1120730873 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0a28076300 0x7f0a28078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:42.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.188+0000 7f0a43e00640 1 -- 172.21.15.114:0/1120730873 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6040+0+0 (secure 0 0 0) 0x7f0a340971d0 con 0x7f0a4c068990 2024-05-27T09:32:42.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.188+0000 7f0a4b400640 1 --2- 172.21.15.114:0/1120730873 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0a28076300 0x7f0a28078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:42.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.189+0000 7f0a4b400640 1 --2- 172.21.15.114:0/1120730873 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0a28076300 0x7f0a28078810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c034000 tx=0x7f0a3c02f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:42.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.193+0000 7f0a43e00640 1 -- 172.21.15.114:0/1120730873 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0a34060380 con 0x7f0a4c068990 2024-05-27T09:32:42.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.441+0000 7f0a51600640 1 -- 172.21.15.114:0/1120730873 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a4c15d3e0 con 0x7f0a4c068990 2024-05-27T09:32:42.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.442+0000 7f0a43e00640 1 -- 172.21.15.114:0/1120730873 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6058 (secure 0 0 0) 0x7f0a34064680 con 0x7f0a4c068990 2024-05-27T09:32:42.443 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:32:42.443 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:32:41.388124+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:reconnect","state_seq":21,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4828]},"id":1}]} 2024-05-27T09:32:42.444 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 27 2024-05-27T09:32:42.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.445+0000 7f0a51600640 1 -- 172.21.15.114:0/1120730873 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0a28076300 msgr2=0x7f0a28078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:42.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.445+0000 7f0a51600640 1 --2- 172.21.15.114:0/1120730873 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0a28076300 0x7f0a28078810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c034000 tx=0x7f0a3c02f000 comp rx=0 tx=0).stop 2024-05-27T09:32:42.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.445+0000 7f0a51600640 1 -- 172.21.15.114:0/1120730873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a4c068990 msgr2=0x7f0a4c1553f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:42.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.445+0000 7f0a51600640 1 --2- 172.21.15.114:0/1120730873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a4c068990 0x7f0a4c1553f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400c3e0 tx=0x7f0a3400c8a0 comp rx=0 tx=0).stop 2024-05-27T09:32:42.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.445+0000 7f0a51600640 1 -- 172.21.15.114:0/1120730873 shutdown_connections 2024-05-27T09:32:42.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.445+0000 7f0a51600640 1 --2- 172.21.15.114:0/1120730873 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0a28076300 0x7f0a28078810 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:42.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.445+0000 7f0a51600640 1 --2- 172.21.15.114:0/1120730873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a4c156790 0x7f0a4c148990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:42.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.445+0000 7f0a51600640 1 --2- 172.21.15.114:0/1120730873 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a4c0c9960 0x7f0a4c1f31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:42.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.445+0000 7f0a51600640 1 --2- 172.21.15.114:0/1120730873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a4c068990 0x7f0a4c1553f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:42.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.445+0000 7f0a51600640 1 -- 172.21.15.114:0/1120730873 >> 172.21.15.114:0/1120730873 conn(0x7f0a4c073840 msgr2=0x7f0a4c071090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:42.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.445+0000 7f0a51600640 1 -- 172.21.15.114:0/1120730873 shutdown_connections 2024-05-27T09:32:42.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:42.445+0000 7f0a51600640 1 -- 172.21.15.114:0/1120730873 wait complete. 2024-05-27T09:32:42.471 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:32:44.472 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:32:44.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.753+0000 7f38e4a00640 1 Processor -- start 2024-05-27T09:32:44.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.754+0000 7f38e4a00640 1 -- start start 2024-05-27T09:32:44.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.756+0000 7f38e4a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f38e0156980 0x7f38e0160330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:44.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.756+0000 7f38e4a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38e0160870 0x7f38e00cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:44.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.756+0000 7f38e4a00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f38e00cc890 0x7f38e00cb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:44.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.756+0000 7f38e4a00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f38e006ed40 con 0x7f38e0156980 2024-05-27T09:32:44.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.756+0000 7f38e4a00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f38e006e1c0 con 0x7f38e00cc890 2024-05-27T09:32:44.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.756+0000 7f38e4a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f38e006e4a0 con 0x7f38e0160870 2024-05-27T09:32:44.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.756+0000 7f38de800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f38e0156980 0x7f38e0160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:44.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.756+0000 7f38de800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f38e0156980 0x7f38e0160330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40818/0 (socket says 172.21.15.114:40818) 2024-05-27T09:32:44.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.756+0000 7f38de800640 1 -- 172.21.15.114:0/2228084188 learned_addr learned my addr 172.21.15.114:0/2228084188 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:44.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.756+0000 7f38dde00640 1 --2- 172.21.15.114:0/2228084188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38e0160870 0x7f38e00cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:44.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.756+0000 7f38df200640 1 --2- 172.21.15.114:0/2228084188 >> v2:172.21.15.163:3300/0 conn(0x7f38e00cc890 0x7f38e00cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:44.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.757+0000 7f38de800640 1 -- 172.21.15.114:0/2228084188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38e0160870 msgr2=0x7f38e00cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:32:44.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.757+0000 7f38de800640 1 --2- 172.21.15.114:0/2228084188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38e0160870 0x7f38e00cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:44.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.757+0000 7f38de800640 1 -- 172.21.15.114:0/2228084188 >> v2:172.21.15.163:3300/0 conn(0x7f38e00cc890 msgr2=0x7f38e00cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:32:44.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.757+0000 7f38de800640 1 --2- 172.21.15.114:0/2228084188 >> v2:172.21.15.163:3300/0 conn(0x7f38e00cc890 0x7f38e00cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:44.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.757+0000 7f38de800640 1 -- 172.21.15.114:0/2228084188 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e006e780 con 0x7f38e0156980 2024-05-27T09:32:44.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.758+0000 7f38de800640 1 --2- 172.21.15.114:0/2228084188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38e0156980 0x7f38e0160330 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f38e006e6b0 tx=0x7f38c802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c8271e41e1ac67 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:44.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.758+0000 7f38dd400640 1 -- 172.21.15.114:0/2228084188 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8037070 con 0x7f38e0156980 2024-05-27T09:32:44.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.758+0000 7f38dd400640 1 -- 172.21.15.114:0/2228084188 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38c8032440 con 0x7f38e0156980 2024-05-27T09:32:44.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.758+0000 7f38dd400640 1 -- 172.21.15.114:0/2228084188 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8032810 con 0x7f38e0156980 2024-05-27T09:32:44.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.759+0000 7f38e4a00640 1 -- 172.21.15.114:0/2228084188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38e0156980 msgr2=0x7f38e0160330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:44.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.759+0000 7f38e4a00640 1 --2- 172.21.15.114:0/2228084188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38e0156980 0x7f38e0160330 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f38e006e6b0 tx=0x7f38c802a130 comp rx=0 tx=0).stop 2024-05-27T09:32:44.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.759+0000 7f38e4a00640 1 -- 172.21.15.114:0/2228084188 shutdown_connections 2024-05-27T09:32:44.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.759+0000 7f38e4a00640 1 --2- 172.21.15.114:0/2228084188 >> v2:172.21.15.163:3300/0 conn(0x7f38e00cc890 0x7f38e00cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:44.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.759+0000 7f38e4a00640 1 --2- 172.21.15.114:0/2228084188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38e0160870 0x7f38e00cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:44.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.759+0000 7f38e4a00640 1 --2- 172.21.15.114:0/2228084188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38e0156980 0x7f38e0160330 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:44.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.759+0000 7f38e4a00640 1 -- 172.21.15.114:0/2228084188 >> 172.21.15.114:0/2228084188 conn(0x7f38e0071460 msgr2=0x7f38e0068600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:44.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.759+0000 7f38e4a00640 1 -- 172.21.15.114:0/2228084188 shutdown_connections 2024-05-27T09:32:44.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.759+0000 7f38e4a00640 1 -- 172.21.15.114:0/2228084188 wait complete. 2024-05-27T09:32:44.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.761+0000 7f38e4a00640 1 Processor -- start 2024-05-27T09:32:44.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.761+0000 7f38e4a00640 1 -- start start 2024-05-27T09:32:44.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.763+0000 7f38e4a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38e00cc890 0x7f38e00b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:44.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.763+0000 7f38e4a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38e0156980 0x7f38e014c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:44.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.763+0000 7f38e4a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38e0160870 0x7f38e01f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:44.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.763+0000 7f38e4a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f38e0156d90 con 0x7f38e0160870 2024-05-27T09:32:44.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.763+0000 7f38e4a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f38e015fe10 con 0x7f38e00cc890 2024-05-27T09:32:44.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.763+0000 7f38e4a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f38e0156f10 con 0x7f38e0156980 2024-05-27T09:32:44.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.763+0000 7f38dde00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38e0156980 0x7f38e014c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:44.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.763+0000 7f38de800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38e00cc890 0x7f38e00b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:44.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.763+0000 7f38de800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38e00cc890 0x7f38e00b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:50686/0 (socket says 172.21.15.114:50686) 2024-05-27T09:32:44.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.763+0000 7f38dde00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38e0156980 0x7f38e014c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:39912/0 (socket says 172.21.15.114:39912) 2024-05-27T09:32:44.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.763+0000 7f38de800640 1 -- 172.21.15.114:0/2615134841 learned_addr learned my addr 172.21.15.114:0/2615134841 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:44.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.763+0000 7f38df200640 1 --2- 172.21.15.114:0/2615134841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38e0160870 0x7f38e01f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:44.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.764+0000 7f38de800640 1 -- 172.21.15.114:0/2615134841 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38e0156980 msgr2=0x7f38e014c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:44.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.764+0000 7f38de800640 1 --2- 172.21.15.114:0/2615134841 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38e0156980 0x7f38e014c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:44.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.764+0000 7f38de800640 1 -- 172.21.15.114:0/2615134841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38e0160870 msgr2=0x7f38e01f24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:44.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.764+0000 7f38de800640 1 --2- 172.21.15.114:0/2615134841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38e0160870 0x7f38e01f24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:44.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.764+0000 7f38de800640 1 -- 172.21.15.114:0/2615134841 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e015e100 con 0x7f38e00cc890 2024-05-27T09:32:44.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.764+0000 7f38de800640 1 --2- 172.21.15.114:0/2615134841 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38e00cc890 0x7f38e00b3c30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f38c802a170 tx=0x7f38c80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:44.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.764+0000 7f38d6a00640 1 -- 172.21.15.114:0/2615134841 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8041020 con 0x7f38e00cc890 2024-05-27T09:32:44.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.764+0000 7f38d6a00640 1 -- 172.21.15.114:0/2615134841 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38c803d620 con 0x7f38e00cc890 2024-05-27T09:32:44.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.764+0000 7f38d6a00640 1 -- 172.21.15.114:0/2615134841 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8037040 con 0x7f38e00cc890 2024-05-27T09:32:44.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.764+0000 7f38e4a00640 1 -- 172.21.15.114:0/2615134841 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e00c5cc0 con 0x7f38e00cc890 2024-05-27T09:32:44.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.765+0000 7f38e4a00640 1 -- 172.21.15.114:0/2615134841 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e00c6280 con 0x7f38e00cc890 2024-05-27T09:32:44.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.766+0000 7f38e4a00640 1 -- 172.21.15.114:0/2615134841 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38e006d5d0 con 0x7f38e00cc890 2024-05-27T09:32:44.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.767+0000 7f38d6a00640 1 -- 172.21.15.114:0/2615134841 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f38c8044020 con 0x7f38e00cc890 2024-05-27T09:32:44.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.768+0000 7f38d6a00640 1 --2- 172.21.15.114:0/2615134841 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f38bc0762d0 0x7f38bc0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:44.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.768+0000 7f38d6a00640 1 -- 172.21.15.114:0/2615134841 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6040+0+0 (secure 0 0 0) 0x7f38c80b7ee0 con 0x7f38e00cc890 2024-05-27T09:32:44.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.768+0000 7f38dde00640 1 --2- 172.21.15.114:0/2615134841 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f38bc0762d0 0x7f38bc0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:44.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.769+0000 7f38dde00640 1 --2- 172.21.15.114:0/2615134841 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f38bc0762d0 0x7f38bc0787e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f38e014d2e0 tx=0x7f38d000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:44.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:44.773+0000 7f38d6a00640 1 -- 172.21.15.114:0/2615134841 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f38c804b030 con 0x7f38e00cc890 2024-05-27T09:32:45.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:45.015+0000 7f38e4a00640 1 -- 172.21.15.114:0/2615134841 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38e006dc60 con 0x7f38e00cc890 2024-05-27T09:32:45.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:45.016+0000 7f38d6a00640 1 -- 172.21.15.114:0/2615134841 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6058 (secure 0 0 0) 0x7f38c8080ef0 con 0x7f38e00cc890 2024-05-27T09:32:45.017 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:32:45.017 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:32:41.388124+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:reconnect","state_seq":21,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4828]},"id":1}]} 2024-05-27T09:32:45.018 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 27 2024-05-27T09:32:45.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:45.018+0000 7f38e4a00640 1 -- 172.21.15.114:0/2615134841 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f38bc0762d0 msgr2=0x7f38bc0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:45.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:45.018+0000 7f38e4a00640 1 --2- 172.21.15.114:0/2615134841 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f38bc0762d0 0x7f38bc0787e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f38e014d2e0 tx=0x7f38d000b040 comp rx=0 tx=0).stop 2024-05-27T09:32:45.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:45.018+0000 7f38e4a00640 1 -- 172.21.15.114:0/2615134841 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38e00cc890 msgr2=0x7f38e00b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:45.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:45.018+0000 7f38e4a00640 1 --2- 172.21.15.114:0/2615134841 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38e00cc890 0x7f38e00b3c30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f38c802a170 tx=0x7f38c80046c0 comp rx=0 tx=0).stop 2024-05-27T09:32:45.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:45.018+0000 7f38e4a00640 1 -- 172.21.15.114:0/2615134841 shutdown_connections 2024-05-27T09:32:45.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:45.018+0000 7f38e4a00640 1 --2- 172.21.15.114:0/2615134841 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f38bc0762d0 0x7f38bc0787e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:45.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:45.018+0000 7f38e4a00640 1 --2- 172.21.15.114:0/2615134841 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f38e0160870 0x7f38e01f24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:45.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:45.018+0000 7f38e4a00640 1 --2- 172.21.15.114:0/2615134841 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38e0156980 0x7f38e014c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:45.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:45.018+0000 7f38e4a00640 1 --2- 172.21.15.114:0/2615134841 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38e00cc890 0x7f38e00b3c30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:45.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:45.018+0000 7f38e4a00640 1 -- 172.21.15.114:0/2615134841 >> 172.21.15.114:0/2615134841 conn(0x7f38e0071460 msgr2=0x7f38e0072d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:45.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:45.019+0000 7f38e4a00640 1 -- 172.21.15.114:0/2615134841 shutdown_connections 2024-05-27T09:32:45.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:45.019+0000 7f38e4a00640 1 -- 172.21.15.114:0/2615134841 wait complete. 2024-05-27T09:32:45.045 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 27, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-05-27T09:27:18.079214+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 27, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4300": { "addr": "172.21.15.114:6837/1448950017", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6836", "nonce": 1448950017, "type": "v2" }, { "addr": "172.21.15.114:6837", "nonce": 1448950017, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4300, "incarnation": 4, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 4 }, "gid_4828": { "addr": "172.21.15.163:6839/3371890103", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6837", "nonce": 3371890103, "type": "v2" }, { "addr": "172.21.15.163:6839", "nonce": 3371890103, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4828, "incarnation": 19, "join_fscid": -1, "name": "f", "rank": 2, "state": "up:active", "state_seq": 20 }, "gid_4875": { "addr": "172.21.15.114:6835/2747052034", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6833", "nonce": 2747052034, "type": "v2" }, { "addr": "172.21.15.114:6835", "nonce": 2747052034, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4875, "incarnation": 25, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:reconnect", "state_seq": 21 } }, "last_failure": 0, "last_failure_osd_epoch": 36, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-05-27T09:32:41.388124+0000", "qdb_cluster": [ 4300, 4828 ], "qdb_leader": 4300, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4300, "mds_1": 4875, "mds_2": 4828 } } } ], "standbys": [ { "addr": "172.21.15.163:6835/3513522318", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 3513522318, "type": "v2" }, { "addr": "172.21.15.163:6835", "nonce": 3513522318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6838/1753596112", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6836", "nonce": 1753596112, "type": "v2" }, { "addr": "172.21.15.163:6838", "nonce": 1753596112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-05-27T09:32:47.046 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:32:47.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.328+0000 7f0ce3200640 1 Processor -- start 2024-05-27T09:32:47.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.329+0000 7f0ce3200640 1 -- start start 2024-05-27T09:32:47.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.331+0000 7f0ce3200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0ce41547d0 0x7f0ce40c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:47.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.331+0000 7f0ce3200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ce406fe50 con 0x7f0ce4074840 2024-05-27T09:32:47.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.331+0000 7f0ce3200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ce40700f0 con 0x7f0ce40741f0 2024-05-27T09:32:47.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.331+0000 7f0ce3200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f0ce40703d0 con 0x7f0ce41547d0 2024-05-27T09:32:47.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.331+0000 7f0ce1400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0ce41547d0 0x7f0ce40c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:47.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.331+0000 7f0ce2800640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f0ce4074840 0x7f0ce4164b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:45940/0 (socket says 172.21.15.114:45940) 2024-05-27T09:32:47.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.331+0000 7f0ce2800640 1 -- 172.21.15.114:0/871570690 learned_addr learned my addr 172.21.15.114:0/871570690 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:32:47.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.332+0000 7f0ce0a00640 1 -- 172.21.15.114:0/871570690 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1801414662 0 0) 0x7f0ce40700f0 con 0x7f0ce40741f0 2024-05-27T09:32:47.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.332+0000 7f0ce0a00640 1 -- 172.21.15.114:0/871570690 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cc8003520 con 0x7f0ce40741f0 2024-05-27T09:32:47.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.332+0000 7f0ce1400640 1 -- 172.21.15.114:0/871570690 >> v1:172.21.15.163:6789/0 conn(0x7f0ce40741f0 legacy=0x7f0ce406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:47.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.332+0000 7f0ce1400640 1 -- 172.21.15.114:0/871570690 >> v1:172.21.15.114:6789/0 conn(0x7f0ce4074840 legacy=0x7f0ce4164b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:47.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.332+0000 7f0ce1400640 1 -- 172.21.15.114:0/871570690 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce40731d0 con 0x7f0ce41547d0 2024-05-27T09:32:47.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.333+0000 7f0ce1400640 1 --2- 172.21.15.114:0/871570690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ce41547d0 0x7f0ce40c63a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009d90 tx=0x7f0cd402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433e821c6ca73baa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:47.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.333+0000 7f0ce0a00640 1 -- 172.21.15.114:0/871570690 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4037070 con 0x7f0ce41547d0 2024-05-27T09:32:47.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.333+0000 7f0ce0a00640 1 -- 172.21.15.114:0/871570690 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cd4007740 con 0x7f0ce41547d0 2024-05-27T09:32:47.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.333+0000 7f0ce0a00640 1 -- 172.21.15.114:0/871570690 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4007af0 con 0x7f0ce41547d0 2024-05-27T09:32:47.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.334+0000 7f0ce3200640 1 -- 172.21.15.114:0/871570690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ce41547d0 msgr2=0x7f0ce40c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:47.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.334+0000 7f0ce3200640 1 --2- 172.21.15.114:0/871570690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ce41547d0 0x7f0ce40c63a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009d90 tx=0x7f0cd402a780 comp rx=0 tx=0).stop 2024-05-27T09:32:47.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.334+0000 7f0ce3200640 1 -- 172.21.15.114:0/871570690 shutdown_connections 2024-05-27T09:32:47.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.334+0000 7f0ce3200640 1 --2- 172.21.15.114:0/871570690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ce41547d0 0x7f0ce40c63a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:47.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.334+0000 7f0ce3200640 1 -- 172.21.15.114:0/871570690 >> 172.21.15.114:0/871570690 conn(0x7f0ce40ca320 msgr2=0x7f0ce40ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:47.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.334+0000 7f0ce3200640 1 -- 172.21.15.114:0/871570690 shutdown_connections 2024-05-27T09:32:47.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.334+0000 7f0ce3200640 1 -- 172.21.15.114:0/871570690 wait complete. 2024-05-27T09:32:47.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.337+0000 7f0ce3200640 1 Processor -- start 2024-05-27T09:32:47.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.337+0000 7f0ce3200640 1 -- start start 2024-05-27T09:32:47.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.338+0000 7f0ce3200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ce40741f0 0x7f0ce40c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:47.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.338+0000 7f0ce3200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ce4074840 0x7f0ce41f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:47.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.338+0000 7f0ce3200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ce41547d0 0x7f0ce4147390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:47.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.338+0000 7f0ce3200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce406f500 con 0x7f0ce4074840 2024-05-27T09:32:47.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.338+0000 7f0ce3200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce4073790 con 0x7f0ce41547d0 2024-05-27T09:32:47.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.338+0000 7f0ce3200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f0ce406f680 con 0x7f0ce40741f0 2024-05-27T09:32:47.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.339+0000 7f0ce1400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ce4074840 0x7f0ce41f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:47.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.339+0000 7f0ce1400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ce4074840 0x7f0ce41f2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59996/0 (socket says 172.21.15.114:59996) 2024-05-27T09:32:47.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.339+0000 7f0ce1400640 1 -- 172.21.15.114:0/3793275301 learned_addr learned my addr 172.21.15.114:0/3793275301 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:47.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.339+0000 7f0ce1e00640 1 --2- 172.21.15.114:0/3793275301 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ce40741f0 0x7f0ce40c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:47.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.339+0000 7f0ce2800640 1 --2- 172.21.15.114:0/3793275301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ce41547d0 0x7f0ce4147390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:47.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.339+0000 7f0ce1400640 1 -- 172.21.15.114:0/3793275301 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ce40741f0 msgr2=0x7f0ce40c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:47.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.339+0000 7f0ce1400640 1 --2- 172.21.15.114:0/3793275301 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ce40741f0 0x7f0ce40c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:47.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.339+0000 7f0ce1400640 1 -- 172.21.15.114:0/3793275301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ce41547d0 msgr2=0x7f0ce4147390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:47.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.339+0000 7f0ce1400640 1 --2- 172.21.15.114:0/3793275301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ce41547d0 0x7f0ce4147390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:47.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.339+0000 7f0ce1400640 1 -- 172.21.15.114:0/3793275301 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce41f31a0 con 0x7f0ce4074840 2024-05-27T09:32:47.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.340+0000 7f0ce1400640 1 --2- 172.21.15.114:0/3793275301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ce4074840 0x7f0ce41f2c60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4035040 tx=0x7f0cd4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:47.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.340+0000 7f0cda000640 1 -- 172.21.15.114:0/3793275301 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4042070 con 0x7f0ce4074840 2024-05-27T09:32:47.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.340+0000 7f0cda000640 1 -- 172.21.15.114:0/3793275301 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cd4003990 con 0x7f0ce4074840 2024-05-27T09:32:47.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.340+0000 7f0cda000640 1 -- 172.21.15.114:0/3793275301 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4037040 con 0x7f0ce4074840 2024-05-27T09:32:47.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.340+0000 7f0ce3200640 1 -- 172.21.15.114:0/3793275301 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce4156fd0 con 0x7f0ce4074840 2024-05-27T09:32:47.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.340+0000 7f0ce3200640 1 -- 172.21.15.114:0/3793275301 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce4157590 con 0x7f0ce4074840 2024-05-27T09:32:47.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.341+0000 7f0ce3200640 1 -- 172.21.15.114:0/3793275301 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ce40c85a0 con 0x7f0ce4074840 2024-05-27T09:32:47.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.342+0000 7f0cda000640 1 -- 172.21.15.114:0/3793275301 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0cd40327f0 con 0x7f0ce4074840 2024-05-27T09:32:47.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.343+0000 7f0cda000640 1 --2- 172.21.15.114:0/3793275301 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0cc0076300 0x7f0cc0078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:47.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.343+0000 7f0cda000640 1 -- 172.21.15.114:0/3793275301 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6040+0+0 (secure 0 0 0) 0x7f0cd40b8850 con 0x7f0ce4074840 2024-05-27T09:32:47.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.344+0000 7f0ce1e00640 1 --2- 172.21.15.114:0/3793275301 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0cc0076300 0x7f0cc0078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:47.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.344+0000 7f0ce1e00640 1 --2- 172.21.15.114:0/3793275301 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0cc0076300 0x7f0cc0078810 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc007fd0 tx=0x7f0ccc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:47.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.348+0000 7f0cda000640 1 -- 172.21.15.114:0/3793275301 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0cd4081a00 con 0x7f0ce4074840 2024-05-27T09:32:47.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.593+0000 7f0ce3200640 1 -- 172.21.15.114:0/3793275301 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ce4074620 con 0x7f0ce4074840 2024-05-27T09:32:47.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.595+0000 7f0cda000640 1 -- 172.21.15.114:0/3793275301 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+6055 (secure 0 0 0) 0x7f0cd4085d00 con 0x7f0ce4074840 2024-05-27T09:32:47.596 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:32:47.596 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:32:46.926607+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:rejoin","state_seq":23,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4828,4300]},"id":1}]} 2024-05-27T09:32:47.596 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2024-05-27T09:32:47.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.597+0000 7f0ce3200640 1 -- 172.21.15.114:0/3793275301 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0cc0076300 msgr2=0x7f0cc0078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:47.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.597+0000 7f0ce3200640 1 --2- 172.21.15.114:0/3793275301 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0cc0076300 0x7f0cc0078810 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc007fd0 tx=0x7f0ccc009040 comp rx=0 tx=0).stop 2024-05-27T09:32:47.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.597+0000 7f0ce3200640 1 -- 172.21.15.114:0/3793275301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ce4074840 msgr2=0x7f0ce41f2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:47.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.597+0000 7f0ce3200640 1 --2- 172.21.15.114:0/3793275301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ce4074840 0x7f0ce41f2c60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4035040 tx=0x7f0cd4004a40 comp rx=0 tx=0).stop 2024-05-27T09:32:47.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.597+0000 7f0ce3200640 1 -- 172.21.15.114:0/3793275301 shutdown_connections 2024-05-27T09:32:47.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.597+0000 7f0ce3200640 1 --2- 172.21.15.114:0/3793275301 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0cc0076300 0x7f0cc0078810 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:47.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.597+0000 7f0ce3200640 1 --2- 172.21.15.114:0/3793275301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ce41547d0 0x7f0ce4147390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:47.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.597+0000 7f0ce3200640 1 --2- 172.21.15.114:0/3793275301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0ce4074840 0x7f0ce41f2c60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:47.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.597+0000 7f0ce3200640 1 --2- 172.21.15.114:0/3793275301 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ce40741f0 0x7f0ce40c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:47.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.597+0000 7f0ce3200640 1 -- 172.21.15.114:0/3793275301 >> 172.21.15.114:0/3793275301 conn(0x7f0ce40ca320 msgr2=0x7f0ce4158210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:47.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.598+0000 7f0ce3200640 1 -- 172.21.15.114:0/3793275301 shutdown_connections 2024-05-27T09:32:47.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:47.598+0000 7f0ce3200640 1 -- 172.21.15.114:0/3793275301 wait complete. 2024-05-27T09:32:47.623 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:32:48.927 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:32:48.935 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:32:49.624 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:32:49.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.905+0000 7f9355200640 1 Processor -- start 2024-05-27T09:32:49.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.905+0000 7f9355200640 1 -- start start 2024-05-27T09:32:49.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.906+0000 7f9355200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9350074a60 0x7f9350154820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:49.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.907+0000 7f9355200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9350073260 0x7f9350073670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.907+0000 7f9355200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f935015a810 con 0x7f93500ca2d0 2024-05-27T09:32:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.907+0000 7f9355200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f935015db60 con 0x7f9350073260 2024-05-27T09:32:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.907+0000 7f9355200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f935015aaf0 con 0x7f9350074a60 2024-05-27T09:32:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.907+0000 7f934d400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9350073260 0x7f9350073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.907+0000 7f934d400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9350073260 0x7f9350073670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60000/0 (socket says 172.21.15.114:60000) 2024-05-27T09:32:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.907+0000 7f934d400640 1 -- 172.21.15.114:0/752600362 learned_addr learned my addr 172.21.15.114:0/752600362 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.907+0000 7f934de00640 1 --2- 172.21.15.114:0/752600362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9350074a60 0x7f9350154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.907+0000 7f934d400640 1 -- 172.21.15.114:0/752600362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9350074a60 msgr2=0x7f9350154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:32:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.907+0000 7f934d400640 1 --2- 172.21.15.114:0/752600362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9350074a60 0x7f9350154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.907+0000 7f934d400640 1 -- 172.21.15.114:0/752600362 >> v1:172.21.15.163:6789/0 conn(0x7f93500ca2d0 legacy=0x7f93500ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:49.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.907+0000 7f934d400640 1 -- 172.21.15.114:0/752600362 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935015add0 con 0x7f9350073260 2024-05-27T09:32:49.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.909+0000 7f934d400640 1 --2- 172.21.15.114:0/752600362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9350073260 0x7f9350073670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9338009870 tx=0x7f933802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46564a927d4edcb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:49.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.909+0000 7f934ca00640 1 -- 172.21.15.114:0/752600362 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338037070 con 0x7f9350073260 2024-05-27T09:32:49.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.909+0000 7f934ca00640 1 -- 172.21.15.114:0/752600362 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f933802ae50 con 0x7f9350073260 2024-05-27T09:32:49.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.909+0000 7f934ca00640 1 -- 172.21.15.114:0/752600362 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338032860 con 0x7f9350073260 2024-05-27T09:32:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.910+0000 7f9355200640 1 -- 172.21.15.114:0/752600362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9350073260 msgr2=0x7f9350073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.910+0000 7f9355200640 1 --2- 172.21.15.114:0/752600362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9350073260 0x7f9350073670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9338009870 tx=0x7f933802a300 comp rx=0 tx=0).stop 2024-05-27T09:32:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.910+0000 7f9355200640 1 -- 172.21.15.114:0/752600362 shutdown_connections 2024-05-27T09:32:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.910+0000 7f9355200640 1 --2- 172.21.15.114:0/752600362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9350073260 0x7f9350073670 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.910+0000 7f9355200640 1 --2- 172.21.15.114:0/752600362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9350074a60 0x7f9350154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.910+0000 7f9355200640 1 -- 172.21.15.114:0/752600362 >> 172.21.15.114:0/752600362 conn(0x7f935006f6e0 msgr2=0x7f9350070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.910+0000 7f9355200640 1 -- 172.21.15.114:0/752600362 shutdown_connections 2024-05-27T09:32:49.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.910+0000 7f9355200640 1 -- 172.21.15.114:0/752600362 wait complete. 2024-05-27T09:32:49.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.912+0000 7f9355200640 1 Processor -- start 2024-05-27T09:32:49.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.912+0000 7f9355200640 1 -- start start 2024-05-27T09:32:49.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.914+0000 7f9355200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9350073260 0x7f93501f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:49.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.914+0000 7f9355200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9350074a60 0x7f93500c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:49.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.914+0000 7f9355200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93500ca2d0 0x7f93501e6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:49.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.914+0000 7f9355200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f935015a4e0 con 0x7f9350074a60 2024-05-27T09:32:49.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.914+0000 7f9355200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f935015b390 con 0x7f9350073260 2024-05-27T09:32:49.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.914+0000 7f9355200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f935015dff0 con 0x7f93500ca2d0 2024-05-27T09:32:49.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.914+0000 7f934d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9350074a60 0x7f93500c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:49.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.914+0000 7f934d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9350074a60 0x7f93500c6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60002/0 (socket says 172.21.15.114:60002) 2024-05-27T09:32:49.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.914+0000 7f934d400640 1 -- 172.21.15.114:0/1744109804 learned_addr learned my addr 172.21.15.114:0/1744109804 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:49.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.914+0000 7f934de00640 1 --2- 172.21.15.114:0/1744109804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9350073260 0x7f93501f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:49.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.914+0000 7f934e800640 1 --2- 172.21.15.114:0/1744109804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93500ca2d0 0x7f93501e6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:49.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.914+0000 7f934d400640 1 -- 172.21.15.114:0/1744109804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93500ca2d0 msgr2=0x7f93501e6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:49.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.914+0000 7f934d400640 1 --2- 172.21.15.114:0/1744109804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93500ca2d0 0x7f93501e6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:49.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.914+0000 7f934d400640 1 -- 172.21.15.114:0/1744109804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9350073260 msgr2=0x7f93501f30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:49.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.914+0000 7f934d400640 1 --2- 172.21.15.114:0/1744109804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9350073260 0x7f93501f30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:49.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.914+0000 7f934d400640 1 -- 172.21.15.114:0/1744109804 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93501f3610 con 0x7f9350074a60 2024-05-27T09:32:49.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.915+0000 7f934d400640 1 --2- 172.21.15.114:0/1744109804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9350074a60 0x7f93500c6e60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9338000c00 tx=0x7f93380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:49.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.915+0000 7f9346000640 1 -- 172.21.15.114:0/1744109804 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338041070 con 0x7f9350074a60 2024-05-27T09:32:49.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.915+0000 7f9346000640 1 -- 172.21.15.114:0/1744109804 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f933803c660 con 0x7f9350074a60 2024-05-27T09:32:49.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.915+0000 7f9346000640 1 -- 172.21.15.114:0/1744109804 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338037040 con 0x7f9350074a60 2024-05-27T09:32:49.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.915+0000 7f9355200640 1 -- 172.21.15.114:0/1744109804 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93500c4360 con 0x7f9350074a60 2024-05-27T09:32:49.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.915+0000 7f9355200640 1 -- 172.21.15.114:0/1744109804 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93500c4620 con 0x7f9350074a60 2024-05-27T09:32:49.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.917+0000 7f9355200640 1 -- 172.21.15.114:0/1744109804 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f935006a380 con 0x7f9350074a60 2024-05-27T09:32:49.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.918+0000 7f9346000640 1 -- 172.21.15.114:0/1744109804 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9338044070 con 0x7f9350074a60 2024-05-27T09:32:49.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.919+0000 7f9346000640 1 --2- 172.21.15.114:0/1744109804 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f932c0762d0 0x7f932c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:49.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.919+0000 7f9346000640 1 -- 172.21.15.114:0/1744109804 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6040+0+0 (secure 0 0 0) 0x7f93380b7c40 con 0x7f9350074a60 2024-05-27T09:32:49.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.919+0000 7f934de00640 1 --2- 172.21.15.114:0/1744109804 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f932c0762d0 0x7f932c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:49.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.920+0000 7f934de00640 1 --2- 172.21.15.114:0/1744109804 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f932c0762d0 0x7f932c0787e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9340009aa0 tx=0x7f9340009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:49.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:49.924+0000 7f9346000640 1 -- 172.21.15.114:0/1744109804 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9338080e90 con 0x7f9350074a60 2024-05-27T09:32:50.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:50.171+0000 7f9355200640 1 -- 172.21.15.114:0/1744109804 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9350073ca0 con 0x7f9350074a60 2024-05-27T09:32:50.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:50.173+0000 7f9346000640 1 -- 172.21.15.114:0/1744109804 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+6055 (secure 0 0 0) 0x7f9338085190 con 0x7f9350074a60 2024-05-27T09:32:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:32:50.174 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:32:46.926607+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:rejoin","state_seq":23,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4828,4300]},"id":1}]} 2024-05-27T09:32:50.174 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2024-05-27T09:32:50.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:50.175+0000 7f9355200640 1 -- 172.21.15.114:0/1744109804 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f932c0762d0 msgr2=0x7f932c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:50.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:50.175+0000 7f9355200640 1 --2- 172.21.15.114:0/1744109804 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f932c0762d0 0x7f932c0787e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9340009aa0 tx=0x7f9340009190 comp rx=0 tx=0).stop 2024-05-27T09:32:50.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:50.175+0000 7f9355200640 1 -- 172.21.15.114:0/1744109804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9350074a60 msgr2=0x7f93500c6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:50.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:50.175+0000 7f9355200640 1 --2- 172.21.15.114:0/1744109804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9350074a60 0x7f93500c6e60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9338000c00 tx=0x7f93380046c0 comp rx=0 tx=0).stop 2024-05-27T09:32:50.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:50.176+0000 7f9355200640 1 -- 172.21.15.114:0/1744109804 shutdown_connections 2024-05-27T09:32:50.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:50.176+0000 7f9355200640 1 --2- 172.21.15.114:0/1744109804 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f932c0762d0 0x7f932c0787e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:50.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:50.176+0000 7f9355200640 1 --2- 172.21.15.114:0/1744109804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93500ca2d0 0x7f93501e6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:50.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:50.176+0000 7f9355200640 1 --2- 172.21.15.114:0/1744109804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9350074a60 0x7f93500c6e60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:50.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:50.176+0000 7f9355200640 1 --2- 172.21.15.114:0/1744109804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9350073260 0x7f93501f30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:50.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:50.176+0000 7f9355200640 1 -- 172.21.15.114:0/1744109804 >> 172.21.15.114:0/1744109804 conn(0x7f935006f6e0 msgr2=0x7f93500b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:50.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:50.176+0000 7f9355200640 1 -- 172.21.15.114:0/1744109804 shutdown_connections 2024-05-27T09:32:50.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:50.176+0000 7f9355200640 1 -- 172.21.15.114:0/1744109804 wait complete. 2024-05-27T09:32:50.204 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:32:51.992 INFO:tasks.workunit.client.0.smithi114.stderr:+ set -e 2024-05-27T09:32:51.993 INFO:tasks.workunit.client.0.smithi114.stderr:++ seq 1 20 2024-05-27T09:32:51.995 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 20` 2024-05-27T09:32:51.995 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a 2024-05-27T09:32:52.001 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/one 2024-05-27T09:32:52.002 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat 2024-05-27T09:32:52.204 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:32:52.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.485+0000 7fccc7e00640 1 Processor -- start 2024-05-27T09:32:52.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.485+0000 7fccc7e00640 1 -- start start 2024-05-27T09:32:52.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.488+0000 7fccc7e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fccc8074a80 0x7fccc815cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:52.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.488+0000 7fccc7e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccc8156880 0x7fccc806edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:52.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.488+0000 7fccc7e00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fccc8070060 0x7fccc806f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:52.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.488+0000 7fccc7e00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fccc80c8220 con 0x7fccc8074a80 2024-05-27T09:32:52.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.488+0000 7fccc7e00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fccc80c8750 con 0x7fccc8070060 2024-05-27T09:32:52.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.488+0000 7fccc7e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fccc815c8e0 con 0x7fccc8156880 2024-05-27T09:32:52.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.488+0000 7fccc6a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fccc8074a80 0x7fccc815cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:52.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.488+0000 7fccbfe00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccc8156880 0x7fccc806edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:52.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.488+0000 7fccbfe00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccc8156880 0x7fccc806edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:58830/0 (socket says 172.21.15.114:58830) 2024-05-27T09:32:52.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.488+0000 7fccbfe00640 1 -- 172.21.15.114:0/1781602043 learned_addr learned my addr 172.21.15.114:0/1781602043 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:52.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.488+0000 7fccc7400640 1 --2- 172.21.15.114:0/1781602043 >> v2:172.21.15.163:3300/0 conn(0x7fccc8070060 0x7fccc806f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:52.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.489+0000 7fccbfe00640 1 -- 172.21.15.114:0/1781602043 >> v2:172.21.15.163:3300/0 conn(0x7fccc8070060 msgr2=0x7fccc806f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:32:52.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.489+0000 7fccbfe00640 1 --2- 172.21.15.114:0/1781602043 >> v2:172.21.15.163:3300/0 conn(0x7fccc8070060 0x7fccc806f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.489+0000 7fccbfe00640 1 -- 172.21.15.114:0/1781602043 >> v2:172.21.15.114:3300/0 conn(0x7fccc8074a80 msgr2=0x7fccc815cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:32:52.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.489+0000 7fccbfe00640 1 --2- 172.21.15.114:0/1781602043 >> v2:172.21.15.114:3300/0 conn(0x7fccc8074a80 0x7fccc815cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.489+0000 7fccbfe00640 1 -- 172.21.15.114:0/1781602043 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc80c8470 con 0x7fccc8156880 2024-05-27T09:32:52.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.490+0000 7fccbfe00640 1 --2- 172.21.15.114:0/1781602043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccc8156880 0x7fccc806edf0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fccc8072fc0 tx=0x7fccb002a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fe2a52d320b709b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:52.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.490+0000 7fccc6000640 1 -- 172.21.15.114:0/1781602043 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0037070 con 0x7fccc8156880 2024-05-27T09:32:52.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.490+0000 7fccc6000640 1 -- 172.21.15.114:0/1781602043 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccb0032440 con 0x7fccc8156880 2024-05-27T09:32:52.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.490+0000 7fccc6000640 1 -- 172.21.15.114:0/1781602043 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0032810 con 0x7fccc8156880 2024-05-27T09:32:52.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.491+0000 7fccc7e00640 1 -- 172.21.15.114:0/1781602043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccc8156880 msgr2=0x7fccc806edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:52.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.491+0000 7fccc7e00640 1 --2- 172.21.15.114:0/1781602043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccc8156880 0x7fccc806edf0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fccc8072fc0 tx=0x7fccb002a1e0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.491+0000 7fccc7e00640 1 -- 172.21.15.114:0/1781602043 shutdown_connections 2024-05-27T09:32:52.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.491+0000 7fccc7e00640 1 --2- 172.21.15.114:0/1781602043 >> v2:172.21.15.163:3300/0 conn(0x7fccc8070060 0x7fccc806f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.491+0000 7fccc7e00640 1 --2- 172.21.15.114:0/1781602043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccc8156880 0x7fccc806edf0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.491+0000 7fccc7e00640 1 --2- 172.21.15.114:0/1781602043 >> v2:172.21.15.114:3300/0 conn(0x7fccc8074a80 0x7fccc815cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.491+0000 7fccc7e00640 1 -- 172.21.15.114:0/1781602043 >> 172.21.15.114:0/1781602043 conn(0x7fccc80ca320 msgr2=0x7fccc80ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:52.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.491+0000 7fccc7e00640 1 -- 172.21.15.114:0/1781602043 shutdown_connections 2024-05-27T09:32:52.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.491+0000 7fccc7e00640 1 -- 172.21.15.114:0/1781602043 wait complete. 2024-05-27T09:32:52.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.494+0000 7fccc7e00640 1 Processor -- start 2024-05-27T09:32:52.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.494+0000 7fccc7e00640 1 -- start start 2024-05-27T09:32:52.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.496+0000 7fccc7e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccc8070060 0x7fccc81f3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:52.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.496+0000 7fccc7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccc8074a80 0x7fccc815ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:52.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.496+0000 7fccc7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccc8156880 0x7fccc81ec920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:52.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.496+0000 7fccc7e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fccc815fa50 con 0x7fccc8074a80 2024-05-27T09:32:52.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.496+0000 7fccc7e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fccc815c570 con 0x7fccc8156880 2024-05-27T09:32:52.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.496+0000 7fccc7e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fccc815d620 con 0x7fccc8070060 2024-05-27T09:32:52.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.496+0000 7fccc6a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccc8070060 0x7fccc81f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:52.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.496+0000 7fccbfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccc8074a80 0x7fccc815ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:52.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.496+0000 7fccc7400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccc8156880 0x7fccc81ec920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:52.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.496+0000 7fccc6a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccc8070060 0x7fccc81f3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:58832/0 (socket says 172.21.15.114:58832) 2024-05-27T09:32:52.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.496+0000 7fccc6a00640 1 -- 172.21.15.114:0/2985570528 learned_addr learned my addr 172.21.15.114:0/2985570528 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:52.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.497+0000 7fccc7400640 1 -- 172.21.15.114:0/2985570528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccc8070060 msgr2=0x7fccc81f3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:52.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.497+0000 7fccc7400640 1 --2- 172.21.15.114:0/2985570528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccc8070060 0x7fccc81f3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.497+0000 7fccc7400640 1 -- 172.21.15.114:0/2985570528 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccc8074a80 msgr2=0x7fccc815ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:52.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.497+0000 7fccc7400640 1 --2- 172.21.15.114:0/2985570528 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccc8074a80 0x7fccc815ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.497+0000 7fccc7400640 1 -- 172.21.15.114:0/2985570528 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc815c300 con 0x7fccc8156880 2024-05-27T09:32:52.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.497+0000 7fccc7400640 1 --2- 172.21.15.114:0/2985570528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccc8156880 0x7fccc81ec920 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fccc8160b70 tx=0x7fccc000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:52.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.497+0000 7fccbea00640 1 -- 172.21.15.114:0/2985570528 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0018020 con 0x7fccc8156880 2024-05-27T09:32:52.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.497+0000 7fccbea00640 1 -- 172.21.15.114:0/2985570528 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccc000bb80 con 0x7fccc8156880 2024-05-27T09:32:52.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.497+0000 7fccbea00640 1 -- 172.21.15.114:0/2985570528 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0014400 con 0x7fccc8156880 2024-05-27T09:32:52.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.498+0000 7fccc7e00640 1 -- 172.21.15.114:0/2985570528 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc81f3b60 con 0x7fccc8156880 2024-05-27T09:32:52.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.498+0000 7fccc7e00640 1 -- 172.21.15.114:0/2985570528 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc81526e0 con 0x7fccc8156880 2024-05-27T09:32:52.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.499+0000 7fccc7e00640 1 -- 172.21.15.114:0/2985570528 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccc8152160 con 0x7fccc8156880 2024-05-27T09:32:52.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.500+0000 7fccbea00640 1 -- 172.21.15.114:0/2985570528 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fccc0005d10 con 0x7fccc8156880 2024-05-27T09:32:52.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.501+0000 7fccbea00640 1 --2- 172.21.15.114:0/2985570528 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcca4076300 0x7fcca4078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:52.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.501+0000 7fccbea00640 1 -- 172.21.15.114:0/2985570528 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6040+0+0 (secure 0 0 0) 0x7fccc0098570 con 0x7fccc8156880 2024-05-27T09:32:52.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.501+0000 7fccc6a00640 1 --2- 172.21.15.114:0/2985570528 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcca4076300 0x7fcca4078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:52.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.502+0000 7fccc6a00640 1 --2- 172.21.15.114:0/2985570528 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcca4076300 0x7fcca4078810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009b40 tx=0x7fccb8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:52.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.505+0000 7fccbea00640 1 -- 172.21.15.114:0/2985570528 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fccc8152160 con 0x7fccc8156880 2024-05-27T09:32:52.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.755+0000 7fccc7e00640 1 -- 172.21.15.114:0/2985570528 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccc8050860 con 0x7fccc8156880 2024-05-27T09:32:52.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.756+0000 7fccbea00640 1 -- 172.21.15.114:0/2985570528 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+6060 (secure 0 0 0) 0x7fccc0022030 con 0x7fccc8156880 2024-05-27T09:32:52.758 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:32:52.758 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:32:51.967298+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4828,4875]},"id":1}]} 2024-05-27T09:32:52.758 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 29 2024-05-27T09:32:52.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.759+0000 7fccc7e00640 1 -- 172.21.15.114:0/2985570528 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcca4076300 msgr2=0x7fcca4078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:52.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.759+0000 7fccc7e00640 1 --2- 172.21.15.114:0/2985570528 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcca4076300 0x7fcca4078810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009b40 tx=0x7fccb8009190 comp rx=0 tx=0).stop 2024-05-27T09:32:52.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.759+0000 7fccc7e00640 1 -- 172.21.15.114:0/2985570528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccc8156880 msgr2=0x7fccc81ec920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:52.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.759+0000 7fccc7e00640 1 --2- 172.21.15.114:0/2985570528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccc8156880 0x7fccc81ec920 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fccc8160b70 tx=0x7fccc000ef10 comp rx=0 tx=0).stop 2024-05-27T09:32:52.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.759+0000 7fccc7e00640 1 -- 172.21.15.114:0/2985570528 shutdown_connections 2024-05-27T09:32:52.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.759+0000 7fccc7e00640 1 --2- 172.21.15.114:0/2985570528 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcca4076300 0x7fcca4078810 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.759+0000 7fccc7e00640 1 --2- 172.21.15.114:0/2985570528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccc8156880 0x7fccc81ec920 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.759+0000 7fccc7e00640 1 --2- 172.21.15.114:0/2985570528 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fccc8074a80 0x7fccc815ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.760+0000 7fccc7e00640 1 --2- 172.21.15.114:0/2985570528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccc8070060 0x7fccc81f3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.760+0000 7fccc7e00640 1 -- 172.21.15.114:0/2985570528 >> 172.21.15.114:0/2985570528 conn(0x7fccc80ca320 msgr2=0x7fccc8073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:52.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.760+0000 7fccc7e00640 1 -- 172.21.15.114:0/2985570528 shutdown_connections 2024-05-27T09:32:52.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:52.760+0000 7fccc7e00640 1 -- 172.21.15.114:0/2985570528 wait complete. 2024-05-27T09:32:52.787 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=1, replacing gid=4297 2024-05-27T09:32:52.787 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 2 secs before reviving mds.c 2024-05-27T09:32:53.059 INFO:tasks.ceph.mds.e.smithi114.stderr:2024-05-27T09:32:53.058+0000 7f4ffb600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-27T09:32:54.788 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2024-05-27T09:32:54.788 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-27T09:32:54.789 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-27T09:32:54.797 INFO:tasks.ceph.mds.c:Started 2024-05-27T09:32:54.797 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:32:55.025 INFO:tasks.ceph.mds.c.smithi114.stdout:starting mds.c at 2024-05-27T09:32:55.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.052+0000 7fb4fe200640 1 Processor -- start 2024-05-27T09:32:55.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.052+0000 7fb4fe200640 1 -- start start 2024-05-27T09:32:55.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.054+0000 7fb4fe200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f81593b0 0x7fb4f8142720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:55.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.054+0000 7fb4fe200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb4f8073260 0x7fb4f815a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:55.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.054+0000 7fb4fe200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4f80739a0 con 0x7fb4f815ade0 2024-05-27T09:32:55.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.054+0000 7fb4fe200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fb4f8074d20 con 0x7fb4f8073260 2024-05-27T09:32:55.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.054+0000 7fb4fe200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fb4f8073c80 con 0x7fb4f81593b0 2024-05-27T09:32:55.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.054+0000 7fb4f7400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f81593b0 0x7fb4f8142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:55.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.054+0000 7fb4eea00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb4f8073260 0x7fb4f815a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:55.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.055+0000 7fb4f7e00640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb4f815ade0 0x7fb4f80c2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:39588/0 (socket says 172.21.15.114:39588) 2024-05-27T09:32:55.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.055+0000 7fb4f7e00640 1 -- 172.21.15.114:0/1241060523 learned_addr learned my addr 172.21.15.114:0/1241060523 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:32:55.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.055+0000 7fb4f7400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f81593b0 0x7fb4f8142720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:47100/0 (socket says 172.21.15.114:47100) 2024-05-27T09:32:55.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.055+0000 7fb4f7400640 1 -- 172.21.15.114:0/1241060523 >> v2:172.21.15.163:3300/0 conn(0x7fb4f8073260 msgr2=0x7fb4f815a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:32:55.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.055+0000 7fb4f7400640 1 --2- 172.21.15.114:0/1241060523 >> v2:172.21.15.163:3300/0 conn(0x7fb4f8073260 0x7fb4f815a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:55.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.055+0000 7fb4f7400640 1 -- 172.21.15.114:0/1241060523 >> v1:172.21.15.163:6789/0 conn(0x7fb4f815ade0 legacy=0x7fb4f80c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:55.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.055+0000 7fb4f7400640 1 -- 172.21.15.114:0/1241060523 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f8073f60 con 0x7fb4f81593b0 2024-05-27T09:32:55.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.056+0000 7fb4f7400640 1 --2- 172.21.15.114:0/1241060523 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f81593b0 0x7fb4f8142720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80097a0 tx=0x7fb4e802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86d593a8c44e3e0e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:55.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.057+0000 7fb4f6a00640 1 -- 172.21.15.114:0/1241060523 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8037070 con 0x7fb4f81593b0 2024-05-27T09:32:55.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.057+0000 7fb4f6a00640 1 -- 172.21.15.114:0/1241060523 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4e8032440 con 0x7fb4f81593b0 2024-05-27T09:32:55.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.057+0000 7fb4f6a00640 1 -- 172.21.15.114:0/1241060523 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8032810 con 0x7fb4f81593b0 2024-05-27T09:32:55.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.057+0000 7fb4fe200640 1 -- 172.21.15.114:0/1241060523 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f81593b0 msgr2=0x7fb4f8142720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:55.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.057+0000 7fb4fe200640 1 --2- 172.21.15.114:0/1241060523 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f81593b0 0x7fb4f8142720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80097a0 tx=0x7fb4e802a210 comp rx=0 tx=0).stop 2024-05-27T09:32:55.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.057+0000 7fb4fe200640 1 -- 172.21.15.114:0/1241060523 shutdown_connections 2024-05-27T09:32:55.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.057+0000 7fb4fe200640 1 --2- 172.21.15.114:0/1241060523 >> v2:172.21.15.163:3300/0 conn(0x7fb4f8073260 0x7fb4f815a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:55.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.057+0000 7fb4fe200640 1 --2- 172.21.15.114:0/1241060523 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f81593b0 0x7fb4f8142720 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:55.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.057+0000 7fb4fe200640 1 -- 172.21.15.114:0/1241060523 >> 172.21.15.114:0/1241060523 conn(0x7fb4f806f6e0 msgr2=0x7fb4f8070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:55.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.058+0000 7fb4fe200640 1 -- 172.21.15.114:0/1241060523 shutdown_connections 2024-05-27T09:32:55.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.058+0000 7fb4fe200640 1 -- 172.21.15.114:0/1241060523 wait complete. 2024-05-27T09:32:55.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.060+0000 7fb4fe200640 1 Processor -- start 2024-05-27T09:32:55.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.060+0000 7fb4fe200640 1 -- start start 2024-05-27T09:32:55.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.061+0000 7fb4fe200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4f8073260 0x7fb4f8154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:55.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.061+0000 7fb4fe200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f81593b0 0x7fb4f8155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:55.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.061+0000 7fb4fe200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4f815ade0 0x7fb4f815c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:55.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.061+0000 7fb4fe200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f8074800 con 0x7fb4f8073260 2024-05-27T09:32:55.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.061+0000 7fb4fe200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f8074520 con 0x7fb4f815ade0 2024-05-27T09:32:55.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.061+0000 7fb4fe200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fb4f8073670 con 0x7fb4f81593b0 2024-05-27T09:32:55.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.062+0000 7fb4f7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4f8073260 0x7fb4f8154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:55.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.062+0000 7fb4eea00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f81593b0 0x7fb4f8155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:55.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.062+0000 7fb4f7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4f815ade0 0x7fb4f815c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:55.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.062+0000 7fb4eea00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f81593b0 0x7fb4f8155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:47108/0 (socket says 172.21.15.114:47108) 2024-05-27T09:32:55.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.062+0000 7fb4eea00640 1 -- 172.21.15.114:0/2267128968 learned_addr learned my addr 172.21.15.114:0/2267128968 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:55.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.062+0000 7fb4eea00640 1 -- 172.21.15.114:0/2267128968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4f815ade0 msgr2=0x7fb4f815c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:55.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.062+0000 7fb4eea00640 1 --2- 172.21.15.114:0/2267128968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4f815ade0 0x7fb4f815c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:55.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.062+0000 7fb4eea00640 1 -- 172.21.15.114:0/2267128968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4f8073260 msgr2=0x7fb4f8154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:55.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.062+0000 7fb4eea00640 1 --2- 172.21.15.114:0/2267128968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4f8073260 0x7fb4f8154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:55.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.062+0000 7fb4eea00640 1 -- 172.21.15.114:0/2267128968 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f80b1020 con 0x7fb4f81593b0 2024-05-27T09:32:55.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.063+0000 7fb4eea00640 1 --2- 172.21.15.114:0/2267128968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f81593b0 0x7fb4f8155710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000e870 tx=0x7fb4e000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:55.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.063+0000 7fb4efe00640 1 -- 172.21.15.114:0/2267128968 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0017070 con 0x7fb4f81593b0 2024-05-27T09:32:55.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.063+0000 7fb4efe00640 1 -- 172.21.15.114:0/2267128968 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4e000c940 con 0x7fb4f81593b0 2024-05-27T09:32:55.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.063+0000 7fb4efe00640 1 -- 172.21.15.114:0/2267128968 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e000cd30 con 0x7fb4f81593b0 2024-05-27T09:32:55.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.063+0000 7fb4fe200640 1 -- 172.21.15.114:0/2267128968 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f80b13d0 con 0x7fb4f81593b0 2024-05-27T09:32:55.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.063+0000 7fb4fe200640 1 -- 172.21.15.114:0/2267128968 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f80b1670 con 0x7fb4f81593b0 2024-05-27T09:32:55.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.064+0000 7fb4fe200640 1 -- 172.21.15.114:0/2267128968 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4f80692a0 con 0x7fb4f81593b0 2024-05-27T09:32:55.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.065+0000 7fb4efe00640 1 -- 172.21.15.114:0/2267128968 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb4e001a070 con 0x7fb4f81593b0 2024-05-27T09:32:55.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.066+0000 7fb4efe00640 1 --2- 172.21.15.114:0/2267128968 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb4d40762d0 0x7fb4d40787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:55.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.066+0000 7fb4efe00640 1 -- 172.21.15.114:0/2267128968 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(37..37 src has 1..37) ==== 6040+0+0 (secure 0 0 0) 0x7fb4e0097690 con 0x7fb4f81593b0 2024-05-27T09:32:55.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.066+0000 7fb4f7400640 1 --2- 172.21.15.114:0/2267128968 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb4d40762d0 0x7fb4d40787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:55.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.067+0000 7fb4f7400640 1 --2- 172.21.15.114:0/2267128968 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb4d40762d0 0x7fb4d40787e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8034000 tx=0x7fb4e802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:55.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.071+0000 7fb4efe00640 1 -- 172.21.15.114:0/2267128968 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb4f80692a0 con 0x7fb4f81593b0 2024-05-27T09:32:55.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.312+0000 7fb4fe200640 1 -- 172.21.15.114:0/2267128968 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4f806a380 con 0x7fb4f81593b0 2024-05-27T09:32:55.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.313+0000 7fb4efe00640 1 -- 172.21.15.114:0/2267128968 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+6060 (secure 0 0 0) 0x7fb4e0060720 con 0x7fb4f81593b0 2024-05-27T09:32:55.315 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:32:55.315 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:32:51.967298+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300,4828,4875]},"id":1}]} 2024-05-27T09:32:55.315 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 29 2024-05-27T09:32:55.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.316+0000 7fb4fe200640 1 -- 172.21.15.114:0/2267128968 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb4d40762d0 msgr2=0x7fb4d40787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:55.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.316+0000 7fb4fe200640 1 --2- 172.21.15.114:0/2267128968 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb4d40762d0 0x7fb4d40787e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8034000 tx=0x7fb4e802e000 comp rx=0 tx=0).stop 2024-05-27T09:32:55.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.316+0000 7fb4fe200640 1 -- 172.21.15.114:0/2267128968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f81593b0 msgr2=0x7fb4f8155710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:55.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.316+0000 7fb4fe200640 1 --2- 172.21.15.114:0/2267128968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f81593b0 0x7fb4f8155710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000e870 tx=0x7fb4e000ed30 comp rx=0 tx=0).stop 2024-05-27T09:32:55.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.316+0000 7fb4fe200640 1 -- 172.21.15.114:0/2267128968 shutdown_connections 2024-05-27T09:32:55.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.316+0000 7fb4fe200640 1 --2- 172.21.15.114:0/2267128968 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb4d40762d0 0x7fb4d40787e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:55.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.316+0000 7fb4fe200640 1 --2- 172.21.15.114:0/2267128968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4f815ade0 0x7fb4f815c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:55.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.316+0000 7fb4fe200640 1 --2- 172.21.15.114:0/2267128968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f81593b0 0x7fb4f8155710 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:55.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.316+0000 7fb4fe200640 1 --2- 172.21.15.114:0/2267128968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4f8073260 0x7fb4f8154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:55.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.316+0000 7fb4fe200640 1 -- 172.21.15.114:0/2267128968 >> 172.21.15.114:0/2267128968 conn(0x7fb4f806f6e0 msgr2=0x7fb4f81597c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:55.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.316+0000 7fb4fe200640 1 -- 172.21.15.114:0/2267128968 shutdown_connections 2024-05-27T09:32:55.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:55.316+0000 7fb4fe200640 1 -- 172.21.15.114:0/2267128968 wait complete. 2024-05-27T09:32:55.343 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-05-27T09:32:57.003 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/two 2024-05-27T09:32:57.038 INFO:tasks.workunit.client.0.smithi114.stderr:+ chmod 777 a/foo 2024-05-27T09:32:57.040 INFO:tasks.workunit.client.0.smithi114.stderr:+ sync 2024-05-27T09:32:57.259 INFO:tasks.workunit.client.0.smithi114.stderr:+ ps 2024-05-27T09:32:57.289 INFO:tasks.workunit.client.0.smithi114.stdout: PID TTY TIME CMD 2024-05-27T09:32:57.289 INFO:tasks.workunit.client.0.smithi114.stdout: 1071 ? 00:00:00 systemd 2024-05-27T09:32:57.289 INFO:tasks.workunit.client.0.smithi114.stdout: 1073 ? 00:00:00 (sd-pam) 2024-05-27T09:32:57.289 INFO:tasks.workunit.client.0.smithi114.stdout: 22870 ? 00:00:02 sshd 2024-05-27T09:32:57.289 INFO:tasks.workunit.client.0.smithi114.stdout: 47948 ? 00:00:00 timeout 2024-05-27T09:32:57.289 INFO:tasks.workunit.client.0.smithi114.stdout: 48697 ? 00:00:00 snaptest-double 2024-05-27T09:32:57.289 INFO:tasks.workunit.client.0.smithi114.stdout: 48831 ? 00:00:00 ps 2024-05-27T09:32:57.291 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a/.snap/one a/.snap/two 2024-05-27T09:32:57.316 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm a/foo 2024-05-27T09:32:57.324 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a 2024-05-27T09:32:57.338 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 20` 2024-05-27T09:32:57.338 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a 2024-05-27T09:32:57.344 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:32:57.353 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat 2024-05-27T09:32:57.353 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/one 2024-05-27T09:32:57.387 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/two 2024-05-27T09:32:57.416 INFO:tasks.workunit.client.0.smithi114.stderr:+ chmod 777 a/foo 2024-05-27T09:32:57.419 INFO:tasks.workunit.client.0.smithi114.stderr:+ sync 2024-05-27T09:32:57.491 INFO:tasks.workunit.client.0.smithi114.stderr:+ ps 2024-05-27T09:32:57.514 INFO:tasks.workunit.client.0.smithi114.stdout: PID TTY TIME CMD 2024-05-27T09:32:57.515 INFO:tasks.workunit.client.0.smithi114.stdout: 1071 ? 00:00:00 systemd 2024-05-27T09:32:57.515 INFO:tasks.workunit.client.0.smithi114.stdout: 1073 ? 00:00:00 (sd-pam) 2024-05-27T09:32:57.515 INFO:tasks.workunit.client.0.smithi114.stdout: 22870 ? 00:00:02 sshd 2024-05-27T09:32:57.515 INFO:tasks.workunit.client.0.smithi114.stdout: 47948 ? 00:00:00 timeout 2024-05-27T09:32:57.515 INFO:tasks.workunit.client.0.smithi114.stdout: 48697 ? 00:00:00 snaptest-double 2024-05-27T09:32:57.515 INFO:tasks.workunit.client.0.smithi114.stdout: 48861 ? 00:00:00 ps 2024-05-27T09:32:57.516 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a/.snap/one a/.snap/two 2024-05-27T09:32:57.541 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm a/foo 2024-05-27T09:32:57.549 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a 2024-05-27T09:32:57.569 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 20` 2024-05-27T09:32:57.569 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a 2024-05-27T09:32:57.575 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/one 2024-05-27T09:32:57.576 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat 2024-05-27T09:32:57.613 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/two 2024-05-27T09:32:57.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.622+0000 7fab47000640 1 Processor -- start 2024-05-27T09:32:57.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.622+0000 7fab47000640 1 -- start start 2024-05-27T09:32:57.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.624+0000 7fab47000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab401593b0 0x7fab40142720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:57.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.624+0000 7fab47000640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab400739a0 con 0x7fab400c2990 2024-05-27T09:32:57.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.624+0000 7fab47000640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab40073c80 con 0x7fab40073260 2024-05-27T09:32:57.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.624+0000 7fab47000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fab40074d20 con 0x7fab401593b0 2024-05-27T09:32:57.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.625+0000 7fab3fe00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab401593b0 0x7fab40142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:57.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.625+0000 7fab3fe00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab401593b0 0x7fab40142720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:47110/0 (socket says 172.21.15.114:47110) 2024-05-27T09:32:57.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.625+0000 7fab3fe00640 1 -- 172.21.15.114:0/2728090803 learned_addr learned my addr 172.21.15.114:0/2728090803 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:57.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.625+0000 7fab3fe00640 1 -- 172.21.15.114:0/2728090803 >> v1:172.21.15.163:6789/0 conn(0x7fab40073260 legacy=0x7fab4015a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:57.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.625+0000 7fab3fe00640 1 -- 172.21.15.114:0/2728090803 >> v1:172.21.15.114:6789/0 conn(0x7fab400c2990 legacy=0x7fab40163d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:57.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.625+0000 7fab3fe00640 1 -- 172.21.15.114:0/2728090803 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab40073f60 con 0x7fab401593b0 2024-05-27T09:32:57.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.626+0000 7fab3fe00640 1 --2- 172.21.15.114:0/2728090803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab401593b0 0x7fab40142720 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fab300098a0 tx=0x7fab3002a370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87855b626a522887 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:57.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.626+0000 7fab3ea00640 1 -- 172.21.15.114:0/2728090803 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab30037070 con 0x7fab401593b0 2024-05-27T09:32:57.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.627+0000 7fab3ea00640 1 -- 172.21.15.114:0/2728090803 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab30032640 con 0x7fab401593b0 2024-05-27T09:32:57.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.627+0000 7fab3ea00640 1 -- 172.21.15.114:0/2728090803 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab30032a10 con 0x7fab401593b0 2024-05-27T09:32:57.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.627+0000 7fab47000640 1 -- 172.21.15.114:0/2728090803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab401593b0 msgr2=0x7fab40142720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:57.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.627+0000 7fab47000640 1 --2- 172.21.15.114:0/2728090803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab401593b0 0x7fab40142720 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fab300098a0 tx=0x7fab3002a370 comp rx=0 tx=0).stop 2024-05-27T09:32:57.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.628+0000 7fab47000640 1 -- 172.21.15.114:0/2728090803 shutdown_connections 2024-05-27T09:32:57.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.628+0000 7fab47000640 1 --2- 172.21.15.114:0/2728090803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab401593b0 0x7fab40142720 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:57.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.628+0000 7fab47000640 1 -- 172.21.15.114:0/2728090803 >> 172.21.15.114:0/2728090803 conn(0x7fab4006f6e0 msgr2=0x7fab40070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:57.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.628+0000 7fab47000640 1 -- 172.21.15.114:0/2728090803 shutdown_connections 2024-05-27T09:32:57.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.628+0000 7fab47000640 1 -- 172.21.15.114:0/2728090803 wait complete. 2024-05-27T09:32:57.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.630+0000 7fab47000640 1 Processor -- start 2024-05-27T09:32:57.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.630+0000 7fab47000640 1 -- start start 2024-05-27T09:32:57.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.631+0000 7fab47000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab40073260 0x7fab400b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:57.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.631+0000 7fab47000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab400c2990 0x7fab40141280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:57.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.631+0000 7fab47000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab401593b0 0x7fab4013c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:57.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.631+0000 7fab47000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fab40074800 con 0x7fab401593b0 2024-05-27T09:32:57.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.631+0000 7fab47000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fab40074520 con 0x7fab40073260 2024-05-27T09:32:57.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.632+0000 7fab47000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fab40073670 con 0x7fab400c2990 2024-05-27T09:32:57.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.632+0000 7fab45000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab401593b0 0x7fab4013c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:57.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.632+0000 7fab45000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab401593b0 0x7fab4013c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37466/0 (socket says 172.21.15.114:37466) 2024-05-27T09:32:57.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.632+0000 7fab3fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab40073260 0x7fab400b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:57.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.632+0000 7fab45000640 1 -- 172.21.15.114:0/1860715943 learned_addr learned my addr 172.21.15.114:0/1860715943 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:32:57.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.632+0000 7fab3f400640 1 --2- 172.21.15.114:0/1860715943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab400c2990 0x7fab40141280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:57.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.632+0000 7fab45000640 1 -- 172.21.15.114:0/1860715943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab400c2990 msgr2=0x7fab40141280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:57.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.632+0000 7fab45000640 1 --2- 172.21.15.114:0/1860715943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab400c2990 0x7fab40141280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:57.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.632+0000 7fab45000640 1 -- 172.21.15.114:0/1860715943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab40073260 msgr2=0x7fab400b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:57.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.632+0000 7fab45000640 1 --2- 172.21.15.114:0/1860715943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab40073260 0x7fab400b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:57.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.632+0000 7fab45000640 1 -- 172.21.15.114:0/1860715943 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab400b1cb0 con 0x7fab401593b0 2024-05-27T09:32:57.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.632+0000 7fab3fe00640 1 --2- 172.21.15.114:0/1860715943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab40073260 0x7fab400b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:32:57.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.632+0000 7fab3f400640 1 --2- 172.21.15.114:0/1860715943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab400c2990 0x7fab40141280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:32:57.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.633+0000 7fab45000640 1 --2- 172.21.15.114:0/1860715943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab401593b0 0x7fab4013c450 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fab3800c6e0 tx=0x7fab3800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:57.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.633+0000 7fab37e00640 1 -- 172.21.15.114:0/1860715943 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab38003b70 con 0x7fab401593b0 2024-05-27T09:32:57.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.633+0000 7fab37e00640 1 -- 172.21.15.114:0/1860715943 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab38003d10 con 0x7fab401593b0 2024-05-27T09:32:57.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.633+0000 7fab47000640 1 -- 172.21.15.114:0/1860715943 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab40154b50 con 0x7fab401593b0 2024-05-27T09:32:57.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.633+0000 7fab37e00640 1 -- 172.21.15.114:0/1860715943 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab380052f0 con 0x7fab401593b0 2024-05-27T09:32:57.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.633+0000 7fab47000640 1 -- 172.21.15.114:0/1860715943 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab401f62e0 con 0x7fab401593b0 2024-05-27T09:32:57.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.634+0000 7fab47000640 1 -- 172.21.15.114:0/1860715943 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab40154e30 con 0x7fab401593b0 2024-05-27T09:32:57.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.636+0000 7fab37e00640 1 -- 172.21.15.114:0/1860715943 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fab38005490 con 0x7fab401593b0 2024-05-27T09:32:57.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.637+0000 7fab37e00640 1 --2- 172.21.15.114:0/1860715943 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fab200762d0 0x7fab200787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:57.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.638+0000 7fab37e00640 1 -- 172.21.15.114:0/1860715943 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6040+0+0 (secure 0 0 0) 0x7fab380973e0 con 0x7fab401593b0 2024-05-27T09:32:57.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.638+0000 7fab3fe00640 1 --2- 172.21.15.114:0/1860715943 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fab200762d0 0x7fab200787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:57.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.638+0000 7fab3fe00640 1 --2- 172.21.15.114:0/1860715943 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fab200762d0 0x7fab200787e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fab30034000 tx=0x7fab3002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:57.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.643+0000 7fab37e00640 1 -- 172.21.15.114:0/1860715943 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fab380605b0 con 0x7fab401593b0 2024-05-27T09:32:57.646 INFO:tasks.workunit.client.0.smithi114.stderr:+ chmod 777 a/foo 2024-05-27T09:32:57.648 INFO:tasks.workunit.client.0.smithi114.stderr:+ sync 2024-05-27T09:32:57.734 INFO:tasks.workunit.client.0.smithi114.stderr:+ ps 2024-05-27T09:32:57.756 INFO:tasks.workunit.client.0.smithi114.stdout: PID TTY TIME CMD 2024-05-27T09:32:57.756 INFO:tasks.workunit.client.0.smithi114.stdout: 1071 ? 00:00:00 systemd 2024-05-27T09:32:57.756 INFO:tasks.workunit.client.0.smithi114.stdout: 1073 ? 00:00:00 (sd-pam) 2024-05-27T09:32:57.756 INFO:tasks.workunit.client.0.smithi114.stdout: 22870 ? 00:00:02 sshd 2024-05-27T09:32:57.756 INFO:tasks.workunit.client.0.smithi114.stdout: 47948 ? 00:00:00 timeout 2024-05-27T09:32:57.756 INFO:tasks.workunit.client.0.smithi114.stdout: 48697 ? 00:00:00 snaptest-double 2024-05-27T09:32:57.756 INFO:tasks.workunit.client.0.smithi114.stdout: 48891 ? 00:00:00 ps 2024-05-27T09:32:57.758 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a/.snap/one a/.snap/two 2024-05-27T09:32:57.784 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm a/foo 2024-05-27T09:32:57.792 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a 2024-05-27T09:32:57.811 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 20` 2024-05-27T09:32:57.811 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a 2024-05-27T09:32:57.819 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/one 2024-05-27T09:32:57.819 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat 2024-05-27T09:32:57.858 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/two 2024-05-27T09:32:57.890 INFO:tasks.workunit.client.0.smithi114.stderr:+ chmod 777 a/foo 2024-05-27T09:32:57.893 INFO:tasks.workunit.client.0.smithi114.stderr:+ sync 2024-05-27T09:32:57.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.898+0000 7fab47000640 1 -- 172.21.15.114:0/1860715943 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab4006a380 con 0x7fab401593b0 2024-05-27T09:32:57.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.899+0000 7fab37e00640 1 -- 172.21.15.114:0/1860715943 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+6888 (secure 0 0 0) 0x7fab380648b0 con 0x7fab401593b0 2024-05-27T09:32:57.900 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:32:57.900 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:32:51.967298+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4875,4828,4300]},"id":1}]} 2024-05-27T09:32:57.900 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 30 2024-05-27T09:32:57.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.901+0000 7fab47000640 1 -- 172.21.15.114:0/1860715943 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fab200762d0 msgr2=0x7fab200787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:57.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.901+0000 7fab47000640 1 --2- 172.21.15.114:0/1860715943 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fab200762d0 0x7fab200787e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fab30034000 tx=0x7fab3002e000 comp rx=0 tx=0).stop 2024-05-27T09:32:57.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.901+0000 7fab47000640 1 -- 172.21.15.114:0/1860715943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab401593b0 msgr2=0x7fab4013c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:57.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.901+0000 7fab47000640 1 --2- 172.21.15.114:0/1860715943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab401593b0 0x7fab4013c450 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fab3800c6e0 tx=0x7fab3800cba0 comp rx=0 tx=0).stop 2024-05-27T09:32:57.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.901+0000 7fab47000640 1 -- 172.21.15.114:0/1860715943 shutdown_connections 2024-05-27T09:32:57.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.901+0000 7fab47000640 1 --2- 172.21.15.114:0/1860715943 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fab200762d0 0x7fab200787e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:57.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.901+0000 7fab47000640 1 --2- 172.21.15.114:0/1860715943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab401593b0 0x7fab4013c450 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:57.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.901+0000 7fab47000640 1 --2- 172.21.15.114:0/1860715943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab400c2990 0x7fab40141280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:57.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.901+0000 7fab47000640 1 --2- 172.21.15.114:0/1860715943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab40073260 0x7fab400b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:57.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.901+0000 7fab47000640 1 -- 172.21.15.114:0/1860715943 >> 172.21.15.114:0/1860715943 conn(0x7fab4006f6e0 msgr2=0x7fab400b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:57.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.901+0000 7fab47000640 1 -- 172.21.15.114:0/1860715943 shutdown_connections 2024-05-27T09:32:57.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:32:57.901+0000 7fab47000640 1 -- 172.21.15.114:0/1860715943 wait complete. 2024-05-27T09:32:57.926 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2024-05-27T09:32:57.926 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 51 secs before thrashing 2024-05-27T09:32:57.967 INFO:tasks.workunit.client.0.smithi114.stderr:+ ps 2024-05-27T09:32:57.990 INFO:tasks.workunit.client.0.smithi114.stdout: PID TTY TIME CMD 2024-05-27T09:32:57.990 INFO:tasks.workunit.client.0.smithi114.stdout: 1071 ? 00:00:00 systemd 2024-05-27T09:32:57.990 INFO:tasks.workunit.client.0.smithi114.stdout: 1073 ? 00:00:00 (sd-pam) 2024-05-27T09:32:57.990 INFO:tasks.workunit.client.0.smithi114.stdout: 22870 ? 00:00:02 sshd 2024-05-27T09:32:57.990 INFO:tasks.workunit.client.0.smithi114.stdout: 47948 ? 00:00:00 timeout 2024-05-27T09:32:57.990 INFO:tasks.workunit.client.0.smithi114.stdout: 48697 ? 00:00:00 snaptest-double 2024-05-27T09:32:57.990 INFO:tasks.workunit.client.0.smithi114.stdout: 48903 ? 00:00:00 ps 2024-05-27T09:32:57.991 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a/.snap/one a/.snap/two 2024-05-27T09:32:58.030 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm a/foo 2024-05-27T09:32:58.039 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a 2024-05-27T09:32:58.063 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 20` 2024-05-27T09:32:58.063 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a 2024-05-27T09:32:58.072 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/one 2024-05-27T09:32:58.072 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat 2024-05-27T09:32:58.115 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/two 2024-05-27T09:32:58.148 INFO:tasks.workunit.client.0.smithi114.stderr:+ chmod 777 a/foo 2024-05-27T09:32:58.151 INFO:tasks.workunit.client.0.smithi114.stderr:+ sync 2024-05-27T09:32:58.260 INFO:tasks.workunit.client.0.smithi114.stderr:+ ps 2024-05-27T09:32:58.284 INFO:tasks.workunit.client.0.smithi114.stdout: PID TTY TIME CMD 2024-05-27T09:32:58.284 INFO:tasks.workunit.client.0.smithi114.stdout: 1071 ? 00:00:00 systemd 2024-05-27T09:32:58.284 INFO:tasks.workunit.client.0.smithi114.stdout: 1073 ? 00:00:00 (sd-pam) 2024-05-27T09:32:58.284 INFO:tasks.workunit.client.0.smithi114.stdout: 22870 ? 00:00:02 sshd 2024-05-27T09:32:58.285 INFO:tasks.workunit.client.0.smithi114.stdout: 47948 ? 00:00:00 timeout 2024-05-27T09:32:58.285 INFO:tasks.workunit.client.0.smithi114.stdout: 48697 ? 00:00:00 snaptest-double 2024-05-27T09:32:58.285 INFO:tasks.workunit.client.0.smithi114.stdout: 48913 ? 00:00:00 ps 2024-05-27T09:32:58.286 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a/.snap/one a/.snap/two 2024-05-27T09:32:58.319 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm a/foo 2024-05-27T09:32:58.328 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a 2024-05-27T09:32:58.346 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 20` 2024-05-27T09:32:58.346 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a 2024-05-27T09:32:58.354 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/one 2024-05-27T09:32:58.354 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat 2024-05-27T09:32:58.382 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/two 2024-05-27T09:32:58.404 INFO:tasks.workunit.client.0.smithi114.stderr:+ chmod 777 a/foo 2024-05-27T09:32:58.407 INFO:tasks.workunit.client.0.smithi114.stderr:+ sync 2024-05-27T09:32:58.493 INFO:tasks.workunit.client.0.smithi114.stderr:+ ps 2024-05-27T09:32:58.517 INFO:tasks.workunit.client.0.smithi114.stdout: PID TTY TIME CMD 2024-05-27T09:32:58.517 INFO:tasks.workunit.client.0.smithi114.stdout: 1071 ? 00:00:00 systemd 2024-05-27T09:32:58.517 INFO:tasks.workunit.client.0.smithi114.stdout: 1073 ? 00:00:00 (sd-pam) 2024-05-27T09:32:58.517 INFO:tasks.workunit.client.0.smithi114.stdout: 22870 ? 00:00:02 sshd 2024-05-27T09:32:58.517 INFO:tasks.workunit.client.0.smithi114.stdout: 47948 ? 00:00:00 timeout 2024-05-27T09:32:58.517 INFO:tasks.workunit.client.0.smithi114.stdout: 48697 ? 00:00:00 snaptest-double 2024-05-27T09:32:58.517 INFO:tasks.workunit.client.0.smithi114.stdout: 48923 ? 00:00:00 ps 2024-05-27T09:32:58.519 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a/.snap/one a/.snap/two 2024-05-27T09:32:58.546 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm a/foo 2024-05-27T09:32:58.557 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a 2024-05-27T09:32:58.578 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 20` 2024-05-27T09:32:58.579 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a 2024-05-27T09:32:58.584 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/one 2024-05-27T09:32:58.584 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat 2024-05-27T09:32:58.617 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/two 2024-05-27T09:32:58.642 INFO:tasks.workunit.client.0.smithi114.stderr:+ chmod 777 a/foo 2024-05-27T09:32:58.645 INFO:tasks.workunit.client.0.smithi114.stderr:+ sync 2024-05-27T09:32:58.726 INFO:tasks.workunit.client.0.smithi114.stderr:+ ps 2024-05-27T09:32:58.750 INFO:tasks.workunit.client.0.smithi114.stdout: PID TTY TIME CMD 2024-05-27T09:32:58.750 INFO:tasks.workunit.client.0.smithi114.stdout: 1071 ? 00:00:00 systemd 2024-05-27T09:32:58.751 INFO:tasks.workunit.client.0.smithi114.stdout: 1073 ? 00:00:00 (sd-pam) 2024-05-27T09:32:58.751 INFO:tasks.workunit.client.0.smithi114.stdout: 22870 ? 00:00:02 sshd 2024-05-27T09:32:58.751 INFO:tasks.workunit.client.0.smithi114.stdout: 47948 ? 00:00:00 timeout 2024-05-27T09:32:58.751 INFO:tasks.workunit.client.0.smithi114.stdout: 48697 ? 00:00:00 snaptest-double 2024-05-27T09:32:58.751 INFO:tasks.workunit.client.0.smithi114.stdout: 48933 ? 00:00:00 ps 2024-05-27T09:32:58.752 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a/.snap/one a/.snap/two 2024-05-27T09:32:58.778 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm a/foo 2024-05-27T09:32:58.785 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a 2024-05-27T09:32:58.808 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 20` 2024-05-27T09:32:58.808 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a 2024-05-27T09:32:58.817 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/one 2024-05-27T09:32:58.818 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat 2024-05-27T09:32:58.857 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/two 2024-05-27T09:32:58.881 INFO:tasks.workunit.client.0.smithi114.stderr:+ chmod 777 a/foo 2024-05-27T09:32:58.884 INFO:tasks.workunit.client.0.smithi114.stderr:+ sync 2024-05-27T09:32:58.969 INFO:tasks.workunit.client.0.smithi114.stderr:+ ps 2024-05-27T09:32:58.993 INFO:tasks.workunit.client.0.smithi114.stdout: PID TTY TIME CMD 2024-05-27T09:32:58.993 INFO:tasks.workunit.client.0.smithi114.stdout: 1071 ? 00:00:00 systemd 2024-05-27T09:32:58.993 INFO:tasks.workunit.client.0.smithi114.stdout: 1073 ? 00:00:00 (sd-pam) 2024-05-27T09:32:58.994 INFO:tasks.workunit.client.0.smithi114.stdout: 22870 ? 00:00:02 sshd 2024-05-27T09:32:58.994 INFO:tasks.workunit.client.0.smithi114.stdout: 47948 ? 00:00:00 timeout 2024-05-27T09:32:58.994 INFO:tasks.workunit.client.0.smithi114.stdout: 48697 ? 00:00:00 snaptest-double 2024-05-27T09:32:58.994 INFO:tasks.workunit.client.0.smithi114.stdout: 48943 ? 00:00:00 ps 2024-05-27T09:32:58.995 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a/.snap/one a/.snap/two 2024-05-27T09:32:59.023 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm a/foo 2024-05-27T09:32:59.035 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a 2024-05-27T09:32:59.069 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 20` 2024-05-27T09:32:59.100 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a 2024-05-27T09:32:59.100 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/one 2024-05-27T09:32:59.101 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat 2024-05-27T09:32:59.139 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/two 2024-05-27T09:32:59.176 INFO:tasks.workunit.client.0.smithi114.stderr:+ chmod 777 a/foo 2024-05-27T09:32:59.178 INFO:tasks.workunit.client.0.smithi114.stderr:+ sync 2024-05-27T09:32:59.370 INFO:tasks.workunit.client.0.smithi114.stderr:+ ps 2024-05-27T09:32:59.394 INFO:tasks.workunit.client.0.smithi114.stdout: PID TTY TIME CMD 2024-05-27T09:32:59.394 INFO:tasks.workunit.client.0.smithi114.stdout: 1071 ? 00:00:00 systemd 2024-05-27T09:32:59.395 INFO:tasks.workunit.client.0.smithi114.stdout: 1073 ? 00:00:00 (sd-pam) 2024-05-27T09:32:59.395 INFO:tasks.workunit.client.0.smithi114.stdout: 22870 ? 00:00:02 sshd 2024-05-27T09:32:59.395 INFO:tasks.workunit.client.0.smithi114.stdout: 47948 ? 00:00:00 timeout 2024-05-27T09:32:59.395 INFO:tasks.workunit.client.0.smithi114.stdout: 48697 ? 00:00:00 snaptest-double 2024-05-27T09:32:59.395 INFO:tasks.workunit.client.0.smithi114.stdout: 48953 ? 00:00:00 ps 2024-05-27T09:32:59.396 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a/.snap/one a/.snap/two 2024-05-27T09:32:59.440 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm a/foo 2024-05-27T09:32:59.448 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a 2024-05-27T09:32:59.463 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 20` 2024-05-27T09:32:59.464 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a 2024-05-27T09:32:59.468 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/one 2024-05-27T09:32:59.468 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat 2024-05-27T09:32:59.493 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/two 2024-05-27T09:32:59.513 INFO:tasks.workunit.client.0.smithi114.stderr:+ chmod 777 a/foo 2024-05-27T09:32:59.516 INFO:tasks.workunit.client.0.smithi114.stderr:+ sync 2024-05-27T09:32:59.644 INFO:tasks.workunit.client.0.smithi114.stderr:+ ps 2024-05-27T09:32:59.669 INFO:tasks.workunit.client.0.smithi114.stdout: PID TTY TIME CMD 2024-05-27T09:32:59.669 INFO:tasks.workunit.client.0.smithi114.stdout: 1071 ? 00:00:00 systemd 2024-05-27T09:32:59.669 INFO:tasks.workunit.client.0.smithi114.stdout: 1073 ? 00:00:00 (sd-pam) 2024-05-27T09:32:59.669 INFO:tasks.workunit.client.0.smithi114.stdout: 22870 ? 00:00:02 sshd 2024-05-27T09:32:59.669 INFO:tasks.workunit.client.0.smithi114.stdout: 47948 ? 00:00:00 timeout 2024-05-27T09:32:59.669 INFO:tasks.workunit.client.0.smithi114.stdout: 48697 ? 00:00:00 snaptest-double 2024-05-27T09:32:59.669 INFO:tasks.workunit.client.0.smithi114.stdout: 48963 ? 00:00:00 ps 2024-05-27T09:32:59.671 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a/.snap/one a/.snap/two 2024-05-27T09:32:59.701 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm a/foo 2024-05-27T09:32:59.709 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a 2024-05-27T09:32:59.729 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 20` 2024-05-27T09:32:59.729 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a 2024-05-27T09:32:59.737 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat 2024-05-27T09:32:59.737 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/one 2024-05-27T09:32:59.767 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/two 2024-05-27T09:32:59.795 INFO:tasks.workunit.client.0.smithi114.stderr:+ chmod 777 a/foo 2024-05-27T09:32:59.799 INFO:tasks.workunit.client.0.smithi114.stderr:+ sync 2024-05-27T09:32:59.886 INFO:tasks.workunit.client.0.smithi114.stderr:+ ps 2024-05-27T09:32:59.912 INFO:tasks.workunit.client.0.smithi114.stdout: PID TTY TIME CMD 2024-05-27T09:32:59.912 INFO:tasks.workunit.client.0.smithi114.stdout: 1071 ? 00:00:00 systemd 2024-05-27T09:32:59.912 INFO:tasks.workunit.client.0.smithi114.stdout: 1073 ? 00:00:00 (sd-pam) 2024-05-27T09:32:59.912 INFO:tasks.workunit.client.0.smithi114.stdout: 22870 ? 00:00:02 sshd 2024-05-27T09:32:59.912 INFO:tasks.workunit.client.0.smithi114.stdout: 47948 ? 00:00:00 timeout 2024-05-27T09:32:59.912 INFO:tasks.workunit.client.0.smithi114.stdout: 48697 ? 00:00:00 snaptest-double 2024-05-27T09:32:59.912 INFO:tasks.workunit.client.0.smithi114.stdout: 48973 ? 00:00:00 ps 2024-05-27T09:32:59.913 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a/.snap/one a/.snap/two 2024-05-27T09:32:59.939 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm a/foo 2024-05-27T09:32:59.947 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a 2024-05-27T09:32:59.971 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 20` 2024-05-27T09:32:59.971 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a 2024-05-27T09:32:59.979 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/one 2024-05-27T09:32:59.979 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat 2024-05-27T09:33:00.017 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/two 2024-05-27T09:33:00.044 INFO:tasks.workunit.client.0.smithi114.stderr:+ chmod 777 a/foo 2024-05-27T09:33:00.050 INFO:tasks.workunit.client.0.smithi114.stderr:+ sync 2024-05-27T09:33:00.145 INFO:tasks.workunit.client.0.smithi114.stderr:+ ps 2024-05-27T09:33:00.166 INFO:tasks.workunit.client.0.smithi114.stdout: PID TTY TIME CMD 2024-05-27T09:33:00.166 INFO:tasks.workunit.client.0.smithi114.stdout: 1071 ? 00:00:00 systemd 2024-05-27T09:33:00.166 INFO:tasks.workunit.client.0.smithi114.stdout: 1073 ? 00:00:00 (sd-pam) 2024-05-27T09:33:00.166 INFO:tasks.workunit.client.0.smithi114.stdout: 22870 ? 00:00:02 sshd 2024-05-27T09:33:00.166 INFO:tasks.workunit.client.0.smithi114.stdout: 47948 ? 00:00:00 timeout 2024-05-27T09:33:00.166 INFO:tasks.workunit.client.0.smithi114.stdout: 48697 ? 00:00:00 snaptest-double 2024-05-27T09:33:00.167 INFO:tasks.workunit.client.0.smithi114.stdout: 48983 ? 00:00:00 ps 2024-05-27T09:33:00.168 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a/.snap/one a/.snap/two 2024-05-27T09:33:00.205 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm a/foo 2024-05-27T09:33:00.213 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a 2024-05-27T09:33:00.236 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 20` 2024-05-27T09:33:00.236 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a 2024-05-27T09:33:00.246 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/one 2024-05-27T09:33:00.246 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat 2024-05-27T09:33:00.276 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/two 2024-05-27T09:33:00.302 INFO:tasks.workunit.client.0.smithi114.stderr:+ chmod 777 a/foo 2024-05-27T09:33:00.307 INFO:tasks.workunit.client.0.smithi114.stderr:+ sync 2024-05-27T09:33:00.428 INFO:tasks.workunit.client.0.smithi114.stderr:+ ps 2024-05-27T09:33:00.451 INFO:tasks.workunit.client.0.smithi114.stdout: PID TTY TIME CMD 2024-05-27T09:33:00.451 INFO:tasks.workunit.client.0.smithi114.stdout: 1071 ? 00:00:00 systemd 2024-05-27T09:33:00.452 INFO:tasks.workunit.client.0.smithi114.stdout: 1073 ? 00:00:00 (sd-pam) 2024-05-27T09:33:00.452 INFO:tasks.workunit.client.0.smithi114.stdout: 22870 ? 00:00:02 sshd 2024-05-27T09:33:00.452 INFO:tasks.workunit.client.0.smithi114.stdout: 47948 ? 00:00:00 timeout 2024-05-27T09:33:00.452 INFO:tasks.workunit.client.0.smithi114.stdout: 48697 ? 00:00:00 snaptest-double 2024-05-27T09:33:00.452 INFO:tasks.workunit.client.0.smithi114.stdout: 48993 ? 00:00:00 ps 2024-05-27T09:33:00.453 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a/.snap/one a/.snap/two 2024-05-27T09:33:00.489 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm a/foo 2024-05-27T09:33:00.496 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a 2024-05-27T09:33:00.514 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 20` 2024-05-27T09:33:00.514 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a 2024-05-27T09:33:00.523 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/one 2024-05-27T09:33:00.523 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat 2024-05-27T09:33:00.554 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/two 2024-05-27T09:33:00.580 INFO:tasks.workunit.client.0.smithi114.stderr:+ chmod 777 a/foo 2024-05-27T09:33:00.584 INFO:tasks.workunit.client.0.smithi114.stderr:+ sync 2024-05-27T09:33:00.680 INFO:tasks.workunit.client.0.smithi114.stderr:+ ps 2024-05-27T09:33:00.703 INFO:tasks.workunit.client.0.smithi114.stdout: PID TTY TIME CMD 2024-05-27T09:33:00.704 INFO:tasks.workunit.client.0.smithi114.stdout: 1071 ? 00:00:00 systemd 2024-05-27T09:33:00.704 INFO:tasks.workunit.client.0.smithi114.stdout: 1073 ? 00:00:00 (sd-pam) 2024-05-27T09:33:00.704 INFO:tasks.workunit.client.0.smithi114.stdout: 22870 ? 00:00:02 sshd 2024-05-27T09:33:00.704 INFO:tasks.workunit.client.0.smithi114.stdout: 47948 ? 00:00:00 timeout 2024-05-27T09:33:00.704 INFO:tasks.workunit.client.0.smithi114.stdout: 48697 ? 00:00:00 snaptest-double 2024-05-27T09:33:00.704 INFO:tasks.workunit.client.0.smithi114.stdout: 49003 ? 00:00:00 ps 2024-05-27T09:33:00.705 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a/.snap/one a/.snap/two 2024-05-27T09:33:00.739 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm a/foo 2024-05-27T09:33:00.746 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a 2024-05-27T09:33:00.764 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 20` 2024-05-27T09:33:00.764 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a 2024-05-27T09:33:00.773 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/one 2024-05-27T09:33:00.773 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat 2024-05-27T09:33:00.802 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/two 2024-05-27T09:33:00.826 INFO:tasks.workunit.client.0.smithi114.stderr:+ chmod 777 a/foo 2024-05-27T09:33:00.831 INFO:tasks.workunit.client.0.smithi114.stderr:+ sync 2024-05-27T09:33:00.963 INFO:tasks.workunit.client.0.smithi114.stderr:+ ps 2024-05-27T09:33:00.989 INFO:tasks.workunit.client.0.smithi114.stdout: PID TTY TIME CMD 2024-05-27T09:33:00.989 INFO:tasks.workunit.client.0.smithi114.stdout: 1071 ? 00:00:00 systemd 2024-05-27T09:33:00.989 INFO:tasks.workunit.client.0.smithi114.stdout: 1073 ? 00:00:00 (sd-pam) 2024-05-27T09:33:00.989 INFO:tasks.workunit.client.0.smithi114.stdout: 22870 ? 00:00:02 sshd 2024-05-27T09:33:00.989 INFO:tasks.workunit.client.0.smithi114.stdout: 47948 ? 00:00:00 timeout 2024-05-27T09:33:00.989 INFO:tasks.workunit.client.0.smithi114.stdout: 48697 ? 00:00:00 snaptest-double 2024-05-27T09:33:00.989 INFO:tasks.workunit.client.0.smithi114.stdout: 49013 ? 00:00:00 ps 2024-05-27T09:33:00.991 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a/.snap/one a/.snap/two 2024-05-27T09:33:01.023 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm a/foo 2024-05-27T09:33:01.031 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a 2024-05-27T09:33:01.066 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 20` 2024-05-27T09:33:01.066 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a 2024-05-27T09:33:01.074 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat 2024-05-27T09:33:01.076 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/one 2024-05-27T09:33:01.134 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/two 2024-05-27T09:33:01.164 INFO:tasks.workunit.client.0.smithi114.stderr:+ chmod 777 a/foo 2024-05-27T09:33:01.167 INFO:tasks.workunit.client.0.smithi114.stderr:+ sync 2024-05-27T09:33:01.338 INFO:tasks.workunit.client.0.smithi114.stderr:+ ps 2024-05-27T09:33:01.362 INFO:tasks.workunit.client.0.smithi114.stdout: PID TTY TIME CMD 2024-05-27T09:33:01.362 INFO:tasks.workunit.client.0.smithi114.stdout: 1071 ? 00:00:00 systemd 2024-05-27T09:33:01.362 INFO:tasks.workunit.client.0.smithi114.stdout: 1073 ? 00:00:00 (sd-pam) 2024-05-27T09:33:01.363 INFO:tasks.workunit.client.0.smithi114.stdout: 22870 ? 00:00:02 sshd 2024-05-27T09:33:01.363 INFO:tasks.workunit.client.0.smithi114.stdout: 47948 ? 00:00:00 timeout 2024-05-27T09:33:01.363 INFO:tasks.workunit.client.0.smithi114.stdout: 48697 ? 00:00:00 snaptest-double 2024-05-27T09:33:01.363 INFO:tasks.workunit.client.0.smithi114.stdout: 49024 ? 00:00:00 ps 2024-05-27T09:33:01.364 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a/.snap/one a/.snap/two 2024-05-27T09:33:01.399 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm a/foo 2024-05-27T09:33:01.407 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a 2024-05-27T09:33:01.427 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 20` 2024-05-27T09:33:01.427 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a 2024-05-27T09:33:01.436 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/one 2024-05-27T09:33:01.436 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat 2024-05-27T09:33:01.466 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/two 2024-05-27T09:33:01.495 INFO:tasks.workunit.client.0.smithi114.stderr:+ chmod 777 a/foo 2024-05-27T09:33:01.499 INFO:tasks.workunit.client.0.smithi114.stderr:+ sync 2024-05-27T09:33:01.589 INFO:tasks.workunit.client.0.smithi114.stderr:+ ps 2024-05-27T09:33:01.614 INFO:tasks.workunit.client.0.smithi114.stdout: PID TTY TIME CMD 2024-05-27T09:33:01.614 INFO:tasks.workunit.client.0.smithi114.stdout: 1071 ? 00:00:00 systemd 2024-05-27T09:33:01.614 INFO:tasks.workunit.client.0.smithi114.stdout: 1073 ? 00:00:00 (sd-pam) 2024-05-27T09:33:01.614 INFO:tasks.workunit.client.0.smithi114.stdout: 22870 ? 00:00:02 sshd 2024-05-27T09:33:01.614 INFO:tasks.workunit.client.0.smithi114.stdout: 47948 ? 00:00:00 timeout 2024-05-27T09:33:01.614 INFO:tasks.workunit.client.0.smithi114.stdout: 48697 ? 00:00:00 snaptest-double 2024-05-27T09:33:01.614 INFO:tasks.workunit.client.0.smithi114.stdout: 49034 ? 00:00:00 ps 2024-05-27T09:33:01.616 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a/.snap/one a/.snap/two 2024-05-27T09:33:01.650 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm a/foo 2024-05-27T09:33:01.657 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a 2024-05-27T09:33:01.681 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 20` 2024-05-27T09:33:01.681 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a 2024-05-27T09:33:01.689 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/one 2024-05-27T09:33:01.690 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat 2024-05-27T09:33:01.720 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/two 2024-05-27T09:33:01.747 INFO:tasks.workunit.client.0.smithi114.stderr:+ chmod 777 a/foo 2024-05-27T09:33:01.751 INFO:tasks.workunit.client.0.smithi114.stderr:+ sync 2024-05-27T09:33:01.868 INFO:tasks.workunit.client.0.smithi114.stderr:+ ps 2024-05-27T09:33:01.892 INFO:tasks.workunit.client.0.smithi114.stdout: PID TTY TIME CMD 2024-05-27T09:33:01.893 INFO:tasks.workunit.client.0.smithi114.stdout: 1071 ? 00:00:00 systemd 2024-05-27T09:33:01.893 INFO:tasks.workunit.client.0.smithi114.stdout: 1073 ? 00:00:00 (sd-pam) 2024-05-27T09:33:01.893 INFO:tasks.workunit.client.0.smithi114.stdout: 22870 ? 00:00:02 sshd 2024-05-27T09:33:01.893 INFO:tasks.workunit.client.0.smithi114.stdout: 47948 ? 00:00:00 timeout 2024-05-27T09:33:01.893 INFO:tasks.workunit.client.0.smithi114.stdout: 48697 ? 00:00:00 snaptest-double 2024-05-27T09:33:01.893 INFO:tasks.workunit.client.0.smithi114.stdout: 49044 ? 00:00:00 ps 2024-05-27T09:33:01.894 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a/.snap/one a/.snap/two 2024-05-27T09:33:01.928 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm a/foo 2024-05-27T09:33:01.937 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a 2024-05-27T09:33:01.958 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 20` 2024-05-27T09:33:01.958 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a 2024-05-27T09:33:01.968 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/one 2024-05-27T09:33:01.969 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat 2024-05-27T09:33:02.021 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/two 2024-05-27T09:33:02.063 INFO:tasks.workunit.client.0.smithi114.stderr:+ chmod 777 a/foo 2024-05-27T09:33:02.068 INFO:tasks.workunit.client.0.smithi114.stderr:+ sync 2024-05-27T09:33:02.182 INFO:tasks.workunit.client.0.smithi114.stderr:+ ps 2024-05-27T09:33:02.201 INFO:tasks.workunit.client.0.smithi114.stdout: PID TTY TIME CMD 2024-05-27T09:33:02.201 INFO:tasks.workunit.client.0.smithi114.stdout: 1071 ? 00:00:00 systemd 2024-05-27T09:33:02.201 INFO:tasks.workunit.client.0.smithi114.stdout: 1073 ? 00:00:00 (sd-pam) 2024-05-27T09:33:02.201 INFO:tasks.workunit.client.0.smithi114.stdout: 22870 ? 00:00:02 sshd 2024-05-27T09:33:02.201 INFO:tasks.workunit.client.0.smithi114.stdout: 47948 ? 00:00:00 timeout 2024-05-27T09:33:02.201 INFO:tasks.workunit.client.0.smithi114.stdout: 48697 ? 00:00:00 snaptest-double 2024-05-27T09:33:02.202 INFO:tasks.workunit.client.0.smithi114.stdout: 49054 ? 00:00:00 ps 2024-05-27T09:33:02.202 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a/.snap/one a/.snap/two 2024-05-27T09:33:02.218 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm a/foo 2024-05-27T09:33:02.222 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a 2024-05-27T09:33:02.233 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 20` 2024-05-27T09:33:02.233 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a 2024-05-27T09:33:02.237 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/one 2024-05-27T09:33:02.238 INFO:tasks.workunit.client.0.smithi114.stderr:+ cat 2024-05-27T09:33:02.272 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir a/.snap/two 2024-05-27T09:33:02.301 INFO:tasks.workunit.client.0.smithi114.stderr:+ chmod 777 a/foo 2024-05-27T09:33:02.303 INFO:tasks.workunit.client.0.smithi114.stderr:+ sync 2024-05-27T09:33:02.481 INFO:tasks.workunit.client.0.smithi114.stderr:+ ps 2024-05-27T09:33:02.506 INFO:tasks.workunit.client.0.smithi114.stdout: PID TTY TIME CMD 2024-05-27T09:33:02.506 INFO:tasks.workunit.client.0.smithi114.stdout: 1071 ? 00:00:00 systemd 2024-05-27T09:33:02.506 INFO:tasks.workunit.client.0.smithi114.stdout: 1073 ? 00:00:00 (sd-pam) 2024-05-27T09:33:02.506 INFO:tasks.workunit.client.0.smithi114.stdout: 22870 ? 00:00:02 sshd 2024-05-27T09:33:02.506 INFO:tasks.workunit.client.0.smithi114.stdout: 47948 ? 00:00:00 timeout 2024-05-27T09:33:02.506 INFO:tasks.workunit.client.0.smithi114.stdout: 48697 ? 00:00:00 snaptest-double 2024-05-27T09:33:02.506 INFO:tasks.workunit.client.0.smithi114.stdout: 49064 ? 00:00:00 ps 2024-05-27T09:33:02.508 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a/.snap/one a/.snap/two 2024-05-27T09:33:02.544 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm a/foo 2024-05-27T09:33:02.553 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir a 2024-05-27T09:33:02.577 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo OK 2024-05-27T09:33:02.577 INFO:tasks.workunit.client.0.smithi114.stdout:OK 2024-05-27T09:33:02.578 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-27T09:33:02.579 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-27T09:33:02.717 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-estale.sh... 2024-05-27T09:33:02.718 DEBUG:teuthology.orchestra.run.smithi114:workunit test fs/snaps/snaptest-estale.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6507388dd5057528934822c0163b0c347ef1d5d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-estale.sh 2024-05-27T09:33:03.655 INFO:tasks.workunit.client.0.smithi114.stderr:+ mkdir .snap/foo 2024-05-27T09:33:03.693 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo 'We want ENOENT, not ESTALE, here.' 2024-05-27T09:33:03.693 INFO:tasks.workunit.client.0.smithi114.stdout:We want ENOENT, not ESTALE, here. 2024-05-27T09:33:03.694 INFO:tasks.workunit.client.0.smithi114.stderr:++ seq 1 100 2024-05-27T09:33:03.697 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.698 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/1 2024-05-27T09:33:03.698 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.703 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/1': No such file or directory 2024-05-27T09:33:03.703 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.704 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/2 2024-05-27T09:33:03.705 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.707 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/2': No such file or directory 2024-05-27T09:33:03.708 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.708 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/3 2024-05-27T09:33:03.709 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.711 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/3': No such file or directory 2024-05-27T09:33:03.712 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.712 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/4 2024-05-27T09:33:03.713 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.715 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/4': No such file or directory 2024-05-27T09:33:03.716 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.716 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/5 2024-05-27T09:33:03.716 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.719 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/5': No such file or directory 2024-05-27T09:33:03.719 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.720 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/6 2024-05-27T09:33:03.720 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.722 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/6': No such file or directory 2024-05-27T09:33:03.723 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.723 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/7 2024-05-27T09:33:03.724 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.726 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/7': No such file or directory 2024-05-27T09:33:03.727 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.727 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/8 2024-05-27T09:33:03.728 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.730 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/8': No such file or directory 2024-05-27T09:33:03.731 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.732 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/9 2024-05-27T09:33:03.732 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.734 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/9': No such file or directory 2024-05-27T09:33:03.735 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.735 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/10 2024-05-27T09:33:03.736 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.739 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/10': No such file or directory 2024-05-27T09:33:03.739 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.740 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/11 2024-05-27T09:33:03.740 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.742 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/11': No such file or directory 2024-05-27T09:33:03.743 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.743 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/12 2024-05-27T09:33:03.744 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.746 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/12': No such file or directory 2024-05-27T09:33:03.747 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.747 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/13 2024-05-27T09:33:03.748 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.750 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/13': No such file or directory 2024-05-27T09:33:03.750 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.751 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/14 2024-05-27T09:33:03.751 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.754 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/14': No such file or directory 2024-05-27T09:33:03.755 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.755 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/15 2024-05-27T09:33:03.755 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.758 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/15': No such file or directory 2024-05-27T09:33:03.758 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.759 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/16 2024-05-27T09:33:03.759 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.761 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/16': No such file or directory 2024-05-27T09:33:03.762 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.763 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/17 2024-05-27T09:33:03.763 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.765 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/17': No such file or directory 2024-05-27T09:33:03.766 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.766 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/18 2024-05-27T09:33:03.767 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.769 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/18': No such file or directory 2024-05-27T09:33:03.769 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.770 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/19 2024-05-27T09:33:03.770 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.773 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/19': No such file or directory 2024-05-27T09:33:03.773 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.773 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/20 2024-05-27T09:33:03.774 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.776 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/20': No such file or directory 2024-05-27T09:33:03.776 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.777 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/21 2024-05-27T09:33:03.778 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.780 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/21': No such file or directory 2024-05-27T09:33:03.780 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.781 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/22 2024-05-27T09:33:03.782 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.784 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/22': No such file or directory 2024-05-27T09:33:03.785 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.785 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/23 2024-05-27T09:33:03.786 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.788 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/23': No such file or directory 2024-05-27T09:33:03.788 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.789 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/24 2024-05-27T09:33:03.789 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.794 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/24': No such file or directory 2024-05-27T09:33:03.795 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.795 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/25 2024-05-27T09:33:03.795 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.796 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/25': No such file or directory 2024-05-27T09:33:03.797 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.797 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/26 2024-05-27T09:33:03.797 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.800 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/26': No such file or directory 2024-05-27T09:33:03.800 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.801 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/27 2024-05-27T09:33:03.801 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.804 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/27': No such file or directory 2024-05-27T09:33:03.804 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.805 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/28 2024-05-27T09:33:03.805 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.808 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/28': No such file or directory 2024-05-27T09:33:03.808 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.809 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/29 2024-05-27T09:33:03.810 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.812 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/29': No such file or directory 2024-05-27T09:33:03.812 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.813 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/30 2024-05-27T09:33:03.813 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.816 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/30': No such file or directory 2024-05-27T09:33:03.816 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.818 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/31 2024-05-27T09:33:03.818 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.821 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/31': No such file or directory 2024-05-27T09:33:03.821 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.823 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/32 2024-05-27T09:33:03.824 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.826 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/32': No such file or directory 2024-05-27T09:33:03.827 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.828 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/33 2024-05-27T09:33:03.829 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.831 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/33': No such file or directory 2024-05-27T09:33:03.831 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.832 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/34 2024-05-27T09:33:03.833 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.835 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/34': No such file or directory 2024-05-27T09:33:03.836 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.837 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/35 2024-05-27T09:33:03.838 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.840 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/35': No such file or directory 2024-05-27T09:33:03.841 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.842 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/36 2024-05-27T09:33:03.843 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.846 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/36': No such file or directory 2024-05-27T09:33:03.846 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.847 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/37 2024-05-27T09:33:03.848 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.851 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/37': No such file or directory 2024-05-27T09:33:03.851 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.852 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/38 2024-05-27T09:33:03.853 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.855 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/38': No such file or directory 2024-05-27T09:33:03.856 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.857 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/39 2024-05-27T09:33:03.858 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.862 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/39': No such file or directory 2024-05-27T09:33:03.862 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.864 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/40 2024-05-27T09:33:03.865 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.867 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/40': No such file or directory 2024-05-27T09:33:03.868 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.869 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/41 2024-05-27T09:33:03.870 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.873 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/41': No such file or directory 2024-05-27T09:33:03.873 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.875 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/42 2024-05-27T09:33:03.875 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.878 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/42': No such file or directory 2024-05-27T09:33:03.879 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.880 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/43 2024-05-27T09:33:03.881 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.883 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/43': No such file or directory 2024-05-27T09:33:03.884 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.885 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/44 2024-05-27T09:33:03.886 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.888 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/44': No such file or directory 2024-05-27T09:33:03.889 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.890 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/45 2024-05-27T09:33:03.891 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.894 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/45': No such file or directory 2024-05-27T09:33:03.894 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.895 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/46 2024-05-27T09:33:03.897 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.899 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/46': No such file or directory 2024-05-27T09:33:03.899 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.901 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/47 2024-05-27T09:33:03.903 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.925 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/47': No such file or directory 2024-05-27T09:33:03.926 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.927 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/48 2024-05-27T09:33:03.927 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.930 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/48': No such file or directory 2024-05-27T09:33:03.931 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.931 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/49 2024-05-27T09:33:03.932 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.935 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/49': No such file or directory 2024-05-27T09:33:03.935 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.936 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/50 2024-05-27T09:33:03.936 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.939 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/50': No such file or directory 2024-05-27T09:33:03.939 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.940 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/51 2024-05-27T09:33:03.940 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.943 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/51': No such file or directory 2024-05-27T09:33:03.944 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.945 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/52 2024-05-27T09:33:03.945 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.947 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/52': No such file or directory 2024-05-27T09:33:03.948 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.949 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/53 2024-05-27T09:33:03.950 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.953 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/53': No such file or directory 2024-05-27T09:33:03.953 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.954 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/54 2024-05-27T09:33:03.955 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.958 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/54': No such file or directory 2024-05-27T09:33:03.959 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.960 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/55 2024-05-27T09:33:03.961 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.963 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/55': No such file or directory 2024-05-27T09:33:03.964 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.965 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/56 2024-05-27T09:33:03.966 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.968 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/56': No such file or directory 2024-05-27T09:33:03.968 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.970 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/57 2024-05-27T09:33:03.970 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.973 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/57': No such file or directory 2024-05-27T09:33:03.974 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.975 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/58 2024-05-27T09:33:03.976 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.978 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/58': No such file or directory 2024-05-27T09:33:03.979 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.980 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/59 2024-05-27T09:33:03.981 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.983 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/59': No such file or directory 2024-05-27T09:33:03.984 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.985 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/60 2024-05-27T09:33:03.986 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.988 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/60': No such file or directory 2024-05-27T09:33:03.989 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.989 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/61 2024-05-27T09:33:03.990 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.993 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/61': No such file or directory 2024-05-27T09:33:03.993 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.994 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/62 2024-05-27T09:33:03.994 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:03.997 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/62': No such file or directory 2024-05-27T09:33:03.997 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:03.998 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/63 2024-05-27T09:33:03.998 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.000 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/63': No such file or directory 2024-05-27T09:33:04.001 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.001 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/64 2024-05-27T09:33:04.002 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.004 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/64': No such file or directory 2024-05-27T09:33:04.004 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.005 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/65 2024-05-27T09:33:04.005 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.008 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/65': No such file or directory 2024-05-27T09:33:04.008 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.008 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/66 2024-05-27T09:33:04.009 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.011 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/66': No such file or directory 2024-05-27T09:33:04.011 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.012 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/67 2024-05-27T09:33:04.013 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.015 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/67': No such file or directory 2024-05-27T09:33:04.015 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.016 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/68 2024-05-27T09:33:04.016 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.018 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/68': No such file or directory 2024-05-27T09:33:04.019 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.020 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/69 2024-05-27T09:33:04.020 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.022 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/69': No such file or directory 2024-05-27T09:33:04.022 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.023 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/70 2024-05-27T09:33:04.023 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.026 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/70': No such file or directory 2024-05-27T09:33:04.026 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.027 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/71 2024-05-27T09:33:04.027 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.029 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/71': No such file or directory 2024-05-27T09:33:04.030 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.031 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/72 2024-05-27T09:33:04.031 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.033 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/72': No such file or directory 2024-05-27T09:33:04.034 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.034 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/73 2024-05-27T09:33:04.034 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.037 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/73': No such file or directory 2024-05-27T09:33:04.037 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.038 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/74 2024-05-27T09:33:04.038 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.041 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/74': No such file or directory 2024-05-27T09:33:04.041 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.042 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/75 2024-05-27T09:33:04.042 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.045 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/75': No such file or directory 2024-05-27T09:33:04.045 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.045 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/76 2024-05-27T09:33:04.046 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.048 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/76': No such file or directory 2024-05-27T09:33:04.049 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.049 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/77 2024-05-27T09:33:04.050 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.053 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/77': No such file or directory 2024-05-27T09:33:04.053 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.054 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/78 2024-05-27T09:33:04.054 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.057 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/78': No such file or directory 2024-05-27T09:33:04.057 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.058 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/79 2024-05-27T09:33:04.058 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.061 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/79': No such file or directory 2024-05-27T09:33:04.061 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.062 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/80 2024-05-27T09:33:04.062 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.065 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/80': No such file or directory 2024-05-27T09:33:04.065 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.066 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/81 2024-05-27T09:33:04.066 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.070 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/81': No such file or directory 2024-05-27T09:33:04.070 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.075 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/82 2024-05-27T09:33:04.076 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.081 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/82': No such file or directory 2024-05-27T09:33:04.082 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.085 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/83 2024-05-27T09:33:04.086 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.091 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/83': No such file or directory 2024-05-27T09:33:04.092 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.093 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/84 2024-05-27T09:33:04.096 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.099 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/84': No such file or directory 2024-05-27T09:33:04.100 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.101 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/85 2024-05-27T09:33:04.102 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.106 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/85': No such file or directory 2024-05-27T09:33:04.106 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.107 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/86 2024-05-27T09:33:04.108 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.110 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/86': No such file or directory 2024-05-27T09:33:04.111 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.111 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/87 2024-05-27T09:33:04.112 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.114 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/87': No such file or directory 2024-05-27T09:33:04.114 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.115 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/88 2024-05-27T09:33:04.115 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.117 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/88': No such file or directory 2024-05-27T09:33:04.117 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.118 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/89 2024-05-27T09:33:04.118 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.120 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/89': No such file or directory 2024-05-27T09:33:04.121 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.121 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/90 2024-05-27T09:33:04.121 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.123 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/90': No such file or directory 2024-05-27T09:33:04.124 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.124 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/91 2024-05-27T09:33:04.124 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.127 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/91': No such file or directory 2024-05-27T09:33:04.127 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.127 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/92 2024-05-27T09:33:04.128 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.130 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/92': No such file or directory 2024-05-27T09:33:04.130 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.131 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/93 2024-05-27T09:33:04.131 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.133 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/93': No such file or directory 2024-05-27T09:33:04.133 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.134 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/94 2024-05-27T09:33:04.134 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.136 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/94': No such file or directory 2024-05-27T09:33:04.136 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.137 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/95 2024-05-27T09:33:04.137 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.139 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/95': No such file or directory 2024-05-27T09:33:04.140 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.140 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/96 2024-05-27T09:33:04.140 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.142 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/96': No such file or directory 2024-05-27T09:33:04.143 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.143 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/97 2024-05-27T09:33:04.144 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.146 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/97': No such file or directory 2024-05-27T09:33:04.146 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.146 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/98 2024-05-27T09:33:04.147 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.149 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/98': No such file or directory 2024-05-27T09:33:04.149 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.150 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/99 2024-05-27T09:33:04.150 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.152 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/99': No such file or directory 2024-05-27T09:33:04.152 INFO:tasks.workunit.client.0.smithi114.stderr:+ for f in `seq 1 100` 2024-05-27T09:33:04.153 INFO:tasks.workunit.client.0.smithi114.stderr:+ stat .snap/foo/100 2024-05-27T09:33:04.153 INFO:tasks.workunit.client.0.smithi114.stderr:+ grep 'No such file' 2024-05-27T09:33:04.155 INFO:tasks.workunit.client.0.smithi114.stdout:stat: cannot statx '.snap/foo/100': No such file or directory 2024-05-27T09:33:04.156 INFO:tasks.workunit.client.0.smithi114.stderr:+ rmdir .snap/foo 2024-05-27T09:33:04.188 INFO:tasks.workunit.client.0.smithi114.stdout:OK 2024-05-27T09:33:04.189 INFO:tasks.workunit.client.0.smithi114.stderr:+ echo OK 2024-05-27T09:33:04.189 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-27T09:33:04.189 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-27T09:33:04.259 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-git-ceph.sh... 2024-05-27T09:33:04.259 DEBUG:teuthology.orchestra.run.smithi114:workunit test fs/snaps/snaptest-git-ceph.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6507388dd5057528934822c0163b0c347ef1d5d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh 2024-05-27T09:33:08.658 INFO:tasks.workunit.client.0.smithi114.stderr:+ set -e 2024-05-27T09:33:08.658 INFO:tasks.workunit.client.0.smithi114.stderr:+ sudo git config --global http.sslVerify false 2024-05-27T09:33:08.697 INFO:tasks.workunit.client.0.smithi114.stderr:+ sudo git config --global http.postBuffer 1048576000 2024-05-27T09:33:08.729 INFO:tasks.workunit.client.0.smithi114.stderr:+ retried=false 2024-05-27T09:33:08.729 INFO:tasks.workunit.client.0.smithi114.stderr:+ trap -- retry EXIT 2024-05-27T09:33:08.729 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm -rf ceph 2024-05-27T09:33:08.731 INFO:tasks.workunit.client.0.smithi114.stderr:+ timeout 1800 git clone https://git.ceph.com/ceph.git 2024-05-27T09:33:08.739 INFO:tasks.workunit.client.0.smithi114.stderr:Cloning into 'ceph'... 2024-05-27T09:33:19.059 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:33:19.068 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:33:48.928 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:33:49.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.189+0000 7f0c51a00640 1 Processor -- start 2024-05-27T09:33:49.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.189+0000 7f0c51a00640 1 -- start start 2024-05-27T09:33:49.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.190+0000 7f0c51a00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0c4c0741f0 0x7f0c4c0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:49.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.191+0000 7f0c51a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c4c074840 0x7f0c4c15ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:49.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.191+0000 7f0c51a00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c4c06fe50 con 0x7f0c4c1547d0 2024-05-27T09:33:49.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.191+0000 7f0c51a00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f0c4c0703d0 con 0x7f0c4c0741f0 2024-05-27T09:33:49.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.191+0000 7f0c51a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f0c4c15b260 con 0x7f0c4c074840 2024-05-27T09:33:49.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.191+0000 7f0c51000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c4c074840 0x7f0c4c15ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:49.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.191+0000 7f0c4be00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0c4c0741f0 0x7f0c4c0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:49.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.191+0000 7f0c4be00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0c4c0741f0 0x7f0c4c0c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:39832/0 (socket says 172.21.15.114:39832) 2024-05-27T09:33:49.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.191+0000 7f0c4b400640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f0c4c1547d0 0x7f0c4c06edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:36586/0 (socket says 172.21.15.114:36586) 2024-05-27T09:33:49.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.191+0000 7f0c51000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c4c074840 0x7f0c4c15ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:36768/0 (socket says 172.21.15.114:36768) 2024-05-27T09:33:49.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.191+0000 7f0c4be00640 1 -- 172.21.15.114:0/1498139029 learned_addr learned my addr 172.21.15.114:0/1498139029 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:33:49.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.191+0000 7f0c4be00640 1 -- 172.21.15.114:0/1498139029 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c4c074840 msgr2=0x7f0c4c15ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:33:49.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.191+0000 7f0c4be00640 1 --2- 172.21.15.114:0/1498139029 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c4c074840 0x7f0c4c15ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:49.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.191+0000 7f0c4be00640 1 -- 172.21.15.114:0/1498139029 >> v1:172.21.15.163:6789/0 conn(0x7f0c4c1547d0 legacy=0x7f0c4c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:49.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.191+0000 7f0c4be00640 1 -- 172.21.15.114:0/1498139029 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4c0700f0 con 0x7f0c4c0741f0 2024-05-27T09:33:49.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.191+0000 7f0c51000640 1 --2- 172.21.15.114:0/1498139029 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c4c074840 0x7f0c4c15ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:33:49.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.192+0000 7f0c4be00640 1 --2- 172.21.15.114:0/1498139029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4c0741f0 0x7f0c4c0c63a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c009870 tx=0x7f0c3c02a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58e8545316503ac9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:49.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.193+0000 7f0c4aa00640 1 -- 172.21.15.114:0/1498139029 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c037070 con 0x7f0c4c0741f0 2024-05-27T09:33:49.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.193+0000 7f0c4aa00640 1 -- 172.21.15.114:0/1498139029 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c3c02ae50 con 0x7f0c4c0741f0 2024-05-27T09:33:49.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.193+0000 7f0c4aa00640 1 -- 172.21.15.114:0/1498139029 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c032860 con 0x7f0c4c0741f0 2024-05-27T09:33:49.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.193+0000 7f0c51a00640 1 -- 172.21.15.114:0/1498139029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4c0741f0 msgr2=0x7f0c4c0c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:49.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.193+0000 7f0c51a00640 1 --2- 172.21.15.114:0/1498139029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4c0741f0 0x7f0c4c0c63a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c009870 tx=0x7f0c3c02a300 comp rx=0 tx=0).stop 2024-05-27T09:33:49.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.193+0000 7f0c51a00640 1 -- 172.21.15.114:0/1498139029 shutdown_connections 2024-05-27T09:33:49.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.193+0000 7f0c51a00640 1 --2- 172.21.15.114:0/1498139029 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c4c074840 0x7f0c4c15ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:49.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.193+0000 7f0c51a00640 1 --2- 172.21.15.114:0/1498139029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4c0741f0 0x7f0c4c0c63a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:49.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.193+0000 7f0c51a00640 1 -- 172.21.15.114:0/1498139029 >> 172.21.15.114:0/1498139029 conn(0x7f0c4c0ca320 msgr2=0x7f0c4c0ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:49.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.194+0000 7f0c51a00640 1 -- 172.21.15.114:0/1498139029 shutdown_connections 2024-05-27T09:33:49.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.194+0000 7f0c51a00640 1 -- 172.21.15.114:0/1498139029 wait complete. 2024-05-27T09:33:49.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.196+0000 7f0c51a00640 1 Processor -- start 2024-05-27T09:33:49.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.196+0000 7f0c51a00640 1 -- start start 2024-05-27T09:33:49.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.197+0000 7f0c51a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4c0741f0 0x7f0c4c1f6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:49.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.197+0000 7f0c51a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c4c074840 0x7f0c4c1e5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:49.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.197+0000 7f0c51a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c4c1547d0 0x7f0c4c156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:49.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.197+0000 7f0c51a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c073790 con 0x7f0c4c074840 2024-05-27T09:33:49.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.197+0000 7f0c51a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c0734b0 con 0x7f0c4c0741f0 2024-05-27T09:33:49.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.197+0000 7f0c51a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f0c4c06f500 con 0x7f0c4c1547d0 2024-05-27T09:33:49.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.198+0000 7f0c4be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4c0741f0 0x7f0c4c1f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:49.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.198+0000 7f0c4b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c4c074840 0x7f0c4c1e5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:49.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.198+0000 7f0c51000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c4c1547d0 0x7f0c4c156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:49.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.198+0000 7f0c4be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4c0741f0 0x7f0c4c1f6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:39844/0 (socket says 172.21.15.114:39844) 2024-05-27T09:33:49.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.198+0000 7f0c4be00640 1 -- 172.21.15.114:0/2750794160 learned_addr learned my addr 172.21.15.114:0/2750794160 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:33:49.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.198+0000 7f0c51000640 1 -- 172.21.15.114:0/2750794160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4c0741f0 msgr2=0x7f0c4c1f6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:49.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.198+0000 7f0c51000640 1 --2- 172.21.15.114:0/2750794160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4c0741f0 0x7f0c4c1f6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:49.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.198+0000 7f0c51000640 1 -- 172.21.15.114:0/2750794160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c4c074840 msgr2=0x7f0c4c1e5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:49.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.198+0000 7f0c51000640 1 --2- 172.21.15.114:0/2750794160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c4c074840 0x7f0c4c1e5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:49.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.198+0000 7f0c51000640 1 -- 172.21.15.114:0/2750794160 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4c152c60 con 0x7f0c4c1547d0 2024-05-27T09:33:49.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.198+0000 7f0c4be00640 1 --2- 172.21.15.114:0/2750794160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4c0741f0 0x7f0c4c1f6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:33:49.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.199+0000 7f0c51000640 1 --2- 172.21.15.114:0/2750794160 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c4c1547d0 0x7f0c4c156b00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400ef90 tx=0x7f0c4400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:49.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.199+0000 7f0c43e00640 1 -- 172.21.15.114:0/2750794160 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c44018030 con 0x7f0c4c1547d0 2024-05-27T09:33:49.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.199+0000 7f0c43e00640 1 -- 172.21.15.114:0/2750794160 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c4400cd30 con 0x7f0c4c1547d0 2024-05-27T09:33:49.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.199+0000 7f0c43e00640 1 -- 172.21.15.114:0/2750794160 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c44003bb0 con 0x7f0c4c1547d0 2024-05-27T09:33:49.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.199+0000 7f0c51a00640 1 -- 172.21.15.114:0/2750794160 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4c0c82c0 con 0x7f0c4c1547d0 2024-05-27T09:33:49.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.199+0000 7f0c51a00640 1 -- 172.21.15.114:0/2750794160 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c4c1f5a90 con 0x7f0c4c1547d0 2024-05-27T09:33:49.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.200+0000 7f0c51a00640 1 -- 172.21.15.114:0/2750794160 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c4c1592f0 con 0x7f0c4c1547d0 2024-05-27T09:33:49.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.201+0000 7f0c43e00640 1 -- 172.21.15.114:0/2750794160 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0c44005d10 con 0x7f0c4c1547d0 2024-05-27T09:33:49.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.202+0000 7f0c43e00640 1 --2- 172.21.15.114:0/2750794160 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0c2c0762d0 0x7f0c2c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:49.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.202+0000 7f0c43e00640 1 -- 172.21.15.114:0/2750794160 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(47..47 src has 1..47) ==== 6024+0+0 (secure 0 0 0) 0x7f0c44097f60 con 0x7f0c4c1547d0 2024-05-27T09:33:49.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.202+0000 7f0c4be00640 1 --2- 172.21.15.114:0/2750794160 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0c2c0762d0 0x7f0c2c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:49.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.203+0000 7f0c4be00640 1 --2- 172.21.15.114:0/2750794160 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0c2c0762d0 0x7f0c2c0787e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c0090b0 tx=0x7f0c3c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:49.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.207+0000 7f0c43e00640 1 -- 172.21.15.114:0/2750794160 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0c44022070 con 0x7f0c4c1547d0 2024-05-27T09:33:49.207 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:33:49.212 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:33:49.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.471+0000 7f0c51a00640 1 -- 172.21.15.114:0/2750794160 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c4c158360 con 0x7f0c4c1547d0 2024-05-27T09:33:49.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.472+0000 7f0c43e00640 1 -- 172.21.15.114:0/2750794160 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+6888 (secure 0 0 0) 0x7f0c44010040 con 0x7f0c4c1547d0 2024-05-27T09:33:49.473 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:33:49.473 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:32:51.967298+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4875,4828,4300]},"id":1}]} 2024-05-27T09:33:49.473 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 30 2024-05-27T09:33:49.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.475+0000 7f0c51a00640 1 -- 172.21.15.114:0/2750794160 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0c2c0762d0 msgr2=0x7f0c2c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:49.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.475+0000 7f0c51a00640 1 --2- 172.21.15.114:0/2750794160 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0c2c0762d0 0x7f0c2c0787e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c0090b0 tx=0x7f0c3c02e000 comp rx=0 tx=0).stop 2024-05-27T09:33:49.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.475+0000 7f0c51a00640 1 -- 172.21.15.114:0/2750794160 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c4c1547d0 msgr2=0x7f0c4c156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:49.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.475+0000 7f0c51a00640 1 --2- 172.21.15.114:0/2750794160 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c4c1547d0 0x7f0c4c156b00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400ef90 tx=0x7f0c4400c490 comp rx=0 tx=0).stop 2024-05-27T09:33:49.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.475+0000 7f0c51a00640 1 -- 172.21.15.114:0/2750794160 shutdown_connections 2024-05-27T09:33:49.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.475+0000 7f0c51a00640 1 --2- 172.21.15.114:0/2750794160 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0c2c0762d0 0x7f0c2c0787e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:49.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.475+0000 7f0c51a00640 1 --2- 172.21.15.114:0/2750794160 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c4c1547d0 0x7f0c4c156b00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:49.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.475+0000 7f0c51a00640 1 --2- 172.21.15.114:0/2750794160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c4c074840 0x7f0c4c1e5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:49.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.475+0000 7f0c51a00640 1 --2- 172.21.15.114:0/2750794160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4c0741f0 0x7f0c4c1f6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:49.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.475+0000 7f0c51a00640 1 -- 172.21.15.114:0/2750794160 >> 172.21.15.114:0/2750794160 conn(0x7f0c4c0ca320 msgr2=0x7f0c4c06edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:49.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.475+0000 7f0c51a00640 1 -- 172.21.15.114:0/2750794160 shutdown_connections 2024-05-27T09:33:49.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.476+0000 7f0c51a00640 1 -- 172.21.15.114:0/2750794160 wait complete. 2024-05-27T09:33:49.506 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=0) 2024-05-27T09:33:49.506 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-05-27T09:33:49.506 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:33:49.558 INFO:tasks.ceph.mds.e:Stopped 2024-05-27T09:33:49.558 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:33:49.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.810+0000 7fcc0b200640 1 Processor -- start 2024-05-27T09:33:49.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.810+0000 7fcc0b200640 1 -- start start 2024-05-27T09:33:49.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.813+0000 7fcc0b200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc0c0741f0 0x7fcc0c0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:49.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.813+0000 7fcc0b200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fcc0c074840 0x7fcc0c15ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:49.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.813+0000 7fcc0b200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc0c06fe50 con 0x7fcc0c1547d0 2024-05-27T09:33:49.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.813+0000 7fcc0b200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fcc0c0703d0 con 0x7fcc0c074840 2024-05-27T09:33:49.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.813+0000 7fcc0b200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fcc0c15b260 con 0x7fcc0c0741f0 2024-05-27T09:33:49.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.814+0000 7fcc09400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fcc0c1547d0 0x7fcc0c06edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:50040/0 (socket says 172.21.15.114:50040) 2024-05-27T09:33:49.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.814+0000 7fcc09400640 1 -- 172.21.15.114:0/2233476633 learned_addr learned my addr 172.21.15.114:0/2233476633 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:33:49.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.814+0000 7fcc09e00640 1 --2- 172.21.15.114:0/2233476633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc0c0741f0 0x7fcc0c0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:49.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.814+0000 7fcc0a800640 1 --2- 172.21.15.114:0/2233476633 >> v2:172.21.15.114:3300/0 conn(0x7fcc0c074840 0x7fcc0c15ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:49.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.814+0000 7fcc08a00640 1 -- 172.21.15.114:0/2233476633 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 374833166 0 0) 0x7fcc0c06fe50 con 0x7fcc0c1547d0 2024-05-27T09:33:49.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.814+0000 7fcc08a00640 1 -- 172.21.15.114:0/2233476633 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcbf0003520 con 0x7fcc0c1547d0 2024-05-27T09:33:49.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.814+0000 7fcc0a800640 1 -- 172.21.15.114:0/2233476633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc0c0741f0 msgr2=0x7fcc0c0c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:33:49.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.814+0000 7fcc0a800640 1 --2- 172.21.15.114:0/2233476633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc0c0741f0 0x7fcc0c0c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:49.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.814+0000 7fcc0a800640 1 -- 172.21.15.114:0/2233476633 >> v1:172.21.15.114:6789/0 conn(0x7fcc0c1547d0 legacy=0x7fcc0c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:49.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.814+0000 7fcc0a800640 1 -- 172.21.15.114:0/2233476633 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0c0700f0 con 0x7fcc0c074840 2024-05-27T09:33:49.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.814+0000 7fcc09e00640 1 --2- 172.21.15.114:0/2233476633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc0c0741f0 0x7fcc0c0c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:33:49.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.815+0000 7fcc0a800640 1 --2- 172.21.15.114:0/2233476633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc0c074840 0x7fcc0c15ad20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009d90 tx=0x7fcc0402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fd5d96ea22116e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:49.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.816+0000 7fcc08a00640 1 -- 172.21.15.114:0/2233476633 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04007740 con 0x7fcc0c074840 2024-05-27T09:33:49.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.816+0000 7fcc08a00640 1 -- 172.21.15.114:0/2233476633 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc040078e0 con 0x7fcc0c074840 2024-05-27T09:33:49.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.816+0000 7fcc08a00640 1 -- 172.21.15.114:0/2233476633 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04007cb0 con 0x7fcc0c074840 2024-05-27T09:33:49.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.816+0000 7fcc0b200640 1 -- 172.21.15.114:0/2233476633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc0c074840 msgr2=0x7fcc0c15ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:49.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.816+0000 7fcc0b200640 1 --2- 172.21.15.114:0/2233476633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc0c074840 0x7fcc0c15ad20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009d90 tx=0x7fcc0402a6f0 comp rx=0 tx=0).stop 2024-05-27T09:33:49.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.816+0000 7fcc0b200640 1 -- 172.21.15.114:0/2233476633 shutdown_connections 2024-05-27T09:33:49.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.816+0000 7fcc0b200640 1 --2- 172.21.15.114:0/2233476633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc0c074840 0x7fcc0c15ad20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:49.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.816+0000 7fcc0b200640 1 --2- 172.21.15.114:0/2233476633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc0c0741f0 0x7fcc0c0c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:49.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.816+0000 7fcc0b200640 1 -- 172.21.15.114:0/2233476633 >> 172.21.15.114:0/2233476633 conn(0x7fcc0c0ca320 msgr2=0x7fcc0c0ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:49.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.817+0000 7fcc0b200640 1 -- 172.21.15.114:0/2233476633 shutdown_connections 2024-05-27T09:33:49.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.817+0000 7fcc0b200640 1 -- 172.21.15.114:0/2233476633 wait complete. 2024-05-27T09:33:49.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.819+0000 7fcc0b200640 1 Processor -- start 2024-05-27T09:33:49.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.819+0000 7fcc0b200640 1 -- start start 2024-05-27T09:33:49.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.820+0000 7fcc0b200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc0c0741f0 0x7fcc0c0c2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:49.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.820+0000 7fcc0b200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc0c074840 0x7fcc0c1572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:49.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.820+0000 7fcc0b200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc0c1547d0 0x7fcc0c1577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:49.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.820+0000 7fcc0b200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c073790 con 0x7fcc0c1547d0 2024-05-27T09:33:49.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.820+0000 7fcc0b200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c0734b0 con 0x7fcc0c074840 2024-05-27T09:33:49.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.820+0000 7fcc0b200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fcc0c06f500 con 0x7fcc0c0741f0 2024-05-27T09:33:49.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.821+0000 7fcc0a800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc0c1547d0 0x7fcc0c1577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:49.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.821+0000 7fcc0a800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc0c1547d0 0x7fcc0c1577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59000/0 (socket says 172.21.15.114:59000) 2024-05-27T09:33:49.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.821+0000 7fcc0a800640 1 -- 172.21.15.114:0/819978714 learned_addr learned my addr 172.21.15.114:0/819978714 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:33:49.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.821+0000 7fcc09400640 1 --2- 172.21.15.114:0/819978714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc0c074840 0x7fcc0c1572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:49.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.821+0000 7fcc09e00640 1 --2- 172.21.15.114:0/819978714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc0c0741f0 0x7fcc0c0c2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:49.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.821+0000 7fcc0a800640 1 -- 172.21.15.114:0/819978714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc0c0741f0 msgr2=0x7fcc0c0c2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:49.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.821+0000 7fcc0a800640 1 --2- 172.21.15.114:0/819978714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc0c0741f0 0x7fcc0c0c2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:49.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.821+0000 7fcc0a800640 1 -- 172.21.15.114:0/819978714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc0c074840 msgr2=0x7fcc0c1572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:49.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.821+0000 7fcc0a800640 1 --2- 172.21.15.114:0/819978714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc0c074840 0x7fcc0c1572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:49.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.821+0000 7fcc0a800640 1 -- 172.21.15.114:0/819978714 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0c0c2f80 con 0x7fcc0c1547d0 2024-05-27T09:33:49.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.822+0000 7fcc0a800640 1 --2- 172.21.15.114:0/819978714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc0c1547d0 0x7fcc0c1577f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009eb0 tx=0x7fcc04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:49.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.822+0000 7fcc02000640 1 -- 172.21.15.114:0/819978714 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0403f070 con 0x7fcc0c1547d0 2024-05-27T09:33:49.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.822+0000 7fcc02000640 1 -- 172.21.15.114:0/819978714 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc04031bc0 con 0x7fcc0c1547d0 2024-05-27T09:33:49.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.822+0000 7fcc0b200640 1 -- 172.21.15.114:0/819978714 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc0c0c75f0 con 0x7fcc0c1547d0 2024-05-27T09:33:49.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.822+0000 7fcc02000640 1 -- 172.21.15.114:0/819978714 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04009450 con 0x7fcc0c1547d0 2024-05-27T09:33:49.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.822+0000 7fcc0b200640 1 -- 172.21.15.114:0/819978714 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc0c0c7310 con 0x7fcc0c1547d0 2024-05-27T09:33:49.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.823+0000 7fcc0b200640 1 -- 172.21.15.114:0/819978714 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc0c155e80 con 0x7fcc0c1547d0 2024-05-27T09:33:49.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.826+0000 7fcc02000640 1 -- 172.21.15.114:0/819978714 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcc0403b410 con 0x7fcc0c1547d0 2024-05-27T09:33:49.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.827+0000 7fcc02000640 1 --2- 172.21.15.114:0/819978714 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcbec0762d0 0x7fcbec0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:49.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.827+0000 7fcc02000640 1 -- 172.21.15.114:0/819978714 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6024+0+0 (secure 0 0 0) 0x7fcc040b89e0 con 0x7fcc0c1547d0 2024-05-27T09:33:49.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.827+0000 7fcc09e00640 1 --2- 172.21.15.114:0/819978714 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcbec0762d0 0x7fcbec0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:49.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.828+0000 7fcc09e00640 1 --2- 172.21.15.114:0/819978714 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcbec0762d0 0x7fcbec0787e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009560 tx=0x7fcbfc009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:49.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:49.833+0000 7fcc02000640 1 -- 172.21.15.114:0/819978714 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcc04081ba0 con 0x7fcc0c1547d0 2024-05-27T09:33:50.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:50.152+0000 7fcc0b200640 1 -- 172.21.15.114:0/819978714 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc0c076740 con 0x7fcc0c1547d0 2024-05-27T09:33:50.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:50.154+0000 7fcc02000640 1 -- 172.21.15.114:0/819978714 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+6888 (secure 0 0 0) 0x7fcc04085ea0 con 0x7fcc0c1547d0 2024-05-27T09:33:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:33:50.154 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:32:51.967298+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4875,4828,4300]},"id":1}]} 2024-05-27T09:33:50.154 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 30 2024-05-27T09:33:50.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:50.157+0000 7fcc0b200640 1 -- 172.21.15.114:0/819978714 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcbec0762d0 msgr2=0x7fcbec0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:50.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:50.157+0000 7fcc0b200640 1 --2- 172.21.15.114:0/819978714 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcbec0762d0 0x7fcbec0787e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009560 tx=0x7fcbfc009190 comp rx=0 tx=0).stop 2024-05-27T09:33:50.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:50.157+0000 7fcc0b200640 1 -- 172.21.15.114:0/819978714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc0c1547d0 msgr2=0x7fcc0c1577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:50.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:50.157+0000 7fcc0b200640 1 --2- 172.21.15.114:0/819978714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc0c1547d0 0x7fcc0c1577f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009eb0 tx=0x7fcc04004a40 comp rx=0 tx=0).stop 2024-05-27T09:33:50.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:50.157+0000 7fcc0b200640 1 -- 172.21.15.114:0/819978714 shutdown_connections 2024-05-27T09:33:50.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:50.157+0000 7fcc0b200640 1 --2- 172.21.15.114:0/819978714 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcbec0762d0 0x7fcbec0787e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:50.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:50.157+0000 7fcc0b200640 1 --2- 172.21.15.114:0/819978714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc0c1547d0 0x7fcc0c1577f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:50.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:50.157+0000 7fcc0b200640 1 --2- 172.21.15.114:0/819978714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc0c074840 0x7fcc0c1572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:50.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:50.157+0000 7fcc0b200640 1 --2- 172.21.15.114:0/819978714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc0c0741f0 0x7fcc0c0c2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:50.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:50.157+0000 7fcc0b200640 1 -- 172.21.15.114:0/819978714 >> 172.21.15.114:0/819978714 conn(0x7fcc0c0ca320 msgr2=0x7fcc0c151590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:50.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:50.157+0000 7fcc0b200640 1 -- 172.21.15.114:0/819978714 shutdown_connections 2024-05-27T09:33:50.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:50.157+0000 7fcc0b200640 1 -- 172.21.15.114:0/819978714 wait complete. 2024-05-27T09:33:50.184 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-05-27T09:33:52.185 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:33:52.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.458+0000 7f248e000640 1 Processor -- start 2024-05-27T09:33:52.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.458+0000 7f248e000640 1 -- start start 2024-05-27T09:33:52.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.459+0000 7f248e000640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2488156980 0x7f2488160330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:52.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.459+0000 7f248e000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2488160870 0x7f24880cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:52.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.459+0000 7f248e000640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f248806e1c0 con 0x7f24880cc890 2024-05-27T09:33:52.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.459+0000 7f248e000640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f248806ed40 con 0x7f2488156980 2024-05-27T09:33:52.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.459+0000 7f248e000640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f248806e4a0 con 0x7f2488160870 2024-05-27T09:33:52.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.460+0000 7f248cc00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2488156980 0x7f2488160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:52.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.460+0000 7f248d600640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f24880cc890 0x7f24880cb740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:50046/0 (socket says 172.21.15.114:50046) 2024-05-27T09:33:52.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.460+0000 7f248d600640 1 -- 172.21.15.114:0/4117769099 learned_addr learned my addr 172.21.15.114:0/4117769099 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:33:52.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.460+0000 7f2487e00640 1 --2- 172.21.15.114:0/4117769099 >> v2:172.21.15.163:3300/0 conn(0x7f2488160870 0x7f24880cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:52.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.460+0000 7f248cc00640 1 -- 172.21.15.114:0/4117769099 >> v2:172.21.15.163:3300/0 conn(0x7f2488160870 msgr2=0x7f24880cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:33:52.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.460+0000 7f248cc00640 1 --2- 172.21.15.114:0/4117769099 >> v2:172.21.15.163:3300/0 conn(0x7f2488160870 0x7f24880cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:52.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.460+0000 7f248cc00640 1 -- 172.21.15.114:0/4117769099 >> v1:172.21.15.114:6789/0 conn(0x7f24880cc890 legacy=0x7f24880cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:52.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.460+0000 7f248cc00640 1 -- 172.21.15.114:0/4117769099 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248806e780 con 0x7f2488156980 2024-05-27T09:33:52.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.461+0000 7f248cc00640 1 --2- 172.21.15.114:0/4117769099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2488156980 0x7f2488160330 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f24880b2810 tx=0x7f247002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77918e350bd9fd36 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:52.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.462+0000 7f2487400640 1 -- 172.21.15.114:0/4117769099 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470037070 con 0x7f2488156980 2024-05-27T09:33:52.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.462+0000 7f2487400640 1 -- 172.21.15.114:0/4117769099 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2470032440 con 0x7f2488156980 2024-05-27T09:33:52.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.462+0000 7f2487400640 1 -- 172.21.15.114:0/4117769099 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247002ae60 con 0x7f2488156980 2024-05-27T09:33:52.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.462+0000 7f248e000640 1 -- 172.21.15.114:0/4117769099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2488156980 msgr2=0x7f2488160330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:52.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.462+0000 7f248e000640 1 --2- 172.21.15.114:0/4117769099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2488156980 0x7f2488160330 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f24880b2810 tx=0x7f247002a110 comp rx=0 tx=0).stop 2024-05-27T09:33:52.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.463+0000 7f248e000640 1 -- 172.21.15.114:0/4117769099 shutdown_connections 2024-05-27T09:33:52.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.463+0000 7f248e000640 1 --2- 172.21.15.114:0/4117769099 >> v2:172.21.15.163:3300/0 conn(0x7f2488160870 0x7f24880cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:52.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.463+0000 7f248e000640 1 --2- 172.21.15.114:0/4117769099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2488156980 0x7f2488160330 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:52.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.463+0000 7f248e000640 1 -- 172.21.15.114:0/4117769099 >> 172.21.15.114:0/4117769099 conn(0x7f2488071460 msgr2=0x7f2488068600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:52.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.463+0000 7f248e000640 1 -- 172.21.15.114:0/4117769099 shutdown_connections 2024-05-27T09:33:52.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.463+0000 7f248e000640 1 -- 172.21.15.114:0/4117769099 wait complete. 2024-05-27T09:33:52.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.465+0000 7f248e000640 1 Processor -- start 2024-05-27T09:33:52.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.465+0000 7f248e000640 1 -- start start 2024-05-27T09:33:52.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.466+0000 7f248e000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24880cc890 0x7f248815de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:52.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.466+0000 7f248e000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2488156980 0x7f24881529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:52.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.466+0000 7f248e000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2488160870 0x7f24881f6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:52.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.466+0000 7f248e000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2488156d90 con 0x7f2488156980 2024-05-27T09:33:52.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.466+0000 7f248e000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f248815fe10 con 0x7f2488160870 2024-05-27T09:33:52.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.466+0000 7f248e000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f2488156f10 con 0x7f24880cc890 2024-05-27T09:33:52.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.467+0000 7f2487e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2488156980 0x7f24881529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:52.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.467+0000 7f2487e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2488156980 0x7f24881529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59012/0 (socket says 172.21.15.114:59012) 2024-05-27T09:33:52.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.467+0000 7f2487e00640 1 -- 172.21.15.114:0/3093593737 learned_addr learned my addr 172.21.15.114:0/3093593737 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:33:52.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.467+0000 7f248cc00640 1 --2- 172.21.15.114:0/3093593737 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24880cc890 0x7f248815de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:52.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.467+0000 7f248d600640 1 --2- 172.21.15.114:0/3093593737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2488160870 0x7f24881f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:52.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.467+0000 7f2487e00640 1 -- 172.21.15.114:0/3093593737 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24880cc890 msgr2=0x7f248815de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:52.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.467+0000 7f2487e00640 1 --2- 172.21.15.114:0/3093593737 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24880cc890 0x7f248815de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:52.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.467+0000 7f2487e00640 1 -- 172.21.15.114:0/3093593737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2488160870 msgr2=0x7f24881f6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:52.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.467+0000 7f2487e00640 1 --2- 172.21.15.114:0/3093593737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2488160870 0x7f24881f6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:52.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.467+0000 7f2487e00640 1 -- 172.21.15.114:0/3093593737 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248815e3d0 con 0x7f2488156980 2024-05-27T09:33:52.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.467+0000 7f2487e00640 1 --2- 172.21.15.114:0/3093593737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2488156980 0x7f24881529d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2478009c90 tx=0x7f247800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:52.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.468+0000 7f2484c00640 1 -- 172.21.15.114:0/3093593737 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478018020 con 0x7f2488156980 2024-05-27T09:33:52.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.468+0000 7f2484c00640 1 -- 172.21.15.114:0/3093593737 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2478014480 con 0x7f2488156980 2024-05-27T09:33:52.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.468+0000 7f2484c00640 1 -- 172.21.15.114:0/3093593737 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478014830 con 0x7f2488156980 2024-05-27T09:33:52.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.468+0000 7f248e000640 1 -- 172.21.15.114:0/3093593737 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24880b2f60 con 0x7f2488156980 2024-05-27T09:33:52.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.468+0000 7f248e000640 1 -- 172.21.15.114:0/3093593737 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24880b3520 con 0x7f2488156980 2024-05-27T09:33:52.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.469+0000 7f248e000640 1 -- 172.21.15.114:0/3093593737 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f248806d5d0 con 0x7f2488156980 2024-05-27T09:33:52.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.470+0000 7f2484c00640 1 -- 172.21.15.114:0/3093593737 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2478005d10 con 0x7f2488156980 2024-05-27T09:33:52.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.471+0000 7f2484c00640 1 --2- 172.21.15.114:0/3093593737 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f24640762d0 0x7f24640787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:52.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.471+0000 7f2484c00640 1 -- 172.21.15.114:0/3093593737 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6024+0+0 (secure 0 0 0) 0x7f24780983c0 con 0x7f2488156980 2024-05-27T09:33:52.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.471+0000 7f248cc00640 1 --2- 172.21.15.114:0/3093593737 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f24640762d0 0x7f24640787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:52.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.472+0000 7f248cc00640 1 --2- 172.21.15.114:0/3093593737 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f24640762d0 0x7f24640787e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2470034000 tx=0x7f247002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:52.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.476+0000 7f2484c00640 1 -- 172.21.15.114:0/3093593737 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2478022030 con 0x7f2488156980 2024-05-27T09:33:52.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.709+0000 7f248e000640 1 -- 172.21.15.114:0/3093593737 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2488050860 con 0x7f2488156980 2024-05-27T09:33:52.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.710+0000 7f2484c00640 1 -- 172.21.15.114:0/3093593737 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+6888 (secure 0 0 0) 0x7f2478010040 con 0x7f2488156980 2024-05-27T09:33:52.711 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:33:52.711 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:32:51.967298+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4875,4828,4300]},"id":1}]} 2024-05-27T09:33:52.711 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 30 2024-05-27T09:33:52.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.713+0000 7f248e000640 1 -- 172.21.15.114:0/3093593737 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f24640762d0 msgr2=0x7f24640787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:52.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.713+0000 7f248e000640 1 --2- 172.21.15.114:0/3093593737 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f24640762d0 0x7f24640787e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2470034000 tx=0x7f247002e000 comp rx=0 tx=0).stop 2024-05-27T09:33:52.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.713+0000 7f248e000640 1 -- 172.21.15.114:0/3093593737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2488156980 msgr2=0x7f24881529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:52.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.713+0000 7f248e000640 1 --2- 172.21.15.114:0/3093593737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2488156980 0x7f24881529d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2478009c90 tx=0x7f247800c490 comp rx=0 tx=0).stop 2024-05-27T09:33:52.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.713+0000 7f248e000640 1 -- 172.21.15.114:0/3093593737 shutdown_connections 2024-05-27T09:33:52.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.713+0000 7f248e000640 1 --2- 172.21.15.114:0/3093593737 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f24640762d0 0x7f24640787e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:52.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.713+0000 7f248e000640 1 --2- 172.21.15.114:0/3093593737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2488160870 0x7f24881f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:52.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.713+0000 7f248e000640 1 --2- 172.21.15.114:0/3093593737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2488156980 0x7f24881529d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:52.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.713+0000 7f248e000640 1 --2- 172.21.15.114:0/3093593737 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24880cc890 0x7f248815de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:52.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.713+0000 7f248e000640 1 -- 172.21.15.114:0/3093593737 >> 172.21.15.114:0/3093593737 conn(0x7f2488071460 msgr2=0x7f2488072d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:52.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.713+0000 7f248e000640 1 -- 172.21.15.114:0/3093593737 shutdown_connections 2024-05-27T09:33:52.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:52.714+0000 7f248e000640 1 -- 172.21.15.114:0/3093593737 wait complete. 2024-05-27T09:33:52.737 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-05-27T09:33:54.739 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:33:55.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.025+0000 7fbb21000640 1 Processor -- start 2024-05-27T09:33:55.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.025+0000 7fbb21000640 1 -- start start 2024-05-27T09:33:55.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.027+0000 7fbb21000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbb1c1593b0 0x7fbb1c142720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:55.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.027+0000 7fbb21000640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbb1c073260 0x7fbb1c15a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:55.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.027+0000 7fbb21000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb1c15ade0 0x7fbb1c0c2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:55.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.027+0000 7fbb21000640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fbb1c074d20 con 0x7fbb1c073260 2024-05-27T09:33:55.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.027+0000 7fbb21000640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fbb1c0739a0 con 0x7fbb1c1593b0 2024-05-27T09:33:55.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.027+0000 7fbb21000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fbb1c073c80 con 0x7fbb1c15ade0 2024-05-27T09:33:55.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.027+0000 7fbb19400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbb1c073260 0x7fbb1c15a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:55.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.027+0000 7fbb1a800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb1c15ade0 0x7fbb1c0c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:55.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.027+0000 7fbb19e00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbb1c1593b0 0x7fbb1c142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:55.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.027+0000 7fbb19400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbb1c073260 0x7fbb1c15a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42818/0 (socket says 172.21.15.114:42818) 2024-05-27T09:33:55.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.028+0000 7fbb19400640 1 -- 172.21.15.114:0/3387893815 learned_addr learned my addr 172.21.15.114:0/3387893815 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:33:55.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.028+0000 7fbb1a800640 1 -- 172.21.15.114:0/3387893815 >> v2:172.21.15.163:3300/0 conn(0x7fbb1c1593b0 msgr2=0x7fbb1c142720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:33:55.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.028+0000 7fbb1a800640 1 --2- 172.21.15.114:0/3387893815 >> v2:172.21.15.163:3300/0 conn(0x7fbb1c1593b0 0x7fbb1c142720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:55.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.028+0000 7fbb1a800640 1 -- 172.21.15.114:0/3387893815 >> v2:172.21.15.114:3300/0 conn(0x7fbb1c073260 msgr2=0x7fbb1c15a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:33:55.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.028+0000 7fbb1a800640 1 --2- 172.21.15.114:0/3387893815 >> v2:172.21.15.114:3300/0 conn(0x7fbb1c073260 0x7fbb1c15a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:55.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.028+0000 7fbb1a800640 1 -- 172.21.15.114:0/3387893815 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1c073f60 con 0x7fbb1c15ade0 2024-05-27T09:33:55.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.029+0000 7fbb1a800640 1 --2- 172.21.15.114:0/3387893815 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb1c15ade0 0x7fbb1c0c2990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0cb530 tx=0x7fbb1402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=429f93e2ead0bd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:55.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.029+0000 7fbb18a00640 1 -- 172.21.15.114:0/3387893815 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14007740 con 0x7fbb1c15ade0 2024-05-27T09:33:55.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.029+0000 7fbb18a00640 1 -- 172.21.15.114:0/3387893815 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb140078e0 con 0x7fbb1c15ade0 2024-05-27T09:33:55.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.029+0000 7fbb18a00640 1 -- 172.21.15.114:0/3387893815 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14007cb0 con 0x7fbb1c15ade0 2024-05-27T09:33:55.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.030+0000 7fbb21000640 1 -- 172.21.15.114:0/3387893815 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb1c15ade0 msgr2=0x7fbb1c0c2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:55.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.030+0000 7fbb21000640 1 --2- 172.21.15.114:0/3387893815 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb1c15ade0 0x7fbb1c0c2990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0cb530 tx=0x7fbb1402a660 comp rx=0 tx=0).stop 2024-05-27T09:33:55.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.030+0000 7fbb21000640 1 -- 172.21.15.114:0/3387893815 shutdown_connections 2024-05-27T09:33:55.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.030+0000 7fbb21000640 1 --2- 172.21.15.114:0/3387893815 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb1c15ade0 0x7fbb1c0c2990 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:55.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.030+0000 7fbb21000640 1 --2- 172.21.15.114:0/3387893815 >> v2:172.21.15.114:3300/0 conn(0x7fbb1c073260 0x7fbb1c15a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:55.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.030+0000 7fbb21000640 1 --2- 172.21.15.114:0/3387893815 >> v2:172.21.15.163:3300/0 conn(0x7fbb1c1593b0 0x7fbb1c142720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:55.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.030+0000 7fbb21000640 1 -- 172.21.15.114:0/3387893815 >> 172.21.15.114:0/3387893815 conn(0x7fbb1c06f6e0 msgr2=0x7fbb1c070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:55.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.030+0000 7fbb21000640 1 -- 172.21.15.114:0/3387893815 shutdown_connections 2024-05-27T09:33:55.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.030+0000 7fbb21000640 1 -- 172.21.15.114:0/3387893815 wait complete. 2024-05-27T09:33:55.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.032+0000 7fbb21000640 1 Processor -- start 2024-05-27T09:33:55.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.033+0000 7fbb21000640 1 -- start start 2024-05-27T09:33:55.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.034+0000 7fbb21000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb1c073260 0x7fbb1c0c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:55.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.034+0000 7fbb21000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb1c1593b0 0x7fbb1c154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:55.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.034+0000 7fbb21000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb1c15ade0 0x7fbb1c1f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:55.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.034+0000 7fbb21000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c074800 con 0x7fbb1c15ade0 2024-05-27T09:33:55.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.034+0000 7fbb21000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c074520 con 0x7fbb1c073260 2024-05-27T09:33:55.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.034+0000 7fbb21000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fbb1c073670 con 0x7fbb1c1593b0 2024-05-27T09:33:55.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.034+0000 7fbb1a800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb1c15ade0 0x7fbb1c1f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:55.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.034+0000 7fbb19e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb1c073260 0x7fbb1c0c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:55.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.034+0000 7fbb19400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb1c1593b0 0x7fbb1c154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:55.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.034+0000 7fbb19e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb1c073260 0x7fbb1c0c9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:59554/0 (socket says 172.21.15.114:59554) 2024-05-27T09:33:55.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.034+0000 7fbb1a800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb1c15ade0 0x7fbb1c1f6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42832/0 (socket says 172.21.15.114:42832) 2024-05-27T09:33:55.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.034+0000 7fbb19e00640 1 -- 172.21.15.114:0/144226697 learned_addr learned my addr 172.21.15.114:0/144226697 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:33:55.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.034+0000 7fbb19400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb1c1593b0 0x7fbb1c154090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:47352/0 (socket says 172.21.15.114:47352) 2024-05-27T09:33:55.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.035+0000 7fbb19400640 1 -- 172.21.15.114:0/144226697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb1c073260 msgr2=0x7fbb1c0c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:55.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.035+0000 7fbb19400640 1 --2- 172.21.15.114:0/144226697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb1c073260 0x7fbb1c0c9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:55.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.035+0000 7fbb19400640 1 -- 172.21.15.114:0/144226697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb1c15ade0 msgr2=0x7fbb1c1f6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:55.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.035+0000 7fbb19400640 1 --2- 172.21.15.114:0/144226697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb1c15ade0 0x7fbb1c1f6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:55.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.035+0000 7fbb19400640 1 -- 172.21.15.114:0/144226697 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1c0c9950 con 0x7fbb1c1593b0 2024-05-27T09:33:55.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.035+0000 7fbb19e00640 1 --2- 172.21.15.114:0/144226697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb1c073260 0x7fbb1c0c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:33:55.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.035+0000 7fbb19400640 1 --2- 172.21.15.114:0/144226697 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb1c1593b0 0x7fbb1c154090 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800e9d0 tx=0x7fbb0800ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:55.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.036+0000 7fbb12000640 1 -- 172.21.15.114:0/144226697 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0800de00 con 0x7fbb1c1593b0 2024-05-27T09:33:55.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.036+0000 7fbb12000640 1 -- 172.21.15.114:0/144226697 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb08019480 con 0x7fbb1c1593b0 2024-05-27T09:33:55.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.036+0000 7fbb12000640 1 -- 172.21.15.114:0/144226697 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08019830 con 0x7fbb1c1593b0 2024-05-27T09:33:55.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.036+0000 7fbb21000640 1 -- 172.21.15.114:0/144226697 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb1c0c8b30 con 0x7fbb1c1593b0 2024-05-27T09:33:55.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.036+0000 7fbb21000640 1 -- 172.21.15.114:0/144226697 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb1c15a5e0 con 0x7fbb1c1593b0 2024-05-27T09:33:55.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.037+0000 7fbb21000640 1 -- 172.21.15.114:0/144226697 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb1c0692a0 con 0x7fbb1c1593b0 2024-05-27T09:33:55.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.038+0000 7fbb12000640 1 -- 172.21.15.114:0/144226697 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbb08010440 con 0x7fbb1c1593b0 2024-05-27T09:33:55.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.039+0000 7fbb12000640 1 --2- 172.21.15.114:0/144226697 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbafc076300 0x7fbafc078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:55.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.039+0000 7fbb12000640 1 -- 172.21.15.114:0/144226697 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(47..47 src has 1..47) ==== 6024+0+0 (secure 0 0 0) 0x7fbb08098070 con 0x7fbb1c1593b0 2024-05-27T09:33:55.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.039+0000 7fbb19e00640 1 --2- 172.21.15.114:0/144226697 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbafc076300 0x7fbafc078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:55.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.040+0000 7fbb19e00640 1 --2- 172.21.15.114:0/144226697 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbafc076300 0x7fbafc078810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009ed0 tx=0x7fbb0c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:55.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.044+0000 7fbb12000640 1 -- 172.21.15.114:0/144226697 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbb1c0692a0 con 0x7fbb1c1593b0 2024-05-27T09:33:55.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.275+0000 7fbb21000640 1 -- 172.21.15.114:0/144226697 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb1c06a380 con 0x7fbb1c1593b0 2024-05-27T09:33:55.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.276+0000 7fbb12000640 1 -- 172.21.15.114:0/144226697 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+6888 (secure 0 0 0) 0x7fbb08061230 con 0x7fbb1c1593b0 2024-05-27T09:33:55.277 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:33:55.277 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:32:51.967298+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4875,4828,4300]},"id":1}]} 2024-05-27T09:33:55.277 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 30 2024-05-27T09:33:55.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.279+0000 7fbb21000640 1 -- 172.21.15.114:0/144226697 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbafc076300 msgr2=0x7fbafc078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:55.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.279+0000 7fbb21000640 1 --2- 172.21.15.114:0/144226697 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbafc076300 0x7fbafc078810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009ed0 tx=0x7fbb0c009190 comp rx=0 tx=0).stop 2024-05-27T09:33:55.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.279+0000 7fbb21000640 1 -- 172.21.15.114:0/144226697 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb1c1593b0 msgr2=0x7fbb1c154090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:55.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.279+0000 7fbb21000640 1 --2- 172.21.15.114:0/144226697 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb1c1593b0 0x7fbb1c154090 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800e9d0 tx=0x7fbb0800ee90 comp rx=0 tx=0).stop 2024-05-27T09:33:55.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.279+0000 7fbb21000640 1 -- 172.21.15.114:0/144226697 shutdown_connections 2024-05-27T09:33:55.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.279+0000 7fbb21000640 1 --2- 172.21.15.114:0/144226697 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbafc076300 0x7fbafc078810 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:55.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.279+0000 7fbb21000640 1 --2- 172.21.15.114:0/144226697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbb1c15ade0 0x7fbb1c1f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:55.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.279+0000 7fbb21000640 1 --2- 172.21.15.114:0/144226697 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb1c1593b0 0x7fbb1c154090 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:55.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.279+0000 7fbb21000640 1 --2- 172.21.15.114:0/144226697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb1c073260 0x7fbb1c0c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:55.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.279+0000 7fbb21000640 1 -- 172.21.15.114:0/144226697 >> 172.21.15.114:0/144226697 conn(0x7fbb1c06f6e0 msgr2=0x7fbb1c153370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:55.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.279+0000 7fbb21000640 1 -- 172.21.15.114:0/144226697 shutdown_connections 2024-05-27T09:33:55.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:55.280+0000 7fbb21000640 1 -- 172.21.15.114:0/144226697 wait complete. 2024-05-27T09:33:55.304 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-05-27T09:33:57.306 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:33:57.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.590+0000 7f159b200640 1 Processor -- start 2024-05-27T09:33:57.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.591+0000 7f159b200640 1 -- start start 2024-05-27T09:33:57.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.593+0000 7f159b200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f159c0c2a00 0x7f159c159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:57.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.593+0000 7f159b200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f159c0c96d0 con 0x7f159c159e40 2024-05-27T09:33:57.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.593+0000 7f159b200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f159c0c8930 con 0x7f159c1534c0 2024-05-27T09:33:57.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.593+0000 7f159b200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f159c159190 con 0x7f159c0c2a00 2024-05-27T09:33:57.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.593+0000 7f1599e00640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f159c1534c0 0x7f159c0c8220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:49894/0 (socket says 172.21.15.114:49894) 2024-05-27T09:33:57.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.593+0000 7f1599e00640 1 -- 172.21.15.114:0/36068496 learned_addr learned my addr 172.21.15.114:0/36068496 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:33:57.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.593+0000 7f1599400640 1 --2- 172.21.15.114:0/36068496 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f159c0c2a00 0x7f159c159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:57.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.593+0000 7f1598a00640 1 -- 172.21.15.114:0/36068496 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3973752864 0 0) 0x7f159c0c8930 con 0x7f159c1534c0 2024-05-27T09:33:57.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.593+0000 7f1598a00640 1 -- 172.21.15.114:0/36068496 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1580003520 con 0x7f159c1534c0 2024-05-27T09:33:57.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.594+0000 7f1598a00640 1 -- 172.21.15.114:0/36068496 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 93625041 0 0) 0x7f159c0c96d0 con 0x7f159c159e40 2024-05-27T09:33:57.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.594+0000 7f1598a00640 1 -- 172.21.15.114:0/36068496 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f159c0c8930 con 0x7f159c159e40 2024-05-27T09:33:57.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.594+0000 7f1599400640 1 -- 172.21.15.114:0/36068496 >> v1:172.21.15.163:6789/0 conn(0x7f159c1534c0 legacy=0x7f159c0c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:57.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.594+0000 7f1599400640 1 -- 172.21.15.114:0/36068496 >> v1:172.21.15.114:6789/0 conn(0x7f159c159e40 legacy=0x7f159c074140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:57.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.594+0000 7f1599400640 1 -- 172.21.15.114:0/36068496 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f159c0c9970 con 0x7f159c0c2a00 2024-05-27T09:33:57.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.595+0000 7f1599400640 1 --2- 172.21.15.114:0/36068496 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f159c0c2a00 0x7f159c159900 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f15880097d0 tx=0x7f158802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c4341c21617e85e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:57.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.595+0000 7f1598a00640 1 -- 172.21.15.114:0/36068496 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588037070 con 0x7f159c0c2a00 2024-05-27T09:33:57.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.595+0000 7f1598a00640 1 -- 172.21.15.114:0/36068496 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1588032440 con 0x7f159c0c2a00 2024-05-27T09:33:57.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.595+0000 7f1598a00640 1 -- 172.21.15.114:0/36068496 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588032810 con 0x7f159c0c2a00 2024-05-27T09:33:57.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.596+0000 7f159b200640 1 -- 172.21.15.114:0/36068496 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f159c0c2a00 msgr2=0x7f159c159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:57.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.596+0000 7f159b200640 1 --2- 172.21.15.114:0/36068496 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f159c0c2a00 0x7f159c159900 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f15880097d0 tx=0x7f158802a210 comp rx=0 tx=0).stop 2024-05-27T09:33:57.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.596+0000 7f159b200640 1 -- 172.21.15.114:0/36068496 shutdown_connections 2024-05-27T09:33:57.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.596+0000 7f159b200640 1 --2- 172.21.15.114:0/36068496 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f159c0c2a00 0x7f159c159900 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:57.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.596+0000 7f159b200640 1 -- 172.21.15.114:0/36068496 >> 172.21.15.114:0/36068496 conn(0x7f159c0ca320 msgr2=0x7f159c0ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:57.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.596+0000 7f159b200640 1 -- 172.21.15.114:0/36068496 shutdown_connections 2024-05-27T09:33:57.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.596+0000 7f159b200640 1 -- 172.21.15.114:0/36068496 wait complete. 2024-05-27T09:33:57.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.598+0000 7f159b200640 1 Processor -- start 2024-05-27T09:33:57.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.598+0000 7f159b200640 1 -- start start 2024-05-27T09:33:57.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.600+0000 7f159b200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f159c0c2a00 0x7f159c14cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:57.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.600+0000 7f159b200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f159c1534c0 0x7f159c14b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:57.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.600+0000 7f159b200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f159c159e40 0x7f159c146d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:57.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.600+0000 7f159b200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f159c0c9f30 con 0x7f159c159e40 2024-05-27T09:33:57.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.600+0000 7f159b200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f159c1538d0 con 0x7f159c0c2a00 2024-05-27T09:33:57.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.600+0000 7f159b200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f159c1593e0 con 0x7f159c1534c0 2024-05-27T09:33:57.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.600+0000 7f1599e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f159c0c2a00 0x7f159c14cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:57.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.600+0000 7f1599e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f159c0c2a00 0x7f159c14cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:59570/0 (socket says 172.21.15.114:59570) 2024-05-27T09:33:57.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.600+0000 7f1599e00640 1 -- 172.21.15.114:0/1945614606 learned_addr learned my addr 172.21.15.114:0/1945614606 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:33:57.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.600+0000 7f159a800640 1 --2- 172.21.15.114:0/1945614606 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f159c159e40 0x7f159c146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:57.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.600+0000 7f1599400640 1 --2- 172.21.15.114:0/1945614606 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f159c1534c0 0x7f159c14b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:57.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.601+0000 7f1599e00640 1 -- 172.21.15.114:0/1945614606 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f159c1534c0 msgr2=0x7f159c14b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:57.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.601+0000 7f1599e00640 1 --2- 172.21.15.114:0/1945614606 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f159c1534c0 0x7f159c14b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:57.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.601+0000 7f1599e00640 1 -- 172.21.15.114:0/1945614606 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f159c159e40 msgr2=0x7f159c146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:57.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.601+0000 7f1599e00640 1 --2- 172.21.15.114:0/1945614606 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f159c159e40 0x7f159c146d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:57.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.601+0000 7f1599e00640 1 -- 172.21.15.114:0/1945614606 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f159c14b900 con 0x7f159c0c2a00 2024-05-27T09:33:57.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.601+0000 7f1599e00640 1 --2- 172.21.15.114:0/1945614606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f159c0c2a00 0x7f159c14cab0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f158c00c410 tx=0x7f158c00c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:57.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.601+0000 7f1592000640 1 -- 172.21.15.114:0/1945614606 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c017070 con 0x7f159c0c2a00 2024-05-27T09:33:57.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.601+0000 7f1592000640 1 -- 172.21.15.114:0/1945614606 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f158c005d00 con 0x7f159c0c2a00 2024-05-27T09:33:57.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.601+0000 7f159b200640 1 -- 172.21.15.114:0/1945614606 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f159c1f0980 con 0x7f159c0c2a00 2024-05-27T09:33:57.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.601+0000 7f1592000640 1 -- 172.21.15.114:0/1945614606 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c007310 con 0x7f159c0c2a00 2024-05-27T09:33:57.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.601+0000 7f159b200640 1 -- 172.21.15.114:0/1945614606 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f159c1f0f20 con 0x7f159c0c2a00 2024-05-27T09:33:57.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.602+0000 7f159b200640 1 -- 172.21.15.114:0/1945614606 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f159c0b3bc0 con 0x7f159c0c2a00 2024-05-27T09:33:57.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.604+0000 7f1592000640 1 -- 172.21.15.114:0/1945614606 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f158c01a070 con 0x7f159c0c2a00 2024-05-27T09:33:57.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.605+0000 7f1592000640 1 --2- 172.21.15.114:0/1945614606 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f157c076300 0x7f157c078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:57.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.605+0000 7f1592000640 1 -- 172.21.15.114:0/1945614606 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6024+0+0 (secure 0 0 0) 0x7f158c096b50 con 0x7f159c0c2a00 2024-05-27T09:33:57.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.605+0000 7f1599400640 1 --2- 172.21.15.114:0/1945614606 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f157c076300 0x7f157c078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:57.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.606+0000 7f1599400640 1 --2- 172.21.15.114:0/1945614606 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f157c076300 0x7f157c078810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1588034000 tx=0x7f158802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:57.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.610+0000 7f1592000640 1 -- 172.21.15.114:0/1945614606 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f159c0b3bc0 con 0x7f159c0c2a00 2024-05-27T09:33:57.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.860+0000 7f159b200640 1 -- 172.21.15.114:0/1945614606 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f159c050860 con 0x7f159c0c2a00 2024-05-27T09:33:57.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.861+0000 7f1592000640 1 -- 172.21.15.114:0/1945614606 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+6888 (secure 0 0 0) 0x7f158c05fbf0 con 0x7f159c0c2a00 2024-05-27T09:33:57.862 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:33:57.862 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:32:51.967298+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4875,4828,4300]},"id":1}]} 2024-05-27T09:33:57.862 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 30 2024-05-27T09:33:57.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.864+0000 7f159b200640 1 -- 172.21.15.114:0/1945614606 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f157c076300 msgr2=0x7f157c078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:57.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.864+0000 7f159b200640 1 --2- 172.21.15.114:0/1945614606 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f157c076300 0x7f157c078810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1588034000 tx=0x7f158802e000 comp rx=0 tx=0).stop 2024-05-27T09:33:57.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.864+0000 7f159b200640 1 -- 172.21.15.114:0/1945614606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f159c0c2a00 msgr2=0x7f159c14cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:57.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.864+0000 7f159b200640 1 --2- 172.21.15.114:0/1945614606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f159c0c2a00 0x7f159c14cab0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f158c00c410 tx=0x7f158c00c8d0 comp rx=0 tx=0).stop 2024-05-27T09:33:57.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.864+0000 7f159b200640 1 -- 172.21.15.114:0/1945614606 shutdown_connections 2024-05-27T09:33:57.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.864+0000 7f159b200640 1 --2- 172.21.15.114:0/1945614606 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f157c076300 0x7f157c078810 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:57.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.864+0000 7f159b200640 1 --2- 172.21.15.114:0/1945614606 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f159c159e40 0x7f159c146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:57.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.864+0000 7f159b200640 1 --2- 172.21.15.114:0/1945614606 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f159c1534c0 0x7f159c14b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:57.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.864+0000 7f159b200640 1 --2- 172.21.15.114:0/1945614606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f159c0c2a00 0x7f159c14cab0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:57.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.864+0000 7f159b200640 1 -- 172.21.15.114:0/1945614606 >> 172.21.15.114:0/1945614606 conn(0x7f159c0ca320 msgr2=0x7f159c157060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:57.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.864+0000 7f159b200640 1 -- 172.21.15.114:0/1945614606 shutdown_connections 2024-05-27T09:33:57.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:33:57.864+0000 7f159b200640 1 -- 172.21.15.114:0/1945614606 wait complete. 2024-05-27T09:33:57.889 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-05-27T09:33:59.890 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:34:00.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.174+0000 7f08cee00640 1 Processor -- start 2024-05-27T09:34:00.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.174+0000 7f08cee00640 1 -- start start 2024-05-27T09:34:00.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.176+0000 7f08cee00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08c80b2c60 0x7f08c815d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:00.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.176+0000 7f08cee00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08c8156f80 con 0x7f08c80ca140 2024-05-27T09:34:00.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.176+0000 7f08cee00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08c80cac50 con 0x7f08c80ca550 2024-05-27T09:34:00.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.176+0000 7f08cee00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f08c815d7c0 con 0x7f08c80b2c60 2024-05-27T09:34:00.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.177+0000 7f08c7400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f08c80ca140 0x7f08c8073310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:48506/0 (socket says 172.21.15.114:48506) 2024-05-27T09:34:00.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.177+0000 7f08c7400640 1 -- 172.21.15.114:0/147811590 learned_addr learned my addr 172.21.15.114:0/147811590 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:34:00.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.177+0000 7f08c7e00640 1 --2- 172.21.15.114:0/147811590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08c80b2c60 0x7f08c815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:00.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.177+0000 7f08c6a00640 1 -- 172.21.15.114:0/147811590 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3749399189 0 0) 0x7f08c8156f80 con 0x7f08c80ca140 2024-05-27T09:34:00.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.177+0000 7f08c6a00640 1 -- 172.21.15.114:0/147811590 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08ac003520 con 0x7f08c80ca140 2024-05-27T09:34:00.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.177+0000 7f08c6a00640 1 -- 172.21.15.114:0/147811590 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2366123095 0 0) 0x7f08c80cac50 con 0x7f08c80ca550 2024-05-27T09:34:00.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.177+0000 7f08c6a00640 1 -- 172.21.15.114:0/147811590 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08c8156f80 con 0x7f08c80ca550 2024-05-27T09:34:00.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.178+0000 7f08c7e00640 1 -- 172.21.15.114:0/147811590 >> v1:172.21.15.163:6789/0 conn(0x7f08c80ca550 legacy=0x7f08c80c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:00.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.178+0000 7f08c7e00640 1 -- 172.21.15.114:0/147811590 >> v1:172.21.15.114:6789/0 conn(0x7f08c80ca140 legacy=0x7f08c8073310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:00.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.178+0000 7f08c7e00640 1 -- 172.21.15.114:0/147811590 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c815ffa0 con 0x7f08c80b2c60 2024-05-27T09:34:00.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.179+0000 7f08c7e00640 1 --2- 172.21.15.114:0/147811590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08c80b2c60 0x7f08c815d280 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f08b80097d0 tx=0x7f08b802a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcbfe2b7407e3e58 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:00.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.180+0000 7f08c6a00640 1 -- 172.21.15.114:0/147811590 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8037070 con 0x7f08c80b2c60 2024-05-27T09:34:00.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.180+0000 7f08c6a00640 1 -- 172.21.15.114:0/147811590 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08b8032440 con 0x7f08c80b2c60 2024-05-27T09:34:00.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.180+0000 7f08c6a00640 1 -- 172.21.15.114:0/147811590 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b80327f0 con 0x7f08c80b2c60 2024-05-27T09:34:00.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.181+0000 7f08cee00640 1 -- 172.21.15.114:0/147811590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08c80b2c60 msgr2=0x7f08c815d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:00.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.181+0000 7f08cee00640 1 --2- 172.21.15.114:0/147811590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08c80b2c60 0x7f08c815d280 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f08b80097d0 tx=0x7f08b802a290 comp rx=0 tx=0).stop 2024-05-27T09:34:00.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.181+0000 7f08cee00640 1 -- 172.21.15.114:0/147811590 shutdown_connections 2024-05-27T09:34:00.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.181+0000 7f08cee00640 1 --2- 172.21.15.114:0/147811590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08c80b2c60 0x7f08c815d280 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:00.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.181+0000 7f08cee00640 1 -- 172.21.15.114:0/147811590 >> 172.21.15.114:0/147811590 conn(0x7f08c806f6e0 msgr2=0x7f08c8070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:00.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.181+0000 7f08cee00640 1 -- 172.21.15.114:0/147811590 shutdown_connections 2024-05-27T09:34:00.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.181+0000 7f08cee00640 1 -- 172.21.15.114:0/147811590 wait complete. 2024-05-27T09:34:00.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.183+0000 7f08cee00640 1 Processor -- start 2024-05-27T09:34:00.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.183+0000 7f08cee00640 1 -- start start 2024-05-27T09:34:00.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.185+0000 7f08cee00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08c80b2c60 0x7f08c815e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:00.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.185+0000 7f08cee00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f08c80ca140 0x7f08c80b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:00.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.185+0000 7f08cee00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08c80ca550 0x7f08c8149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:00.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.185+0000 7f08cee00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f08c8160560 con 0x7f08c80ca140 2024-05-27T09:34:00.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.185+0000 7f08cee00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f08c815fd50 con 0x7f08c80ca550 2024-05-27T09:34:00.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.185+0000 7f08cee00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f08c8160840 con 0x7f08c80b2c60 2024-05-27T09:34:00.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.185+0000 7f08c7e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08c80b2c60 0x7f08c815e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:00.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.185+0000 7f08c7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f08c80ca140 0x7f08c80b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:00.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.185+0000 7f08c7e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08c80b2c60 0x7f08c815e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:47384/0 (socket says 172.21.15.114:47384) 2024-05-27T09:34:00.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.185+0000 7f08c7e00640 1 -- 172.21.15.114:0/565305664 learned_addr learned my addr 172.21.15.114:0/565305664 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:34:00.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.185+0000 7f08cce00640 1 --2- 172.21.15.114:0/565305664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08c80ca550 0x7f08c8149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:00.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.186+0000 7f08c7e00640 1 -- 172.21.15.114:0/565305664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08c80ca550 msgr2=0x7f08c8149370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:00.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.186+0000 7f08c7e00640 1 --2- 172.21.15.114:0/565305664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08c80ca550 0x7f08c8149370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:00.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.186+0000 7f08c7e00640 1 -- 172.21.15.114:0/565305664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f08c80ca140 msgr2=0x7f08c80b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:00.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.186+0000 7f08c7e00640 1 --2- 172.21.15.114:0/565305664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f08c80ca140 0x7f08c80b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:00.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.186+0000 7f08c7e00640 1 -- 172.21.15.114:0/565305664 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c815ef20 con 0x7f08c80b2c60 2024-05-27T09:34:00.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.186+0000 7f08c7e00640 1 --2- 172.21.15.114:0/565305664 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08c80b2c60 0x7f08c815e9e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f08b8035040 tx=0x7f08b80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:00.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.186+0000 7f08bfe00640 1 -- 172.21.15.114:0/565305664 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8041070 con 0x7f08c80b2c60 2024-05-27T09:34:00.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.187+0000 7f08bfe00640 1 -- 172.21.15.114:0/565305664 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08b8004250 con 0x7f08c80b2c60 2024-05-27T09:34:00.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.187+0000 7f08bfe00640 1 -- 172.21.15.114:0/565305664 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8037040 con 0x7f08c80b2c60 2024-05-27T09:34:00.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.187+0000 7f08cee00640 1 -- 172.21.15.114:0/565305664 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c80af450 con 0x7f08c80b2c60 2024-05-27T09:34:00.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.187+0000 7f08cee00640 1 -- 172.21.15.114:0/565305664 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c80cc960 con 0x7f08c80b2c60 2024-05-27T09:34:00.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.188+0000 7f08cee00640 1 -- 172.21.15.114:0/565305664 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08c806a380 con 0x7f08c80b2c60 2024-05-27T09:34:00.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.189+0000 7f08bfe00640 1 -- 172.21.15.114:0/565305664 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f08b8003610 con 0x7f08c80b2c60 2024-05-27T09:34:00.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.190+0000 7f08bfe00640 1 --2- 172.21.15.114:0/565305664 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f08a8076300 0x7f08a8078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:00.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.190+0000 7f08bfe00640 1 -- 172.21.15.114:0/565305664 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(47..47 src has 1..47) ==== 6024+0+0 (secure 0 0 0) 0x7f08b80b8750 con 0x7f08c80b2c60 2024-05-27T09:34:00.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.191+0000 7f08c7400640 1 --2- 172.21.15.114:0/565305664 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f08a8076300 0x7f08a8078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:00.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.192+0000 7f08c7400640 1 --2- 172.21.15.114:0/565305664 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f08a8076300 0x7f08a8078810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f08c80c9b10 tx=0x7f08b4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:00.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.194+0000 7f08bfe00640 1 -- 172.21.15.114:0/565305664 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f08b8081900 con 0x7f08c80b2c60 2024-05-27T09:34:00.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.445+0000 7f08cee00640 1 -- 172.21.15.114:0/565305664 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08c8074660 con 0x7f08c80b2c60 2024-05-27T09:34:00.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.446+0000 7f08bfe00640 1 -- 172.21.15.114:0/565305664 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+6888 (secure 0 0 0) 0x7f08b8085c00 con 0x7f08c80b2c60 2024-05-27T09:34:00.447 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:34:00.447 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:32:51.967298+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4875,4828,4300]},"id":1}]} 2024-05-27T09:34:00.447 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 30 2024-05-27T09:34:00.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.450+0000 7f08cee00640 1 -- 172.21.15.114:0/565305664 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f08a8076300 msgr2=0x7f08a8078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:00.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.450+0000 7f08cee00640 1 --2- 172.21.15.114:0/565305664 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f08a8076300 0x7f08a8078810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f08c80c9b10 tx=0x7f08b4009040 comp rx=0 tx=0).stop 2024-05-27T09:34:00.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.450+0000 7f08cee00640 1 -- 172.21.15.114:0/565305664 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08c80b2c60 msgr2=0x7f08c815e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:00.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.450+0000 7f08cee00640 1 --2- 172.21.15.114:0/565305664 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08c80b2c60 0x7f08c815e9e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f08b8035040 tx=0x7f08b80046c0 comp rx=0 tx=0).stop 2024-05-27T09:34:00.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.450+0000 7f08cee00640 1 -- 172.21.15.114:0/565305664 shutdown_connections 2024-05-27T09:34:00.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.450+0000 7f08cee00640 1 --2- 172.21.15.114:0/565305664 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f08a8076300 0x7f08a8078810 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:00.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.450+0000 7f08cee00640 1 --2- 172.21.15.114:0/565305664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08c80ca550 0x7f08c8149370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:00.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.450+0000 7f08cee00640 1 --2- 172.21.15.114:0/565305664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f08c80ca140 0x7f08c80b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:00.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.450+0000 7f08cee00640 1 --2- 172.21.15.114:0/565305664 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08c80b2c60 0x7f08c815e9e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:00.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.450+0000 7f08cee00640 1 -- 172.21.15.114:0/565305664 >> 172.21.15.114:0/565305664 conn(0x7f08c806f6e0 msgr2=0x7f08c80c8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:00.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.451+0000 7f08cee00640 1 -- 172.21.15.114:0/565305664 shutdown_connections 2024-05-27T09:34:00.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:00.451+0000 7f08cee00640 1 -- 172.21.15.114:0/565305664 wait complete. 2024-05-27T09:34:00.474 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-05-27T09:34:02.475 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:34:02.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.758+0000 7f7503200640 1 Processor -- start 2024-05-27T09:34:02.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.758+0000 7f7503200640 1 -- start start 2024-05-27T09:34:02.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.760+0000 7f7503200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74fc15d280 0x7f74fc073310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:02.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.760+0000 7f7503200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f74fc073850 0x7f74fc0c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:02.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.760+0000 7f7503200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74fc156f80 con 0x7f74fc0b2c60 2024-05-27T09:34:02.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.760+0000 7f7503200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f74fc15fd50 con 0x7f74fc073850 2024-05-27T09:34:02.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.760+0000 7f7503200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f74fc0cac50 con 0x7f74fc15d280 2024-05-27T09:34:02.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.760+0000 7f7501200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f74fc073850 0x7f74fc0c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:02.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.761+0000 7f7501200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f74fc073850 0x7f74fc0c3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42860/0 (socket says 172.21.15.114:42860) 2024-05-27T09:34:02.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.761+0000 7f7501200640 1 -- 172.21.15.114:0/4198574240 learned_addr learned my addr 172.21.15.114:0/4198574240 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:34:02.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.761+0000 7f74f3e00640 1 --2- 172.21.15.114:0/4198574240 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74fc15d280 0x7f74fc073310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:02.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.761+0000 7f7501200640 1 -- 172.21.15.114:0/4198574240 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74fc15d280 msgr2=0x7f74fc073310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:34:02.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.761+0000 7f7501200640 1 --2- 172.21.15.114:0/4198574240 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74fc15d280 0x7f74fc073310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:02.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.761+0000 7f7501200640 1 -- 172.21.15.114:0/4198574240 >> v1:172.21.15.114:6789/0 conn(0x7f74fc0b2c60 legacy=0x7f74fc0ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:02.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.761+0000 7f7501200640 1 -- 172.21.15.114:0/4198574240 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74fc15ffa0 con 0x7f74fc073850 2024-05-27T09:34:02.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.762+0000 7f7501200640 1 --2- 172.21.15.114:0/4198574240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74fc073850 0x7f74fc0c3820 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f74f4009e50 tx=0x7f74f402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427fa5ef0ad08ee5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:02.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.762+0000 7f74fb400640 1 -- 172.21.15.114:0/4198574240 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f4036070 con 0x7f74fc073850 2024-05-27T09:34:02.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.763+0000 7f74fb400640 1 -- 172.21.15.114:0/4198574240 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74f4031600 con 0x7f74fc073850 2024-05-27T09:34:02.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.763+0000 7f74fb400640 1 -- 172.21.15.114:0/4198574240 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f40319d0 con 0x7f74fc073850 2024-05-27T09:34:02.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.763+0000 7f7503200640 1 -- 172.21.15.114:0/4198574240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74fc073850 msgr2=0x7f74fc0c3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:02.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.763+0000 7f7503200640 1 --2- 172.21.15.114:0/4198574240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74fc073850 0x7f74fc0c3820 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f74f4009e50 tx=0x7f74f402a7a0 comp rx=0 tx=0).stop 2024-05-27T09:34:02.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.763+0000 7f7503200640 1 -- 172.21.15.114:0/4198574240 shutdown_connections 2024-05-27T09:34:02.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.763+0000 7f7503200640 1 --2- 172.21.15.114:0/4198574240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74fc073850 0x7f74fc0c3820 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:02.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.763+0000 7f7503200640 1 --2- 172.21.15.114:0/4198574240 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74fc15d280 0x7f74fc073310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:02.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.764+0000 7f7503200640 1 -- 172.21.15.114:0/4198574240 >> 172.21.15.114:0/4198574240 conn(0x7f74fc06f6e0 msgr2=0x7f74fc070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:02.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.764+0000 7f7503200640 1 -- 172.21.15.114:0/4198574240 shutdown_connections 2024-05-27T09:34:02.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.764+0000 7f7503200640 1 -- 172.21.15.114:0/4198574240 wait complete. 2024-05-27T09:34:02.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.767+0000 7f7503200640 1 Processor -- start 2024-05-27T09:34:02.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.767+0000 7f7503200640 1 -- start start 2024-05-27T09:34:02.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.769+0000 7f7503200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74fc073850 0x7f74fc0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:02.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.769+0000 7f7503200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74fc0b2c60 0x7f74fc157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:02.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.769+0000 7f7503200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74fc15d280 0x7f74fc14d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:02.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.769+0000 7f7503200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f74fc160840 con 0x7f74fc0b2c60 2024-05-27T09:34:02.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.769+0000 7f7503200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f74fc160560 con 0x7f74fc073850 2024-05-27T09:34:02.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.769+0000 7f7503200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f74fc15d690 con 0x7f74fc15d280 2024-05-27T09:34:02.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.769+0000 7f74f3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74fc0b2c60 0x7f74fc157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:02.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.769+0000 7f74f3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74fc0b2c60 0x7f74fc157ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42872/0 (socket says 172.21.15.114:42872) 2024-05-27T09:34:02.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.769+0000 7f74f3e00640 1 -- 172.21.15.114:0/757085651 learned_addr learned my addr 172.21.15.114:0/757085651 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:34:02.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.769+0000 7f7501200640 1 --2- 172.21.15.114:0/757085651 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74fc15d280 0x7f74fc14d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:02.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.769+0000 7f74fbe00640 1 --2- 172.21.15.114:0/757085651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74fc073850 0x7f74fc0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:02.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.770+0000 7f74f3e00640 1 -- 172.21.15.114:0/757085651 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74fc15d280 msgr2=0x7f74fc14d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:02.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.770+0000 7f74f3e00640 1 --2- 172.21.15.114:0/757085651 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74fc15d280 0x7f74fc14d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:02.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.770+0000 7f74f3e00640 1 -- 172.21.15.114:0/757085651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74fc073850 msgr2=0x7f74fc0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:02.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.770+0000 7f74f3e00640 1 --2- 172.21.15.114:0/757085651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74fc073850 0x7f74fc0cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:02.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.770+0000 7f74f3e00640 1 -- 172.21.15.114:0/757085651 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74fc14db80 con 0x7f74fc0b2c60 2024-05-27T09:34:02.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.770+0000 7f74f3e00640 1 --2- 172.21.15.114:0/757085651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74fc0b2c60 0x7f74fc157ff0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f74e400e870 tx=0x7f74e400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:02.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.770+0000 7f74f8c00640 1 -- 172.21.15.114:0/757085651 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e4017070 con 0x7f74fc0b2c60 2024-05-27T09:34:02.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.770+0000 7f74f8c00640 1 -- 172.21.15.114:0/757085651 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74e400c940 con 0x7f74fc0b2c60 2024-05-27T09:34:02.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.770+0000 7f74f8c00640 1 -- 172.21.15.114:0/757085651 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e400cd30 con 0x7f74fc0b2c60 2024-05-27T09:34:02.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.770+0000 7f7503200640 1 -- 172.21.15.114:0/757085651 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74fc0cbf10 con 0x7f74fc0b2c60 2024-05-27T09:34:02.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.771+0000 7f7503200640 1 -- 172.21.15.114:0/757085651 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74fc0cbc30 con 0x7f74fc0b2c60 2024-05-27T09:34:02.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.772+0000 7f7503200640 1 -- 172.21.15.114:0/757085651 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74fc06a380 con 0x7f74fc0b2c60 2024-05-27T09:34:02.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.773+0000 7f74f8c00640 1 -- 172.21.15.114:0/757085651 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f74e401a070 con 0x7f74fc0b2c60 2024-05-27T09:34:02.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.774+0000 7f74f8c00640 1 --2- 172.21.15.114:0/757085651 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f74d80762d0 0x7f74d80787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:02.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.774+0000 7f74f8c00640 1 -- 172.21.15.114:0/757085651 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6024+0+0 (secure 0 0 0) 0x7f74e4097640 con 0x7f74fc0b2c60 2024-05-27T09:34:02.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.774+0000 7f74fbe00640 1 --2- 172.21.15.114:0/757085651 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f74d80762d0 0x7f74d80787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:02.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.775+0000 7f74fbe00640 1 --2- 172.21.15.114:0/757085651 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f74d80762d0 0x7f74d80787e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f74ec001720 tx=0x7f74ec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:02.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:02.781+0000 7f74f8c00640 1 -- 172.21.15.114:0/757085651 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f74fc14db80 con 0x7f74fc0b2c60 2024-05-27T09:34:03.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:03.024+0000 7f7503200640 1 -- 172.21.15.114:0/757085651 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74fc0cb9c0 con 0x7f74fc0b2c60 2024-05-27T09:34:03.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:03.025+0000 7f74f8c00640 1 -- 172.21.15.114:0/757085651 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+6888 (secure 0 0 0) 0x7f74e40606f0 con 0x7f74fc0b2c60 2024-05-27T09:34:03.027 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:34:03.027 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:32:51.967298+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4300,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.114:6837/1448950017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1448950017},{"type":"v1","addr":"172.21.15.114:6837","nonce":1448950017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4875,4828,4300]},"id":1}]} 2024-05-27T09:34:03.027 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 30 2024-05-27T09:34:03.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:03.028+0000 7f7503200640 1 -- 172.21.15.114:0/757085651 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f74d80762d0 msgr2=0x7f74d80787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:03.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:03.028+0000 7f7503200640 1 --2- 172.21.15.114:0/757085651 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f74d80762d0 0x7f74d80787e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f74ec001720 tx=0x7f74ec009040 comp rx=0 tx=0).stop 2024-05-27T09:34:03.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:03.028+0000 7f7503200640 1 -- 172.21.15.114:0/757085651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74fc0b2c60 msgr2=0x7f74fc157ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:03.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:03.028+0000 7f7503200640 1 --2- 172.21.15.114:0/757085651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74fc0b2c60 0x7f74fc157ff0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f74e400e870 tx=0x7f74e400ed30 comp rx=0 tx=0).stop 2024-05-27T09:34:03.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:03.029+0000 7f7503200640 1 -- 172.21.15.114:0/757085651 shutdown_connections 2024-05-27T09:34:03.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:03.029+0000 7f7503200640 1 --2- 172.21.15.114:0/757085651 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f74d80762d0 0x7f74d80787e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:03.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:03.029+0000 7f7503200640 1 --2- 172.21.15.114:0/757085651 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74fc15d280 0x7f74fc14d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:03.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:03.029+0000 7f7503200640 1 --2- 172.21.15.114:0/757085651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74fc0b2c60 0x7f74fc157ff0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:03.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:03.029+0000 7f7503200640 1 --2- 172.21.15.114:0/757085651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74fc073850 0x7f74fc0cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:03.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:03.029+0000 7f7503200640 1 -- 172.21.15.114:0/757085651 >> 172.21.15.114:0/757085651 conn(0x7f74fc06f6e0 msgr2=0x7f74fc155520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:03.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:03.029+0000 7f7503200640 1 -- 172.21.15.114:0/757085651 shutdown_connections 2024-05-27T09:34:03.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:03.029+0000 7f7503200640 1 -- 172.21.15.114:0/757085651 wait complete. 2024-05-27T09:34:03.055 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-05-27T09:34:05.056 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:34:05.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.337+0000 7f9222000640 1 Processor -- start 2024-05-27T09:34:05.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.338+0000 7f9222000640 1 -- start start 2024-05-27T09:34:05.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.340+0000 7f9222000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f921c156b50 0x7f921c0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:05.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.340+0000 7f9222000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c0cf810 0x7f921c0ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:05.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.340+0000 7f9222000640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f921c0c62b0 0x7f921c0ca920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:05.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.340+0000 7f9222000640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f921c06e7f0 con 0x7f921c0c62b0 2024-05-27T09:34:05.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.340+0000 7f9222000640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f921c0c6aa0 con 0x7f921c156b50 2024-05-27T09:34:05.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.340+0000 7f9222000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f921c0cfd50 con 0x7f921c0cf810 2024-05-27T09:34:05.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.340+0000 7f9221600640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f921c0c62b0 0x7f921c0ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:05.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.340+0000 7f9220c00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f921c156b50 0x7f921c0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:05.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.340+0000 7f921be00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c0cf810 0x7f921c0ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:05.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.340+0000 7f9220c00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f921c156b50 0x7f921c0c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:46672/0 (socket says 172.21.15.114:46672) 2024-05-27T09:34:05.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.340+0000 7f921be00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c0cf810 0x7f921c0ca3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:54682/0 (socket says 172.21.15.114:54682) 2024-05-27T09:34:05.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.340+0000 7f9221600640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f921c0c62b0 0x7f921c0ca920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39282/0 (socket says 172.21.15.114:39282) 2024-05-27T09:34:05.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.340+0000 7f9220c00640 1 -- 172.21.15.114:0/919394345 learned_addr learned my addr 172.21.15.114:0/919394345 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:34:05.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.341+0000 7f9220c00640 1 -- 172.21.15.114:0/919394345 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c0cf810 msgr2=0x7f921c0ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:34:05.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.341+0000 7f9220c00640 1 --2- 172.21.15.114:0/919394345 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c0cf810 0x7f921c0ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:05.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.341+0000 7f9220c00640 1 -- 172.21.15.114:0/919394345 >> v2:172.21.15.114:3300/0 conn(0x7f921c0c62b0 msgr2=0x7f921c0ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:34:05.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.341+0000 7f9220c00640 1 --2- 172.21.15.114:0/919394345 >> v2:172.21.15.114:3300/0 conn(0x7f921c0c62b0 0x7f921c0ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:05.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.341+0000 7f9220c00640 1 -- 172.21.15.114:0/919394345 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921c0d0030 con 0x7f921c156b50 2024-05-27T09:34:05.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.341+0000 7f9221600640 1 --2- 172.21.15.114:0/919394345 >> v2:172.21.15.114:3300/0 conn(0x7f921c0c62b0 0x7f921c0ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:34:05.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.342+0000 7f9220c00640 1 --2- 172.21.15.114:0/919394345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f921c156b50 0x7f921c0c52a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f921c0d03c0 tx=0x7f920c02a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=146a45e6c87dc05e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:05.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.342+0000 7f921b400640 1 -- 172.21.15.114:0/919394345 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c037070 con 0x7f921c156b50 2024-05-27T09:34:05.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.342+0000 7f921b400640 1 -- 172.21.15.114:0/919394345 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f920c032440 con 0x7f921c156b50 2024-05-27T09:34:05.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.342+0000 7f921b400640 1 -- 172.21.15.114:0/919394345 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c032810 con 0x7f921c156b50 2024-05-27T09:34:05.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.343+0000 7f9222000640 1 -- 172.21.15.114:0/919394345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f921c156b50 msgr2=0x7f921c0c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:05.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.343+0000 7f9222000640 1 --2- 172.21.15.114:0/919394345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f921c156b50 0x7f921c0c52a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f921c0d03c0 tx=0x7f920c02a1e0 comp rx=0 tx=0).stop 2024-05-27T09:34:05.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.344+0000 7f9222000640 1 -- 172.21.15.114:0/919394345 shutdown_connections 2024-05-27T09:34:05.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.344+0000 7f9222000640 1 --2- 172.21.15.114:0/919394345 >> v2:172.21.15.114:3300/0 conn(0x7f921c0c62b0 0x7f921c0ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:05.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.344+0000 7f9222000640 1 --2- 172.21.15.114:0/919394345 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c0cf810 0x7f921c0ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:05.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.344+0000 7f9222000640 1 --2- 172.21.15.114:0/919394345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f921c156b50 0x7f921c0c52a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:05.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.344+0000 7f9222000640 1 -- 172.21.15.114:0/919394345 >> 172.21.15.114:0/919394345 conn(0x7f921c071410 msgr2=0x7f921c068600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:05.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.344+0000 7f9222000640 1 -- 172.21.15.114:0/919394345 shutdown_connections 2024-05-27T09:34:05.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.344+0000 7f9222000640 1 -- 172.21.15.114:0/919394345 wait complete. 2024-05-27T09:34:05.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.346+0000 7f9222000640 1 Processor -- start 2024-05-27T09:34:05.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.347+0000 7f9222000640 1 -- start start 2024-05-27T09:34:05.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.348+0000 7f9222000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f921c0c62b0 0x7f921c0b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:05.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.348+0000 7f9222000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f921c0cf810 0x7f921c0c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:05.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.348+0000 7f9222000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c156b50 0x7f921c147450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:05.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.348+0000 7f9222000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f921c06ec80 con 0x7f921c0c62b0 2024-05-27T09:34:05.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.348+0000 7f9222000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f921c06e480 con 0x7f921c0cf810 2024-05-27T09:34:05.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.348+0000 7f9222000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f921c06ee00 con 0x7f921c156b50 2024-05-27T09:34:05.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.349+0000 7f9220c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f921c0c62b0 0x7f921c0b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:05.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.349+0000 7f9220c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f921c0c62b0 0x7f921c0b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39296/0 (socket says 172.21.15.114:39296) 2024-05-27T09:34:05.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.349+0000 7f9220c00640 1 -- 172.21.15.114:0/1025570840 learned_addr learned my addr 172.21.15.114:0/1025570840 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:34:05.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.349+0000 7f9221600640 1 --2- 172.21.15.114:0/1025570840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c156b50 0x7f921c147450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:05.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.349+0000 7f921be00640 1 --2- 172.21.15.114:0/1025570840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f921c0cf810 0x7f921c0c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:05.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.349+0000 7f9220c00640 1 -- 172.21.15.114:0/1025570840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c156b50 msgr2=0x7f921c147450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:05.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.349+0000 7f9220c00640 1 --2- 172.21.15.114:0/1025570840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c156b50 0x7f921c147450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:05.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.349+0000 7f9220c00640 1 -- 172.21.15.114:0/1025570840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f921c0cf810 msgr2=0x7f921c0c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:05.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.349+0000 7f9220c00640 1 --2- 172.21.15.114:0/1025570840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f921c0cf810 0x7f921c0c53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:05.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.349+0000 7f9220c00640 1 -- 172.21.15.114:0/1025570840 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921c0b1ad0 con 0x7f921c0c62b0 2024-05-27T09:34:05.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.349+0000 7f9220c00640 1 --2- 172.21.15.114:0/1025570840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f921c0c62b0 0x7f921c0b1590 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f920c02a220 tx=0x7f920c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:05.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.350+0000 7f9218c00640 1 -- 172.21.15.114:0/1025570840 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c041020 con 0x7f921c0c62b0 2024-05-27T09:34:05.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.350+0000 7f9218c00640 1 -- 172.21.15.114:0/1025570840 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f920c03d640 con 0x7f921c0c62b0 2024-05-27T09:34:05.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.350+0000 7f9222000640 1 -- 172.21.15.114:0/1025570840 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f921c150ad0 con 0x7f921c0c62b0 2024-05-27T09:34:05.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.350+0000 7f9218c00640 1 -- 172.21.15.114:0/1025570840 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c037040 con 0x7f921c0c62b0 2024-05-27T09:34:05.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.350+0000 7f9222000640 1 -- 172.21.15.114:0/1025570840 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f921c0b0ab0 con 0x7f921c0c62b0 2024-05-27T09:34:05.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.351+0000 7f9222000640 1 -- 172.21.15.114:0/1025570840 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f921c06d5d0 con 0x7f921c0c62b0 2024-05-27T09:34:05.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.353+0000 7f9218c00640 1 -- 172.21.15.114:0/1025570840 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f920c044020 con 0x7f921c0c62b0 2024-05-27T09:34:05.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.353+0000 7f9218c00640 1 --2- 172.21.15.114:0/1025570840 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f91f80762d0 0x7f91f80787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:05.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.354+0000 7f9218c00640 1 -- 172.21.15.114:0/1025570840 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6024+0+0 (secure 0 0 0) 0x7f920c0b7900 con 0x7f921c0c62b0 2024-05-27T09:34:05.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.354+0000 7f921be00640 1 --2- 172.21.15.114:0/1025570840 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f91f80762d0 0x7f91f80787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:05.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.355+0000 7f921be00640 1 --2- 172.21.15.114:0/1025570840 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f91f80762d0 0x7f91f80787e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f921c1f6bf0 tx=0x7f9204009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:05.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.359+0000 7f9218c00640 1 -- 172.21.15.114:0/1025570840 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f920c04b070 con 0x7f921c0c62b0 2024-05-27T09:34:05.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.610+0000 7f9222000640 1 -- 172.21.15.114:0/1025570840 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f921c151bc0 con 0x7f921c0c62b0 2024-05-27T09:34:05.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.612+0000 7f9218c00640 1 -- 172.21.15.114:0/1025570840 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+6056 (secure 0 0 0) 0x7f921c151bc0 con 0x7f921c0c62b0 2024-05-27T09:34:05.613 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:34:05.613 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:05.408102+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:replay","state_seq":1,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4875,"qdb_cluster":[4828,4875]},"id":1}]} 2024-05-27T09:34:05.613 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 31 2024-05-27T09:34:05.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.614+0000 7f9222000640 1 -- 172.21.15.114:0/1025570840 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f91f80762d0 msgr2=0x7f91f80787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:05.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.614+0000 7f9222000640 1 --2- 172.21.15.114:0/1025570840 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f91f80762d0 0x7f91f80787e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f921c1f6bf0 tx=0x7f9204009190 comp rx=0 tx=0).stop 2024-05-27T09:34:05.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.614+0000 7f9222000640 1 -- 172.21.15.114:0/1025570840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f921c0c62b0 msgr2=0x7f921c0b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:05.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.614+0000 7f9222000640 1 --2- 172.21.15.114:0/1025570840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f921c0c62b0 0x7f921c0b1590 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f920c02a220 tx=0x7f920c0046c0 comp rx=0 tx=0).stop 2024-05-27T09:34:05.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.614+0000 7f9222000640 1 -- 172.21.15.114:0/1025570840 shutdown_connections 2024-05-27T09:34:05.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.614+0000 7f9222000640 1 --2- 172.21.15.114:0/1025570840 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f91f80762d0 0x7f91f80787e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:05.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.614+0000 7f9222000640 1 --2- 172.21.15.114:0/1025570840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c156b50 0x7f921c147450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:05.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.614+0000 7f9222000640 1 --2- 172.21.15.114:0/1025570840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f921c0cf810 0x7f921c0c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:05.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.614+0000 7f9222000640 1 --2- 172.21.15.114:0/1025570840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f921c0c62b0 0x7f921c0b1590 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:05.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.614+0000 7f9222000640 1 -- 172.21.15.114:0/1025570840 >> 172.21.15.114:0/1025570840 conn(0x7f921c071410 msgr2=0x7f921c0cb140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:05.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.614+0000 7f9222000640 1 -- 172.21.15.114:0/1025570840 shutdown_connections 2024-05-27T09:34:05.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.614+0000 7f9222000640 1 -- 172.21.15.114:0/1025570840 wait complete. 2024-05-27T09:34:05.645 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2024-05-27T09:34:05.645 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-05-27T09:34:05.645 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:34:05.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.895+0000 7f245c600640 1 Processor -- start 2024-05-27T09:34:05.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.895+0000 7f245c600640 1 -- start start 2024-05-27T09:34:05.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.897+0000 7f245c600640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f24540c3820 0x7f2454073310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:05.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.897+0000 7f245c600640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24541615d0 con 0x7f2454074860 2024-05-27T09:34:05.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.897+0000 7f245c600640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2454161890 con 0x7f2454164db0 2024-05-27T09:34:05.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.897+0000 7f245c600640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f24540cade0 con 0x7f24540c3820 2024-05-27T09:34:05.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.898+0000 7f245a600640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f24540c3820 0x7f2454073310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:05.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.898+0000 7f2459200640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f2454074860 0x7f2454160b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:52068/0 (socket says 172.21.15.114:52068) 2024-05-27T09:34:05.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.898+0000 7f245a600640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f24540c3820 0x7f2454073310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39308/0 (socket says 172.21.15.114:39308) 2024-05-27T09:34:05.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.898+0000 7f2459200640 1 -- 172.21.15.114:0/3734301542 learned_addr learned my addr 172.21.15.114:0/3734301542 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:34:05.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.898+0000 7f2459c00640 1 --1- 172.21.15.114:0/3734301542 >> v1:172.21.15.163:6789/0 conn(0x7f2454164db0 0x7f245415aac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:53846/0 (socket says 172.21.15.114:53846) 2024-05-27T09:34:05.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.898+0000 7f245a600640 1 -- 172.21.15.114:0/3734301542 >> v1:172.21.15.163:6789/0 conn(0x7f2454164db0 legacy=0x7f245415aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:05.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.898+0000 7f245a600640 1 -- 172.21.15.114:0/3734301542 >> v1:172.21.15.114:6789/0 conn(0x7f2454074860 legacy=0x7f2454160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:05.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.898+0000 7f2453e00640 1 -- 172.21.15.114:0/3734301542 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1761024186 0 0) 0x7f2454161890 con 0x7f2454164db0 2024-05-27T09:34:05.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.898+0000 7f245a600640 1 -- 172.21.15.114:0/3734301542 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24541642d0 con 0x7f24540c3820 2024-05-27T09:34:05.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.899+0000 7f245a600640 1 --2- 172.21.15.114:0/3734301542 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24540c3820 0x7f2454073310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f244c009d90 tx=0x7f244c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3166db0e69d473aa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:05.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.900+0000 7f2453e00640 1 -- 172.21.15.114:0/3734301542 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c007740 con 0x7f24540c3820 2024-05-27T09:34:05.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.900+0000 7f2453e00640 1 -- 172.21.15.114:0/3734301542 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f244c0078e0 con 0x7f24540c3820 2024-05-27T09:34:05.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.900+0000 7f2453e00640 1 -- 172.21.15.114:0/3734301542 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c007cb0 con 0x7f24540c3820 2024-05-27T09:34:05.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.900+0000 7f245c600640 1 -- 172.21.15.114:0/3734301542 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24540c3820 msgr2=0x7f2454073310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:05.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.900+0000 7f245c600640 1 --2- 172.21.15.114:0/3734301542 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24540c3820 0x7f2454073310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f244c009d90 tx=0x7f244c02a5f0 comp rx=0 tx=0).stop 2024-05-27T09:34:05.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.900+0000 7f245c600640 1 -- 172.21.15.114:0/3734301542 shutdown_connections 2024-05-27T09:34:05.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.900+0000 7f245c600640 1 --2- 172.21.15.114:0/3734301542 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24540c3820 0x7f2454073310 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:05.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.900+0000 7f245c600640 1 -- 172.21.15.114:0/3734301542 >> 172.21.15.114:0/3734301542 conn(0x7f245406f6e0 msgr2=0x7f2454070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:05.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.900+0000 7f245c600640 1 -- 172.21.15.114:0/3734301542 shutdown_connections 2024-05-27T09:34:05.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.900+0000 7f245c600640 1 -- 172.21.15.114:0/3734301542 wait complete. 2024-05-27T09:34:05.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.903+0000 7f245c600640 1 Processor -- start 2024-05-27T09:34:05.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.903+0000 7f245c600640 1 -- start start 2024-05-27T09:34:05.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.904+0000 7f245c600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2454074860 0x7f2454162fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:05.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.904+0000 7f245c600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24540c3820 0x7f24541582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:05.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.904+0000 7f245c600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2454164db0 0x7f24541f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:05.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.904+0000 7f245c600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2454161250 con 0x7f24540c3820 2024-05-27T09:34:05.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.904+0000 7f245c600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2454164890 con 0x7f2454074860 2024-05-27T09:34:05.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.904+0000 7f245c600640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f24540c3c30 con 0x7f2454164db0 2024-05-27T09:34:05.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.905+0000 7f2459200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24540c3820 0x7f24541582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:05.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.905+0000 7f245a600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2454164db0 0x7f24541f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:05.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.905+0000 7f245a600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2454164db0 0x7f24541f6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:54704/0 (socket says 172.21.15.114:54704) 2024-05-27T09:34:05.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.905+0000 7f245a600640 1 -- 172.21.15.114:0/437732440 learned_addr learned my addr 172.21.15.114:0/437732440 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:34:05.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.905+0000 7f2459c00640 1 --2- 172.21.15.114:0/437732440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2454074860 0x7f2454162fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:05.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.905+0000 7f2459200640 1 -- 172.21.15.114:0/437732440 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2454164db0 msgr2=0x7f24541f6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:05.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.905+0000 7f2459200640 1 --2- 172.21.15.114:0/437732440 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2454164db0 0x7f24541f6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:05.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.905+0000 7f2459200640 1 -- 172.21.15.114:0/437732440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2454074860 msgr2=0x7f2454162fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:05.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.905+0000 7f2459200640 1 --2- 172.21.15.114:0/437732440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2454074860 0x7f2454162fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:05.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.905+0000 7f2459200640 1 -- 172.21.15.114:0/437732440 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24541f7260 con 0x7f24540c3820 2024-05-27T09:34:05.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.905+0000 7f245a600640 1 --2- 172.21.15.114:0/437732440 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2454164db0 0x7f24541f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:34:05.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.906+0000 7f2459200640 1 --2- 172.21.15.114:0/437732440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24540c3820 0x7f24541582a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f244000c4d0 tx=0x7f244000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:05.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.906+0000 7f2451600640 1 -- 172.21.15.114:0/437732440 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2440018070 con 0x7f24540c3820 2024-05-27T09:34:05.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.906+0000 7f2451600640 1 -- 172.21.15.114:0/437732440 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24400074e0 con 0x7f24540c3820 2024-05-27T09:34:05.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.906+0000 7f245c600640 1 -- 172.21.15.114:0/437732440 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2454163a30 con 0x7f24540c3820 2024-05-27T09:34:05.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.906+0000 7f2451600640 1 -- 172.21.15.114:0/437732440 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2440007690 con 0x7f24540c3820 2024-05-27T09:34:05.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.906+0000 7f245c600640 1 -- 172.21.15.114:0/437732440 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2454157c50 con 0x7f24540c3820 2024-05-27T09:34:05.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.907+0000 7f245c600640 1 -- 172.21.15.114:0/437732440 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f245406a380 con 0x7f24540c3820 2024-05-27T09:34:05.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.908+0000 7f2451600640 1 -- 172.21.15.114:0/437732440 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f244001b070 con 0x7f24540c3820 2024-05-27T09:34:05.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.909+0000 7f2451600640 1 --2- 172.21.15.114:0/437732440 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2434076300 0x7f2434078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:05.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.909+0000 7f2451600640 1 -- 172.21.15.114:0/437732440 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6082+0+0 (secure 0 0 0) 0x7f2440096a20 con 0x7f24540c3820 2024-05-27T09:34:05.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.909+0000 7f2459c00640 1 --2- 172.21.15.114:0/437732440 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2434076300 0x7f2434078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:05.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.910+0000 7f2459c00640 1 --2- 172.21.15.114:0/437732440 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2434076300 0x7f2434078810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2448000f60 tx=0x7f2448007e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:05.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:05.914+0000 7f2451600640 1 -- 172.21.15.114:0/437732440 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f244005fb10 con 0x7f24540c3820 2024-05-27T09:34:06.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:06.153+0000 7f245c600640 1 -- 172.21.15.114:0/437732440 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2454073e30 con 0x7f24540c3820 2024-05-27T09:34:06.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:06.155+0000 7f2451600640 1 -- 172.21.15.114:0/437732440 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+6056 (secure 0 0 0) 0x7f2440063e10 con 0x7f24540c3820 2024-05-27T09:34:06.156 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:34:06.156 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:05.408102+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:replay","state_seq":1,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4875,"qdb_cluster":[4828,4875]},"id":1}]} 2024-05-27T09:34:06.157 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 31 2024-05-27T09:34:06.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:06.157+0000 7f245c600640 1 -- 172.21.15.114:0/437732440 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2434076300 msgr2=0x7f2434078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:06.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:06.157+0000 7f245c600640 1 --2- 172.21.15.114:0/437732440 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2434076300 0x7f2434078810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2448000f60 tx=0x7f2448007e40 comp rx=0 tx=0).stop 2024-05-27T09:34:06.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:06.157+0000 7f245c600640 1 -- 172.21.15.114:0/437732440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24540c3820 msgr2=0x7f24541582a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:06.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:06.157+0000 7f245c600640 1 --2- 172.21.15.114:0/437732440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24540c3820 0x7f24541582a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f244000c4d0 tx=0x7f244000c990 comp rx=0 tx=0).stop 2024-05-27T09:34:06.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:06.158+0000 7f245c600640 1 -- 172.21.15.114:0/437732440 shutdown_connections 2024-05-27T09:34:06.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:06.158+0000 7f245c600640 1 --2- 172.21.15.114:0/437732440 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2434076300 0x7f2434078810 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:06.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:06.158+0000 7f245c600640 1 --2- 172.21.15.114:0/437732440 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2454164db0 0x7f24541f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:06.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:06.158+0000 7f245c600640 1 --2- 172.21.15.114:0/437732440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24540c3820 0x7f24541582a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:06.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:06.158+0000 7f245c600640 1 --2- 172.21.15.114:0/437732440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2454074860 0x7f2454162fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:06.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:06.158+0000 7f245c600640 1 -- 172.21.15.114:0/437732440 >> 172.21.15.114:0/437732440 conn(0x7f245406f6e0 msgr2=0x7f24540c8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:06.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:06.158+0000 7f245c600640 1 -- 172.21.15.114:0/437732440 shutdown_connections 2024-05-27T09:34:06.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:06.158+0000 7f245c600640 1 -- 172.21.15.114:0/437732440 wait complete. 2024-05-27T09:34:06.184 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 31, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-05-27T09:27:18.079214+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 31, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4828": { "addr": "172.21.15.163:6839/3371890103", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6837", "nonce": 3371890103, "type": "v2" }, { "addr": "172.21.15.163:6839", "nonce": 3371890103, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4828, "incarnation": 19, "join_fscid": -1, "name": "f", "rank": 2, "state": "up:active", "state_seq": 20 }, "gid_4875": { "addr": "172.21.15.114:6835/2747052034", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6833", "nonce": 2747052034, "type": "v2" }, { "addr": "172.21.15.114:6835", "nonce": 2747052034, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4875, "incarnation": 25, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 25 }, "gid_5086": { "addr": "172.21.15.114:6834/1371544448", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6832", "nonce": 1371544448, "type": "v2" }, { "addr": "172.21.15.114:6834", "nonce": 1371544448, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5086, "incarnation": 31, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 48, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-05-27T09:34:05.408102+0000", "qdb_cluster": [ 4828, 4875 ], "qdb_leader": 4875, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5086, "mds_1": 4875, "mds_2": 4828 } } } ], "standbys": [ { "addr": "172.21.15.163:6835/3513522318", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 3513522318, "type": "v2" }, { "addr": "172.21.15.163:6835", "nonce": 3513522318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6838/1753596112", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6836", "nonce": 1753596112, "type": "v2" }, { "addr": "172.21.15.163:6838", "nonce": 1753596112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-05-27T09:34:08.184 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:34:08.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.468+0000 7fc17f200640 1 Processor -- start 2024-05-27T09:34:08.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.469+0000 7fc17f200640 1 -- start start 2024-05-27T09:34:08.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.470+0000 7fc17f200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc180074140 0x7fc180074550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:08.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.470+0000 7fc17f200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1800c96d0 con 0x7fc1801534c0 2024-05-27T09:34:08.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.470+0000 7fc17f200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1800c8930 con 0x7fc1800c2a00 2024-05-27T09:34:08.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.470+0000 7fc17f200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc180159190 con 0x7fc180074140 2024-05-27T09:34:08.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.471+0000 7fc17de00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fc1801534c0 0x7fc1800c8220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:52078/0 (socket says 172.21.15.114:52078) 2024-05-27T09:34:08.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.471+0000 7fc17de00640 1 -- 172.21.15.114:0/3868414344 learned_addr learned my addr 172.21.15.114:0/3868414344 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:34:08.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.471+0000 7fc17e800640 1 --2- 172.21.15.114:0/3868414344 >> v2:172.21.15.114:3300/0 conn(0x7fc180074140 0x7fc180074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:08.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.471+0000 7fc17ca00640 1 -- 172.21.15.114:0/3868414344 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1110081040 0 0) 0x7fc1800c96d0 con 0x7fc1801534c0 2024-05-27T09:34:08.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.471+0000 7fc17ca00640 1 -- 172.21.15.114:0/3868414344 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc164003520 con 0x7fc1801534c0 2024-05-27T09:34:08.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.471+0000 7fc17ca00640 1 -- 172.21.15.114:0/3868414344 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 539356981 0 0) 0x7fc1800c8930 con 0x7fc1800c2a00 2024-05-27T09:34:08.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.471+0000 7fc17ca00640 1 -- 172.21.15.114:0/3868414344 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1800c96d0 con 0x7fc1800c2a00 2024-05-27T09:34:08.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.471+0000 7fc17e800640 1 -- 172.21.15.114:0/3868414344 >> v1:172.21.15.163:6789/0 conn(0x7fc1800c2a00 legacy=0x7fc180159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:08.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.471+0000 7fc17e800640 1 -- 172.21.15.114:0/3868414344 >> v1:172.21.15.114:6789/0 conn(0x7fc1801534c0 legacy=0x7fc1800c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:08.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.471+0000 7fc17e800640 1 -- 172.21.15.114:0/3868414344 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1800c9970 con 0x7fc180074140 2024-05-27T09:34:08.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.472+0000 7fc17e800640 1 --2- 172.21.15.114:0/3868414344 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc180074140 0x7fc180074550 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc1780097a0 tx=0x7fc17802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd1ccd22e7872d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.473+0000 7fc17ca00640 1 -- 172.21.15.114:0/3868414344 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178037070 con 0x7fc180074140 2024-05-27T09:34:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.473+0000 7fc17ca00640 1 -- 172.21.15.114:0/3868414344 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc178032440 con 0x7fc180074140 2024-05-27T09:34:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.473+0000 7fc17ca00640 1 -- 172.21.15.114:0/3868414344 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178032810 con 0x7fc180074140 2024-05-27T09:34:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.473+0000 7fc17f200640 1 -- 172.21.15.114:0/3868414344 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc180074140 msgr2=0x7fc180074550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.473+0000 7fc17f200640 1 --2- 172.21.15.114:0/3868414344 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc180074140 0x7fc180074550 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc1780097a0 tx=0x7fc17802a230 comp rx=0 tx=0).stop 2024-05-27T09:34:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.473+0000 7fc17f200640 1 -- 172.21.15.114:0/3868414344 shutdown_connections 2024-05-27T09:34:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.473+0000 7fc17f200640 1 --2- 172.21.15.114:0/3868414344 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc180074140 0x7fc180074550 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.473+0000 7fc17f200640 1 -- 172.21.15.114:0/3868414344 >> 172.21.15.114:0/3868414344 conn(0x7fc1800ca320 msgr2=0x7fc1800ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.474+0000 7fc17f200640 1 -- 172.21.15.114:0/3868414344 shutdown_connections 2024-05-27T09:34:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.474+0000 7fc17f200640 1 -- 172.21.15.114:0/3868414344 wait complete. 2024-05-27T09:34:08.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.476+0000 7fc17f200640 1 Processor -- start 2024-05-27T09:34:08.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.476+0000 7fc17f200640 1 -- start start 2024-05-27T09:34:08.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.477+0000 7fc17f200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc180074140 0x7fc1801f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:08.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.477+0000 7fc17f200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1800c2a00 0x7fc1800c2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:08.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.477+0000 7fc17f200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1801534c0 0x7fc18014ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:08.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.477+0000 7fc17f200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc1800c9f30 con 0x7fc1800c2a00 2024-05-27T09:34:08.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.477+0000 7fc17f200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc1801538d0 con 0x7fc1801534c0 2024-05-27T09:34:08.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.477+0000 7fc17f200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fc1801593e0 con 0x7fc180074140 2024-05-27T09:34:08.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.478+0000 7fc17d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1800c2a00 0x7fc1800c2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:08.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.478+0000 7fc17d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1800c2a00 0x7fc1800c2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39324/0 (socket says 172.21.15.114:39324) 2024-05-27T09:34:08.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.478+0000 7fc17d400640 1 -- 172.21.15.114:0/2680867423 learned_addr learned my addr 172.21.15.114:0/2680867423 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:34:08.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.478+0000 7fc17e800640 1 --2- 172.21.15.114:0/2680867423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1801534c0 0x7fc18014ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:08.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.478+0000 7fc17de00640 1 --2- 172.21.15.114:0/2680867423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc180074140 0x7fc1801f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:08.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.478+0000 7fc17d400640 1 -- 172.21.15.114:0/2680867423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc180074140 msgr2=0x7fc1801f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:08.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.478+0000 7fc17d400640 1 --2- 172.21.15.114:0/2680867423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc180074140 0x7fc1801f3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:08.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.478+0000 7fc17d400640 1 -- 172.21.15.114:0/2680867423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1801534c0 msgr2=0x7fc18014ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:08.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.478+0000 7fc17d400640 1 --2- 172.21.15.114:0/2680867423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1801534c0 0x7fc18014ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:08.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.478+0000 7fc17d400640 1 -- 172.21.15.114:0/2680867423 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1801f3990 con 0x7fc1800c2a00 2024-05-27T09:34:08.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.479+0000 7fc17d400640 1 --2- 172.21.15.114:0/2680867423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1800c2a00 0x7fc1800c2ff0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc17000c770 tx=0x7fc17000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:08.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.479+0000 7fc176000640 1 -- 172.21.15.114:0/2680867423 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170003b70 con 0x7fc1800c2a00 2024-05-27T09:34:08.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.479+0000 7fc176000640 1 -- 172.21.15.114:0/2680867423 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc170003d10 con 0x7fc1800c2a00 2024-05-27T09:34:08.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.479+0000 7fc17f200640 1 -- 172.21.15.114:0/2680867423 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1800c7870 con 0x7fc1800c2a00 2024-05-27T09:34:08.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.479+0000 7fc176000640 1 -- 172.21.15.114:0/2680867423 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17000aa90 con 0x7fc1800c2a00 2024-05-27T09:34:08.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.479+0000 7fc17f200640 1 -- 172.21.15.114:0/2680867423 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1801f6c70 con 0x7fc1800c2a00 2024-05-27T09:34:08.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.480+0000 7fc17f200640 1 -- 172.21.15.114:0/2680867423 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1801f6700 con 0x7fc1800c2a00 2024-05-27T09:34:08.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.481+0000 7fc176000640 1 -- 172.21.15.114:0/2680867423 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc17000ac30 con 0x7fc1800c2a00 2024-05-27T09:34:08.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.482+0000 7fc176000640 1 --2- 172.21.15.114:0/2680867423 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc15c0762d0 0x7fc15c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:08.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.482+0000 7fc176000640 1 -- 172.21.15.114:0/2680867423 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6082+0+0 (secure 0 0 0) 0x7fc1700976b0 con 0x7fc1800c2a00 2024-05-27T09:34:08.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.483+0000 7fc17de00640 1 --2- 172.21.15.114:0/2680867423 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc15c0762d0 0x7fc15c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:08.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.483+0000 7fc17de00640 1 --2- 172.21.15.114:0/2680867423 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc15c0762d0 0x7fc15c0787e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc168007f50 tx=0x7fc168007ee0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:08.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.487+0000 7fc176000640 1 -- 172.21.15.114:0/2680867423 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc170060710 con 0x7fc1800c2a00 2024-05-27T09:34:08.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.730+0000 7fc17f200640 1 -- 172.21.15.114:0/2680867423 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc180050860 con 0x7fc1800c2a00 2024-05-27T09:34:08.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.731+0000 7fc176000640 1 -- 172.21.15.114:0/2680867423 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+6056 (secure 0 0 0) 0x7fc170064a10 con 0x7fc1800c2a00 2024-05-27T09:34:08.733 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:34:08.733 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:05.408102+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:replay","state_seq":1,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4875,"qdb_cluster":[4828,4875]},"id":1}]} 2024-05-27T09:34:08.733 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 31 2024-05-27T09:34:08.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.735+0000 7fc17f200640 1 -- 172.21.15.114:0/2680867423 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc15c0762d0 msgr2=0x7fc15c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:08.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.735+0000 7fc17f200640 1 --2- 172.21.15.114:0/2680867423 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc15c0762d0 0x7fc15c0787e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc168007f50 tx=0x7fc168007ee0 comp rx=0 tx=0).stop 2024-05-27T09:34:08.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.736+0000 7fc17f200640 1 -- 172.21.15.114:0/2680867423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1800c2a00 msgr2=0x7fc1800c2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:08.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.736+0000 7fc17f200640 1 --2- 172.21.15.114:0/2680867423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1800c2a00 0x7fc1800c2ff0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc17000c770 tx=0x7fc17000cc30 comp rx=0 tx=0).stop 2024-05-27T09:34:08.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.736+0000 7fc17f200640 1 -- 172.21.15.114:0/2680867423 shutdown_connections 2024-05-27T09:34:08.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.736+0000 7fc17f200640 1 --2- 172.21.15.114:0/2680867423 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc15c0762d0 0x7fc15c0787e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:08.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.736+0000 7fc17f200640 1 --2- 172.21.15.114:0/2680867423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1801534c0 0x7fc18014ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:08.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.736+0000 7fc17f200640 1 --2- 172.21.15.114:0/2680867423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1800c2a00 0x7fc1800c2ff0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:08.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.736+0000 7fc17f200640 1 --2- 172.21.15.114:0/2680867423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc180074140 0x7fc1801f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:08.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.736+0000 7fc17f200640 1 -- 172.21.15.114:0/2680867423 >> 172.21.15.114:0/2680867423 conn(0x7fc1800ca320 msgr2=0x7fc18006f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:08.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.736+0000 7fc17f200640 1 -- 172.21.15.114:0/2680867423 shutdown_connections 2024-05-27T09:34:08.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:08.736+0000 7fc17f200640 1 -- 172.21.15.114:0/2680867423 wait complete. 2024-05-27T09:34:08.761 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:34:10.762 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:34:11.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.020+0000 7f716b800640 1 Processor -- start 2024-05-27T09:34:11.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.021+0000 7f716b800640 1 -- start start 2024-05-27T09:34:11.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.024+0000 7f716b800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71641593b0 0x7f7164142720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:11.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.024+0000 7f716b800640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7164073260 0x7f716415a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:11.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.024+0000 7f716b800640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71640739a0 con 0x7f716415ade0 2024-05-27T09:34:11.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.024+0000 7f716b800640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f7164074d20 con 0x7f7164073260 2024-05-27T09:34:11.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.024+0000 7f716b800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f7164073c80 con 0x7f71641593b0 2024-05-27T09:34:11.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.024+0000 7f7169800640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f716415ade0 0x7f71640c2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:52086/0 (socket says 172.21.15.114:52086) 2024-05-27T09:34:11.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.024+0000 7f7169800640 1 -- 172.21.15.114:0/576221578 learned_addr learned my addr 172.21.15.114:0/576221578 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:34:11.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.024+0000 7f7163e00640 1 --2- 172.21.15.114:0/576221578 >> v2:172.21.15.163:3300/0 conn(0x7f7164073260 0x7f716415a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:11.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.024+0000 7f7168e00640 1 --2- 172.21.15.114:0/576221578 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71641593b0 0x7f7164142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:11.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.025+0000 7f7163400640 1 -- 172.21.15.114:0/576221578 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2755665788 0 0) 0x7f71640739a0 con 0x7f716415ade0 2024-05-27T09:34:11.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.025+0000 7f7163400640 1 -- 172.21.15.114:0/576221578 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f714c003520 con 0x7f716415ade0 2024-05-27T09:34:11.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.025+0000 7f7168e00640 1 -- 172.21.15.114:0/576221578 >> v2:172.21.15.163:3300/0 conn(0x7f7164073260 msgr2=0x7f716415a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:34:11.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.025+0000 7f7168e00640 1 --2- 172.21.15.114:0/576221578 >> v2:172.21.15.163:3300/0 conn(0x7f7164073260 0x7f716415a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:11.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.025+0000 7f7168e00640 1 -- 172.21.15.114:0/576221578 >> v1:172.21.15.114:6789/0 conn(0x7f716415ade0 legacy=0x7f71640c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:11.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.025+0000 7f7168e00640 1 -- 172.21.15.114:0/576221578 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7164073f60 con 0x7f71641593b0 2024-05-27T09:34:11.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.025+0000 7f7163e00640 1 --2- 172.21.15.114:0/576221578 >> v2:172.21.15.163:3300/0 conn(0x7f7164073260 0x7f716415a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:34:11.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.026+0000 7f7168e00640 1 --2- 172.21.15.114:0/576221578 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71641593b0 0x7f7164142720 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f715800ba40 tx=0x7f715802c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14b237f22e259a7b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:11.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.026+0000 7f7163400640 1 -- 172.21.15.114:0/576221578 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715802f070 con 0x7f71641593b0 2024-05-27T09:34:11.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.026+0000 7f7163400640 1 -- 172.21.15.114:0/576221578 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7158038070 con 0x7f71641593b0 2024-05-27T09:34:11.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.026+0000 7f7163400640 1 -- 172.21.15.114:0/576221578 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71580329e0 con 0x7f71641593b0 2024-05-27T09:34:11.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.027+0000 7f716b800640 1 -- 172.21.15.114:0/576221578 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71641593b0 msgr2=0x7f7164142720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:11.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.027+0000 7f716b800640 1 --2- 172.21.15.114:0/576221578 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71641593b0 0x7f7164142720 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f715800ba40 tx=0x7f715802c8e0 comp rx=0 tx=0).stop 2024-05-27T09:34:11.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.027+0000 7f716b800640 1 -- 172.21.15.114:0/576221578 shutdown_connections 2024-05-27T09:34:11.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.027+0000 7f716b800640 1 --2- 172.21.15.114:0/576221578 >> v2:172.21.15.163:3300/0 conn(0x7f7164073260 0x7f716415a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:11.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.027+0000 7f716b800640 1 --2- 172.21.15.114:0/576221578 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71641593b0 0x7f7164142720 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:11.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.027+0000 7f716b800640 1 -- 172.21.15.114:0/576221578 >> 172.21.15.114:0/576221578 conn(0x7f716406f6e0 msgr2=0x7f7164070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:11.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.028+0000 7f716b800640 1 -- 172.21.15.114:0/576221578 shutdown_connections 2024-05-27T09:34:11.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.028+0000 7f716b800640 1 -- 172.21.15.114:0/576221578 wait complete. 2024-05-27T09:34:11.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.031+0000 7f716b800640 1 Processor -- start 2024-05-27T09:34:11.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.031+0000 7f716b800640 1 -- start start 2024-05-27T09:34:11.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.034+0000 7f716b800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7164073260 0x7f716415ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:11.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.034+0000 7f716b800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71641593b0 0x7f71641ee460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:11.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.034+0000 7f716b800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f716415ade0 0x7f716415c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:11.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.034+0000 7f716b800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7164074800 con 0x7f71641593b0 2024-05-27T09:34:11.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.034+0000 7f716b800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7164074520 con 0x7f7164073260 2024-05-27T09:34:11.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.034+0000 7f716b800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f7164073670 con 0x7f716415ade0 2024-05-27T09:34:11.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.034+0000 7f7163e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71641593b0 0x7f71641ee460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:11.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.034+0000 7f7168e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7164073260 0x7f716415ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:11.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.034+0000 7f7169800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f716415ade0 0x7f716415c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:11.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.034+0000 7f7163e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71641593b0 0x7f71641ee460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39334/0 (socket says 172.21.15.114:39334) 2024-05-27T09:34:11.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.034+0000 7f7163e00640 1 -- 172.21.15.114:0/1120870418 learned_addr learned my addr 172.21.15.114:0/1120870418 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:34:11.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.034+0000 7f7169800640 1 -- 172.21.15.114:0/1120870418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7164073260 msgr2=0x7f716415ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:11.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.034+0000 7f7169800640 1 --2- 172.21.15.114:0/1120870418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7164073260 0x7f716415ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:11.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.034+0000 7f7169800640 1 -- 172.21.15.114:0/1120870418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71641593b0 msgr2=0x7f71641ee460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:11.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.034+0000 7f7169800640 1 --2- 172.21.15.114:0/1120870418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71641593b0 0x7f71641ee460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:11.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.034+0000 7f7169800640 1 -- 172.21.15.114:0/1120870418 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71641ee9a0 con 0x7f716415ade0 2024-05-27T09:34:11.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.035+0000 7f7169800640 1 --2- 172.21.15.114:0/1120870418 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f716415ade0 0x7f716415c4b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f71640c8a60 tx=0x7f715c00ce40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:11.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.035+0000 7f7160c00640 1 -- 172.21.15.114:0/1120870418 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c00ab10 con 0x7f716415ade0 2024-05-27T09:34:11.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.035+0000 7f7160c00640 1 -- 172.21.15.114:0/1120870418 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f715c00acb0 con 0x7f716415ade0 2024-05-27T09:34:11.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.035+0000 7f716b800640 1 -- 172.21.15.114:0/1120870418 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71640c6c00 con 0x7f716415ade0 2024-05-27T09:34:11.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.035+0000 7f7160c00640 1 -- 172.21.15.114:0/1120870418 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c012620 con 0x7f716415ade0 2024-05-27T09:34:11.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.035+0000 7f716b800640 1 -- 172.21.15.114:0/1120870418 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f71640c71c0 con 0x7f716415ade0 2024-05-27T09:34:11.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.036+0000 7f716b800640 1 -- 172.21.15.114:0/1120870418 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f716406a380 con 0x7f716415ade0 2024-05-27T09:34:11.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.039+0000 7f7160c00640 1 -- 172.21.15.114:0/1120870418 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f715c0127c0 con 0x7f716415ade0 2024-05-27T09:34:11.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.040+0000 7f7160c00640 1 --2- 172.21.15.114:0/1120870418 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7144076300 0x7f7144078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:11.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.040+0000 7f7160c00640 1 -- 172.21.15.114:0/1120870418 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(48..48 src has 1..48) ==== 6082+0+0 (secure 0 0 0) 0x7f715c097b70 con 0x7f716415ade0 2024-05-27T09:34:11.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.040+0000 7f7168e00640 1 --2- 172.21.15.114:0/1120870418 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7144076300 0x7f7144078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:11.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.041+0000 7f7168e00640 1 --2- 172.21.15.114:0/1120870418 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7144076300 0x7f7144078810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7158037000 tx=0x7f715803f040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:11.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.045+0000 7f7160c00640 1 -- 172.21.15.114:0/1120870418 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f715c060c60 con 0x7f716415ade0 2024-05-27T09:34:11.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.292+0000 7f716b800640 1 -- 172.21.15.114:0/1120870418 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71641ee110 con 0x7f716415ade0 2024-05-27T09:34:11.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.293+0000 7f7160c00640 1 -- 172.21.15.114:0/1120870418 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+6056 (secure 0 0 0) 0x7f715c064f60 con 0x7f716415ade0 2024-05-27T09:34:11.295 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:34:11.295 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:05.408102+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:replay","state_seq":1,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4875,"qdb_cluster":[4828,4875]},"id":1}]} 2024-05-27T09:34:11.296 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 31 2024-05-27T09:34:11.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.296+0000 7f716b800640 1 -- 172.21.15.114:0/1120870418 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7144076300 msgr2=0x7f7144078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:11.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.296+0000 7f716b800640 1 --2- 172.21.15.114:0/1120870418 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7144076300 0x7f7144078810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7158037000 tx=0x7f715803f040 comp rx=0 tx=0).stop 2024-05-27T09:34:11.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.296+0000 7f716b800640 1 -- 172.21.15.114:0/1120870418 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f716415ade0 msgr2=0x7f716415c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:11.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.296+0000 7f716b800640 1 --2- 172.21.15.114:0/1120870418 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f716415ade0 0x7f716415c4b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f71640c8a60 tx=0x7f715c00ce40 comp rx=0 tx=0).stop 2024-05-27T09:34:11.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.296+0000 7f716b800640 1 -- 172.21.15.114:0/1120870418 shutdown_connections 2024-05-27T09:34:11.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.296+0000 7f716b800640 1 --2- 172.21.15.114:0/1120870418 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7144076300 0x7f7144078810 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:11.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.296+0000 7f716b800640 1 --2- 172.21.15.114:0/1120870418 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f716415ade0 0x7f716415c4b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:11.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.296+0000 7f716b800640 1 --2- 172.21.15.114:0/1120870418 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71641593b0 0x7f71641ee460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:11.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.296+0000 7f716b800640 1 --2- 172.21.15.114:0/1120870418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7164073260 0x7f716415ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:11.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.296+0000 7f716b800640 1 -- 172.21.15.114:0/1120870418 >> 172.21.15.114:0/1120870418 conn(0x7f716406f6e0 msgr2=0x7f716413f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:11.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.296+0000 7f716b800640 1 -- 172.21.15.114:0/1120870418 shutdown_connections 2024-05-27T09:34:11.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:11.296+0000 7f716b800640 1 -- 172.21.15.114:0/1120870418 wait complete. 2024-05-27T09:34:11.322 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:34:13.323 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:34:13.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.609+0000 7f2688400640 1 Processor -- start 2024-05-27T09:34:13.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.609+0000 7f2688400640 1 -- start start 2024-05-27T09:34:13.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.611+0000 7f2688400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f26801593b0 0x7f2680142720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:13.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.611+0000 7f2688400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2680073260 0x7f268015a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:13.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.611+0000 7f2688400640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26800739a0 con 0x7f268015ade0 2024-05-27T09:34:13.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.611+0000 7f2688400640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f2680074d20 con 0x7f26801593b0 2024-05-27T09:34:13.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.611+0000 7f2688400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f2680073c80 con 0x7f2680073260 2024-05-27T09:34:13.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.612+0000 7f2685a00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f26801593b0 0x7f2680142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:13.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.612+0000 7f2685a00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f26801593b0 0x7f2680142720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:46720/0 (socket says 172.21.15.114:46720) 2024-05-27T09:34:13.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.612+0000 7f2685a00640 1 -- 172.21.15.114:0/1184142447 learned_addr learned my addr 172.21.15.114:0/1184142447 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:34:13.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.612+0000 7f2685000640 1 --2- 172.21.15.114:0/1184142447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2680073260 0x7f268015a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:13.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.612+0000 7f2685a00640 1 -- 172.21.15.114:0/1184142447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2680073260 msgr2=0x7f268015a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:34:13.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.612+0000 7f2685a00640 1 --2- 172.21.15.114:0/1184142447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2680073260 0x7f268015a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:13.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.612+0000 7f2685a00640 1 -- 172.21.15.114:0/1184142447 >> v1:172.21.15.163:6789/0 conn(0x7f268015ade0 legacy=0x7f26800c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:13.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.612+0000 7f2685a00640 1 -- 172.21.15.114:0/1184142447 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2680073f60 con 0x7f26801593b0 2024-05-27T09:34:13.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.612+0000 7f2685000640 1 --2- 172.21.15.114:0/1184142447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2680073260 0x7f268015a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:34:13.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.613+0000 7f2685a00640 1 --2- 172.21.15.114:0/1184142447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26801593b0 0x7f2680142720 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2674009870 tx=0x7f267402a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=599f531eef98467d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:13.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.614+0000 7f267fe00640 1 -- 172.21.15.114:0/1184142447 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674037070 con 0x7f26801593b0 2024-05-27T09:34:13.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.614+0000 7f267fe00640 1 -- 172.21.15.114:0/1184142447 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f267402ae50 con 0x7f26801593b0 2024-05-27T09:34:13.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.614+0000 7f267fe00640 1 -- 172.21.15.114:0/1184142447 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674032860 con 0x7f26801593b0 2024-05-27T09:34:13.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.614+0000 7f2688400640 1 -- 172.21.15.114:0/1184142447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26801593b0 msgr2=0x7f2680142720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:13.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.614+0000 7f2688400640 1 --2- 172.21.15.114:0/1184142447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26801593b0 0x7f2680142720 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2674009870 tx=0x7f267402a300 comp rx=0 tx=0).stop 2024-05-27T09:34:13.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.614+0000 7f2688400640 1 -- 172.21.15.114:0/1184142447 shutdown_connections 2024-05-27T09:34:13.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.614+0000 7f2688400640 1 --2- 172.21.15.114:0/1184142447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2680073260 0x7f268015a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:13.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.615+0000 7f2688400640 1 --2- 172.21.15.114:0/1184142447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26801593b0 0x7f2680142720 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:13.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.615+0000 7f2688400640 1 -- 172.21.15.114:0/1184142447 >> 172.21.15.114:0/1184142447 conn(0x7f268006f6e0 msgr2=0x7f2680070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:13.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.615+0000 7f2688400640 1 -- 172.21.15.114:0/1184142447 shutdown_connections 2024-05-27T09:34:13.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.615+0000 7f2688400640 1 -- 172.21.15.114:0/1184142447 wait complete. 2024-05-27T09:34:13.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.617+0000 7f2688400640 1 Processor -- start 2024-05-27T09:34:13.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.617+0000 7f2688400640 1 -- start start 2024-05-27T09:34:13.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.619+0000 7f2688400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2680073260 0x7f268015b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:13.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.619+0000 7f2688400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26801593b0 0x7f26800b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:13.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.619+0000 7f2688400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f268015ade0 0x7f26800cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:13.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.619+0000 7f2688400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2680074800 con 0x7f26801593b0 2024-05-27T09:34:13.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.619+0000 7f2688400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2680074520 con 0x7f268015ade0 2024-05-27T09:34:13.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.619+0000 7f2688400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f2680073670 con 0x7f2680073260 2024-05-27T09:34:13.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.619+0000 7f2685000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26801593b0 0x7f26800b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:13.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.619+0000 7f2685000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26801593b0 0x7f26800b2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39348/0 (socket says 172.21.15.114:39348) 2024-05-27T09:34:13.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.619+0000 7f2685000640 1 -- 172.21.15.114:0/2575192487 learned_addr learned my addr 172.21.15.114:0/2575192487 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:34:13.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.619+0000 7f2685a00640 1 --2- 172.21.15.114:0/2575192487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2680073260 0x7f268015b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:13.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.619+0000 7f2686400640 1 --2- 172.21.15.114:0/2575192487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f268015ade0 0x7f26800cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:13.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.620+0000 7f2685000640 1 -- 172.21.15.114:0/2575192487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2680073260 msgr2=0x7f268015b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:13.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.620+0000 7f2685000640 1 --2- 172.21.15.114:0/2575192487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2680073260 0x7f268015b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:13.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.620+0000 7f2685000640 1 -- 172.21.15.114:0/2575192487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f268015ade0 msgr2=0x7f26800cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:13.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.620+0000 7f2685000640 1 --2- 172.21.15.114:0/2575192487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f268015ade0 0x7f26800cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:13.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.620+0000 7f2685000640 1 -- 172.21.15.114:0/2575192487 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26800cce10 con 0x7f26801593b0 2024-05-27T09:34:13.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.620+0000 7f2685000640 1 --2- 172.21.15.114:0/2575192487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26801593b0 0x7f26800b2830 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f266c00e990 tx=0x7f266c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:13.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.620+0000 7f267d600640 1 -- 172.21.15.114:0/2575192487 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c017070 con 0x7f26801593b0 2024-05-27T09:34:13.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.620+0000 7f267d600640 1 -- 172.21.15.114:0/2575192487 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f266c00c9e0 con 0x7f26801593b0 2024-05-27T09:34:13.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.620+0000 7f2688400640 1 -- 172.21.15.114:0/2575192487 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f268015bc90 con 0x7f26801593b0 2024-05-27T09:34:13.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.620+0000 7f267d600640 1 -- 172.21.15.114:0/2575192487 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c00cdb0 con 0x7f26801593b0 2024-05-27T09:34:13.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.620+0000 7f2688400640 1 -- 172.21.15.114:0/2575192487 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26800cd080 con 0x7f26801593b0 2024-05-27T09:34:13.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.621+0000 7f2688400640 1 -- 172.21.15.114:0/2575192487 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26800ca0d0 con 0x7f26801593b0 2024-05-27T09:34:13.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.623+0000 7f267d600640 1 -- 172.21.15.114:0/2575192487 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f266c01a070 con 0x7f26801593b0 2024-05-27T09:34:13.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.624+0000 7f267d600640 1 --2- 172.21.15.114:0/2575192487 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f26600762d0 0x7f26600787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:13.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.624+0000 7f267d600640 1 -- 172.21.15.114:0/2575192487 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6082+0+0 (secure 0 0 0) 0x7f266c097e50 con 0x7f26801593b0 2024-05-27T09:34:13.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.624+0000 7f2685a00640 1 --2- 172.21.15.114:0/2575192487 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f26600762d0 0x7f26600787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:13.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.625+0000 7f2685a00640 1 --2- 172.21.15.114:0/2575192487 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f26600762d0 0x7f26600787e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2674034000 tx=0x7f267402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:13.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.627+0000 7f267d600640 1 -- 172.21.15.114:0/2575192487 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f26800ca0d0 con 0x7f26801593b0 2024-05-27T09:34:13.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.865+0000 7f2688400640 1 -- 172.21.15.114:0/2575192487 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2680069a40 con 0x7f26801593b0 2024-05-27T09:34:13.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.866+0000 7f267d600640 1 -- 172.21.15.114:0/2575192487 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+6058 (secure 0 0 0) 0x7f2680069a40 con 0x7f26801593b0 2024-05-27T09:34:13.868 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:34:13.868 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:12.877341+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:resolve","state_seq":21,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4875,"qdb_cluster":[4875,4828]},"id":1}]} 2024-05-27T09:34:13.868 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 32 2024-05-27T09:34:13.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.869+0000 7f2688400640 1 -- 172.21.15.114:0/2575192487 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f26600762d0 msgr2=0x7f26600787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:13.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.869+0000 7f2688400640 1 --2- 172.21.15.114:0/2575192487 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f26600762d0 0x7f26600787e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2674034000 tx=0x7f267402e000 comp rx=0 tx=0).stop 2024-05-27T09:34:13.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.869+0000 7f2688400640 1 -- 172.21.15.114:0/2575192487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26801593b0 msgr2=0x7f26800b2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:13.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.869+0000 7f2688400640 1 --2- 172.21.15.114:0/2575192487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26801593b0 0x7f26800b2830 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f266c00e990 tx=0x7f266c00ee50 comp rx=0 tx=0).stop 2024-05-27T09:34:13.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.869+0000 7f2688400640 1 -- 172.21.15.114:0/2575192487 shutdown_connections 2024-05-27T09:34:13.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.869+0000 7f2688400640 1 --2- 172.21.15.114:0/2575192487 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f26600762d0 0x7f26600787e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:13.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.869+0000 7f2688400640 1 --2- 172.21.15.114:0/2575192487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f268015ade0 0x7f26800cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:13.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.869+0000 7f2688400640 1 --2- 172.21.15.114:0/2575192487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26801593b0 0x7f26800b2830 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:13.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.869+0000 7f2688400640 1 --2- 172.21.15.114:0/2575192487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2680073260 0x7f268015b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:13.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.869+0000 7f2688400640 1 -- 172.21.15.114:0/2575192487 >> 172.21.15.114:0/2575192487 conn(0x7f268006f6e0 msgr2=0x7f268006c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:13.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.869+0000 7f2688400640 1 -- 172.21.15.114:0/2575192487 shutdown_connections 2024-05-27T09:34:13.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:13.870+0000 7f2688400640 1 -- 172.21.15.114:0/2575192487 wait complete. 2024-05-27T09:34:13.896 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:34:15.897 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:34:16.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.181+0000 7f560d000640 1 Processor -- start 2024-05-27T09:34:16.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.182+0000 7f560d000640 1 -- start start 2024-05-27T09:34:16.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.184+0000 7f560d000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f56080b2c60 0x7f560815d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:16.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.184+0000 7f560d000640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5608156f80 con 0x7f56080ca140 2024-05-27T09:34:16.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.184+0000 7f560d000640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56080cac50 con 0x7f56080ca550 2024-05-27T09:34:16.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.184+0000 7f560d000640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f560815d7c0 con 0x7f56080b2c60 2024-05-27T09:34:16.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.184+0000 7f5605400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f56080ca140 0x7f5608073310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58390/0 (socket says 172.21.15.114:58390) 2024-05-27T09:34:16.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.184+0000 7f5605400640 1 -- 172.21.15.114:0/2266834710 learned_addr learned my addr 172.21.15.114:0/2266834710 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:34:16.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.184+0000 7f5605e00640 1 --2- 172.21.15.114:0/2266834710 >> v2:172.21.15.163:3300/0 conn(0x7f56080b2c60 0x7f560815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:16.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.185+0000 7f5604a00640 1 -- 172.21.15.114:0/2266834710 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2139197921 0 0) 0x7f5608156f80 con 0x7f56080ca140 2024-05-27T09:34:16.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.185+0000 7f5604a00640 1 -- 172.21.15.114:0/2266834710 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55ec003520 con 0x7f56080ca140 2024-05-27T09:34:16.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.185+0000 7f5605e00640 1 -- 172.21.15.114:0/2266834710 >> v1:172.21.15.163:6789/0 conn(0x7f56080ca550 legacy=0x7f56080c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:16.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.185+0000 7f5605e00640 1 -- 172.21.15.114:0/2266834710 >> v1:172.21.15.114:6789/0 conn(0x7f56080ca140 legacy=0x7f5608073310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:16.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.185+0000 7f5605e00640 1 -- 172.21.15.114:0/2266834710 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560815ffa0 con 0x7f56080b2c60 2024-05-27T09:34:16.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.186+0000 7f5605e00640 1 --2- 172.21.15.114:0/2266834710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56080b2c60 0x7f560815d280 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f55f8009d90 tx=0x7f55f802a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b490f9ebe1cec8a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:16.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.186+0000 7f5604a00640 1 -- 172.21.15.114:0/2266834710 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8037070 con 0x7f56080b2c60 2024-05-27T09:34:16.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.186+0000 7f5604a00640 1 -- 172.21.15.114:0/2266834710 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f8007740 con 0x7f56080b2c60 2024-05-27T09:34:16.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.186+0000 7f5604a00640 1 -- 172.21.15.114:0/2266834710 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8007b10 con 0x7f56080b2c60 2024-05-27T09:34:16.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.187+0000 7f560d000640 1 -- 172.21.15.114:0/2266834710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56080b2c60 msgr2=0x7f560815d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:16.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.187+0000 7f560d000640 1 --2- 172.21.15.114:0/2266834710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56080b2c60 0x7f560815d280 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f55f8009d90 tx=0x7f55f802a780 comp rx=0 tx=0).stop 2024-05-27T09:34:16.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.187+0000 7f560d000640 1 -- 172.21.15.114:0/2266834710 shutdown_connections 2024-05-27T09:34:16.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.187+0000 7f560d000640 1 --2- 172.21.15.114:0/2266834710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56080b2c60 0x7f560815d280 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:16.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.187+0000 7f560d000640 1 -- 172.21.15.114:0/2266834710 >> 172.21.15.114:0/2266834710 conn(0x7f560806f6e0 msgr2=0x7f5608070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:16.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.187+0000 7f560d000640 1 -- 172.21.15.114:0/2266834710 shutdown_connections 2024-05-27T09:34:16.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.187+0000 7f560d000640 1 -- 172.21.15.114:0/2266834710 wait complete. 2024-05-27T09:34:16.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.189+0000 7f560d000640 1 Processor -- start 2024-05-27T09:34:16.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.190+0000 7f560d000640 1 -- start start 2024-05-27T09:34:16.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.191+0000 7f560d000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56080b2c60 0x7f56081528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:16.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.191+0000 7f560d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56080ca140 0x7f56080b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:16.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.191+0000 7f560d000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56080ca550 0x7f56081f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:16.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.191+0000 7f560d000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5608160560 con 0x7f56080ca140 2024-05-27T09:34:16.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.191+0000 7f560d000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f560815fd50 con 0x7f56080b2c60 2024-05-27T09:34:16.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.191+0000 7f560d000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f5608160840 con 0x7f56080ca550 2024-05-27T09:34:16.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.191+0000 7f5605400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56080ca140 0x7f56080b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:16.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.191+0000 7f5605400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56080ca140 0x7f56080b0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60944/0 (socket says 172.21.15.114:60944) 2024-05-27T09:34:16.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.191+0000 7f5605400640 1 -- 172.21.15.114:0/2460197530 learned_addr learned my addr 172.21.15.114:0/2460197530 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:34:16.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.192+0000 7f5605e00640 1 --2- 172.21.15.114:0/2460197530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56080b2c60 0x7f56081528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:16.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.192+0000 7f5606800640 1 --2- 172.21.15.114:0/2460197530 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56080ca550 0x7f56081f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:16.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.192+0000 7f5605400640 1 -- 172.21.15.114:0/2460197530 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56080ca550 msgr2=0x7f56081f37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:16.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.192+0000 7f5605400640 1 --2- 172.21.15.114:0/2460197530 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56080ca550 0x7f56081f37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:16.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.192+0000 7f5605400640 1 -- 172.21.15.114:0/2460197530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56080b2c60 msgr2=0x7f56081528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:16.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.192+0000 7f5605400640 1 --2- 172.21.15.114:0/2460197530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56080b2c60 0x7f56081528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:16.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.192+0000 7f5605400640 1 -- 172.21.15.114:0/2460197530 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56081f3f10 con 0x7f56080ca140 2024-05-27T09:34:16.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.192+0000 7f5605400640 1 --2- 172.21.15.114:0/2460197530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56080ca140 0x7f56080b0e90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f55f400c390 tx=0x7f55f400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:16.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.193+0000 7f55fe000640 1 -- 172.21.15.114:0/2460197530 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4017070 con 0x7f56080ca140 2024-05-27T09:34:16.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.193+0000 7f55fe000640 1 -- 172.21.15.114:0/2460197530 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f4005d00 con 0x7f56080ca140 2024-05-27T09:34:16.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.193+0000 7f55fe000640 1 -- 172.21.15.114:0/2460197530 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4007310 con 0x7f56080ca140 2024-05-27T09:34:16.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.193+0000 7f560d000640 1 -- 172.21.15.114:0/2460197530 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56081f3d00 con 0x7f56080ca140 2024-05-27T09:34:16.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.193+0000 7f560d000640 1 -- 172.21.15.114:0/2460197530 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56081f2fe0 con 0x7f56080ca140 2024-05-27T09:34:16.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.194+0000 7f560d000640 1 -- 172.21.15.114:0/2460197530 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56080715f0 con 0x7f56080ca140 2024-05-27T09:34:16.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.195+0000 7f55fe000640 1 -- 172.21.15.114:0/2460197530 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f55f401a070 con 0x7f56080ca140 2024-05-27T09:34:16.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.196+0000 7f55fe000640 1 --2- 172.21.15.114:0/2460197530 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f55e80762d0 0x7f55e80787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:16.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.196+0000 7f55fe000640 1 -- 172.21.15.114:0/2460197530 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6082+0+0 (secure 0 0 0) 0x7f55f4096ac0 con 0x7f56080ca140 2024-05-27T09:34:16.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.197+0000 7f5605e00640 1 --2- 172.21.15.114:0/2460197530 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f55e80762d0 0x7f55e80787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:16.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.197+0000 7f5605e00640 1 --2- 172.21.15.114:0/2460197530 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f55e80762d0 0x7f55e80787e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f55f8034000 tx=0x7f55f802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:16.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.201+0000 7f55fe000640 1 -- 172.21.15.114:0/2460197530 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f56080715f0 con 0x7f56080ca140 2024-05-27T09:34:16.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.443+0000 7f560d000640 1 -- 172.21.15.114:0/2460197530 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f560806a380 con 0x7f56080ca140 2024-05-27T09:34:16.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.444+0000 7f55fe000640 1 -- 172.21.15.114:0/2460197530 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+6058 (secure 0 0 0) 0x7f56081f3f10 con 0x7f56080ca140 2024-05-27T09:34:16.446 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:34:16.446 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:12.877341+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:resolve","state_seq":21,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4875,"qdb_cluster":[4875,4828]},"id":1}]} 2024-05-27T09:34:16.446 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 32 2024-05-27T09:34:16.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.447+0000 7f560d000640 1 -- 172.21.15.114:0/2460197530 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f55e80762d0 msgr2=0x7f55e80787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:16.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.447+0000 7f560d000640 1 --2- 172.21.15.114:0/2460197530 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f55e80762d0 0x7f55e80787e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f55f8034000 tx=0x7f55f802e000 comp rx=0 tx=0).stop 2024-05-27T09:34:16.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.447+0000 7f560d000640 1 -- 172.21.15.114:0/2460197530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56080ca140 msgr2=0x7f56080b0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:16.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.447+0000 7f560d000640 1 --2- 172.21.15.114:0/2460197530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56080ca140 0x7f56080b0e90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f55f400c390 tx=0x7f55f400c850 comp rx=0 tx=0).stop 2024-05-27T09:34:16.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.447+0000 7f560d000640 1 -- 172.21.15.114:0/2460197530 shutdown_connections 2024-05-27T09:34:16.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.447+0000 7f560d000640 1 --2- 172.21.15.114:0/2460197530 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f55e80762d0 0x7f55e80787e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:16.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.447+0000 7f560d000640 1 --2- 172.21.15.114:0/2460197530 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56080ca550 0x7f56081f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:16.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.447+0000 7f560d000640 1 --2- 172.21.15.114:0/2460197530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f56080ca140 0x7f56080b0e90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:16.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.447+0000 7f560d000640 1 --2- 172.21.15.114:0/2460197530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56080b2c60 0x7f56081528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:16.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.447+0000 7f560d000640 1 -- 172.21.15.114:0/2460197530 >> 172.21.15.114:0/2460197530 conn(0x7f560806f6e0 msgr2=0x7f56080c7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:16.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.448+0000 7f560d000640 1 -- 172.21.15.114:0/2460197530 shutdown_connections 2024-05-27T09:34:16.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:16.448+0000 7f560d000640 1 -- 172.21.15.114:0/2460197530 wait complete. 2024-05-27T09:34:16.475 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:34:18.477 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:34:18.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.762+0000 7f1ed5e00640 1 Processor -- start 2024-05-27T09:34:18.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.762+0000 7f1ed5e00640 1 -- start start 2024-05-27T09:34:18.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.764+0000 7f1ed5e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1ed00741f0 0x7f1ed00c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:18.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.764+0000 7f1ed5e00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ed006fe50 con 0x7f1ed0074840 2024-05-27T09:34:18.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.764+0000 7f1ed5e00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ed00700f0 con 0x7f1ed01547d0 2024-05-27T09:34:18.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.764+0000 7f1ed5e00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f1ed00703d0 con 0x7f1ed00741f0 2024-05-27T09:34:18.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.764+0000 7f1ed4a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1ed00741f0 0x7f1ed00c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:18.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.764+0000 7f1ed4a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f1ed00741f0 0x7f1ed00c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60948/0 (socket says 172.21.15.114:60948) 2024-05-27T09:34:18.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.764+0000 7f1ed4a00640 1 -- 172.21.15.114:0/2834241361 learned_addr learned my addr 172.21.15.114:0/2834241361 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:34:18.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.765+0000 7f1ed4a00640 1 -- 172.21.15.114:0/2834241361 >> v1:172.21.15.163:6789/0 conn(0x7f1ed01547d0 legacy=0x7f1ed006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:18.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.765+0000 7f1ed4a00640 1 -- 172.21.15.114:0/2834241361 >> v1:172.21.15.114:6789/0 conn(0x7f1ed0074840 legacy=0x7f1ed0164b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:18.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.765+0000 7f1ed4a00640 1 -- 172.21.15.114:0/2834241361 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed00731d0 con 0x7f1ed00741f0 2024-05-27T09:34:18.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.766+0000 7f1ed4a00640 1 --2- 172.21.15.114:0/2834241361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ed00741f0 0x7f1ed00c63a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0009870 tx=0x7f1ec002a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532a0fb810350f17 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:18.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.766+0000 7f1ecfe00640 1 -- 172.21.15.114:0/2834241361 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0037070 con 0x7f1ed00741f0 2024-05-27T09:34:18.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.766+0000 7f1ecfe00640 1 -- 172.21.15.114:0/2834241361 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec0032640 con 0x7f1ed00741f0 2024-05-27T09:34:18.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.766+0000 7f1ecfe00640 1 -- 172.21.15.114:0/2834241361 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0032a10 con 0x7f1ed00741f0 2024-05-27T09:34:18.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.767+0000 7f1ed5e00640 1 -- 172.21.15.114:0/2834241361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ed00741f0 msgr2=0x7f1ed00c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:18.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.767+0000 7f1ed5e00640 1 --2- 172.21.15.114:0/2834241361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ed00741f0 0x7f1ed00c63a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0009870 tx=0x7f1ec002a3c0 comp rx=0 tx=0).stop 2024-05-27T09:34:18.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.767+0000 7f1ed5e00640 1 -- 172.21.15.114:0/2834241361 shutdown_connections 2024-05-27T09:34:18.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.767+0000 7f1ed5e00640 1 --2- 172.21.15.114:0/2834241361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ed00741f0 0x7f1ed00c63a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:18.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.767+0000 7f1ed5e00640 1 -- 172.21.15.114:0/2834241361 >> 172.21.15.114:0/2834241361 conn(0x7f1ed00ca320 msgr2=0x7f1ed00ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:18.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.767+0000 7f1ed5e00640 1 -- 172.21.15.114:0/2834241361 shutdown_connections 2024-05-27T09:34:18.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.767+0000 7f1ed5e00640 1 -- 172.21.15.114:0/2834241361 wait complete. 2024-05-27T09:34:18.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.769+0000 7f1ed5e00640 1 Processor -- start 2024-05-27T09:34:18.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.770+0000 7f1ed5e00640 1 -- start start 2024-05-27T09:34:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.771+0000 7f1ed5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ed00741f0 0x7f1ed01f0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.771+0000 7f1ed5e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ed0074840 0x7f1ed014a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.771+0000 7f1ed5e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ed01547d0 0x7f1ed00cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.771+0000 7f1ed5e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed006f500 con 0x7f1ed01547d0 2024-05-27T09:34:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.771+0000 7f1ed5e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed0073790 con 0x7f1ed00741f0 2024-05-27T09:34:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.771+0000 7f1ed5e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed006f680 con 0x7f1ed0074840 2024-05-27T09:34:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.771+0000 7f1ed4a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ed00741f0 0x7f1ed01f0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.771+0000 7f1ed4a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ed00741f0 0x7f1ed01f0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:54628/0 (socket says 172.21.15.114:54628) 2024-05-27T09:34:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.771+0000 7f1ed4a00640 1 -- 172.21.15.114:0/2134044991 learned_addr learned my addr 172.21.15.114:0/2134044991 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:34:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.771+0000 7f1ed5400640 1 --2- 172.21.15.114:0/2134044991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ed01547d0 0x7f1ed00cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:18.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.771+0000 7f1ec7e00640 1 --2- 172.21.15.114:0/2134044991 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ed0074840 0x7f1ed014a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:18.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.772+0000 7f1ed4a00640 1 -- 172.21.15.114:0/2134044991 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ed0074840 msgr2=0x7f1ed014a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:18.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.772+0000 7f1ed4a00640 1 --2- 172.21.15.114:0/2134044991 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ed0074840 0x7f1ed014a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:18.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.772+0000 7f1ed4a00640 1 -- 172.21.15.114:0/2134044991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ed01547d0 msgr2=0x7f1ed00cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:18.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.772+0000 7f1ed4a00640 1 --2- 172.21.15.114:0/2134044991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ed01547d0 0x7f1ed00cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:18.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.772+0000 7f1ed4a00640 1 -- 172.21.15.114:0/2134044991 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed01f1200 con 0x7f1ed00741f0 2024-05-27T09:34:18.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.772+0000 7f1ed4a00640 1 --2- 172.21.15.114:0/2134044991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ed00741f0 0x7f1ed01f0cc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0000c00 tx=0x7f1ec00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:18.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.772+0000 7f1ecd600640 1 -- 172.21.15.114:0/2134044991 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0037040 con 0x7f1ed00741f0 2024-05-27T09:34:18.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.773+0000 7f1ecd600640 1 -- 172.21.15.114:0/2134044991 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec0042070 con 0x7f1ed00741f0 2024-05-27T09:34:18.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.773+0000 7f1ecd600640 1 -- 172.21.15.114:0/2134044991 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0030bd0 con 0x7f1ed00741f0 2024-05-27T09:34:18.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.773+0000 7f1ed5e00640 1 -- 172.21.15.114:0/2134044991 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed01f41b0 con 0x7f1ed00741f0 2024-05-27T09:34:18.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.773+0000 7f1ed5e00640 1 -- 172.21.15.114:0/2134044991 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed01f4770 con 0x7f1ed00741f0 2024-05-27T09:34:18.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.774+0000 7f1ed5e00640 1 -- 172.21.15.114:0/2134044991 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed0050860 con 0x7f1ed00741f0 2024-05-27T09:34:18.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.775+0000 7f1ecd600640 1 -- 172.21.15.114:0/2134044991 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1ec0032500 con 0x7f1ed00741f0 2024-05-27T09:34:18.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.776+0000 7f1ecd600640 1 --2- 172.21.15.114:0/2134044991 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1eac076300 0x7f1eac078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:18.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.776+0000 7f1ecd600640 1 -- 172.21.15.114:0/2134044991 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6082+0+0 (secure 0 0 0) 0x7f1ec00b8240 con 0x7f1ed00741f0 2024-05-27T09:34:18.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.776+0000 7f1ec7e00640 1 --2- 172.21.15.114:0/2134044991 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1eac076300 0x7f1eac078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:18.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.777+0000 7f1ec7e00640 1 --2- 172.21.15.114:0/2134044991 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1eac076300 0x7f1eac078810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1ed01ee1c0 tx=0x7f1eb8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:18.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:18.781+0000 7f1ecd600640 1 -- 172.21.15.114:0/2134044991 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1ec00812a0 con 0x7f1ed00741f0 2024-05-27T09:34:19.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:19.028+0000 7f1ed5e00640 1 -- 172.21.15.114:0/2134044991 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ed014e520 con 0x7f1ed00741f0 2024-05-27T09:34:19.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:19.030+0000 7f1ecd600640 1 -- 172.21.15.114:0/2134044991 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+6058 (secure 0 0 0) 0x7f1ec00855a0 con 0x7f1ed00741f0 2024-05-27T09:34:19.031 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:34:19.032 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:12.877341+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:resolve","state_seq":21,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4875,"qdb_cluster":[4875,4828]},"id":1}]} 2024-05-27T09:34:19.032 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 32 2024-05-27T09:34:19.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:19.033+0000 7f1ed5e00640 1 -- 172.21.15.114:0/2134044991 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1eac076300 msgr2=0x7f1eac078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:19.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:19.033+0000 7f1ed5e00640 1 --2- 172.21.15.114:0/2134044991 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1eac076300 0x7f1eac078810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1ed01ee1c0 tx=0x7f1eb8009040 comp rx=0 tx=0).stop 2024-05-27T09:34:19.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:19.033+0000 7f1ed5e00640 1 -- 172.21.15.114:0/2134044991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ed00741f0 msgr2=0x7f1ed01f0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:19.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:19.033+0000 7f1ed5e00640 1 --2- 172.21.15.114:0/2134044991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ed00741f0 0x7f1ed01f0cc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0000c00 tx=0x7f1ec00046c0 comp rx=0 tx=0).stop 2024-05-27T09:34:19.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:19.033+0000 7f1ed5e00640 1 -- 172.21.15.114:0/2134044991 shutdown_connections 2024-05-27T09:34:19.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:19.033+0000 7f1ed5e00640 1 --2- 172.21.15.114:0/2134044991 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1eac076300 0x7f1eac078810 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:19.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:19.033+0000 7f1ed5e00640 1 --2- 172.21.15.114:0/2134044991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ed01547d0 0x7f1ed00cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:19.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:19.033+0000 7f1ed5e00640 1 --2- 172.21.15.114:0/2134044991 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ed0074840 0x7f1ed014a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:19.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:19.033+0000 7f1ed5e00640 1 --2- 172.21.15.114:0/2134044991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ed00741f0 0x7f1ed01f0cc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:19.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:19.033+0000 7f1ed5e00640 1 -- 172.21.15.114:0/2134044991 >> 172.21.15.114:0/2134044991 conn(0x7f1ed00ca320 msgr2=0x7f1ed00c6330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:19.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:19.033+0000 7f1ed5e00640 1 -- 172.21.15.114:0/2134044991 shutdown_connections 2024-05-27T09:34:19.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:19.033+0000 7f1ed5e00640 1 -- 172.21.15.114:0/2134044991 wait complete. 2024-05-27T09:34:19.060 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 32, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-05-27T09:27:18.079214+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 32, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4828": { "addr": "172.21.15.163:6839/3371890103", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6837", "nonce": 3371890103, "type": "v2" }, { "addr": "172.21.15.163:6839", "nonce": 3371890103, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4828, "incarnation": 19, "join_fscid": -1, "name": "f", "rank": 2, "state": "up:active", "state_seq": 20 }, "gid_4875": { "addr": "172.21.15.114:6835/2747052034", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6833", "nonce": 2747052034, "type": "v2" }, { "addr": "172.21.15.114:6835", "nonce": 2747052034, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4875, "incarnation": 25, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 25 }, "gid_5086": { "addr": "172.21.15.114:6834/1371544448", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6832", "nonce": 1371544448, "type": "v2" }, { "addr": "172.21.15.114:6834", "nonce": 1371544448, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5086, "incarnation": 31, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:resolve", "state_seq": 21 } }, "last_failure": 0, "last_failure_osd_epoch": 48, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-05-27T09:34:12.877341+0000", "qdb_cluster": [ 4875, 4828 ], "qdb_leader": 4875, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5086, "mds_1": 4875, "mds_2": 4828 } } } ], "standbys": [ { "addr": "172.21.15.163:6835/3513522318", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 3513522318, "type": "v2" }, { "addr": "172.21.15.163:6835", "nonce": 3513522318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6838/1753596112", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6836", "nonce": 1753596112, "type": "v2" }, { "addr": "172.21.15.163:6838", "nonce": 1753596112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-05-27T09:34:19.288 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:34:19.296 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:34:21.061 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:34:21.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.341+0000 7f39af200640 1 Processor -- start 2024-05-27T09:34:21.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.342+0000 7f39af200640 1 -- start start 2024-05-27T09:34:21.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.344+0000 7f39af200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f39a81593b0 0x7f39a8142720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:21.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.344+0000 7f39af200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39a80739a0 con 0x7f39a80c2990 2024-05-27T09:34:21.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.344+0000 7f39af200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39a8073c80 con 0x7f39a8073260 2024-05-27T09:34:21.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.344+0000 7f39af200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f39a8074d20 con 0x7f39a81593b0 2024-05-27T09:34:21.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.344+0000 7f39a7e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f39a81593b0 0x7f39a8142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:21.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.344+0000 7f39a7e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f39a81593b0 0x7f39a8142720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60964/0 (socket says 172.21.15.114:60964) 2024-05-27T09:34:21.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.344+0000 7f39a7e00640 1 -- 172.21.15.114:0/1163908142 learned_addr learned my addr 172.21.15.114:0/1163908142 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:34:21.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.344+0000 7f39ad200640 1 --1- 172.21.15.114:0/1163908142 >> v1:172.21.15.114:6789/0 conn(0x7f39a80c2990 0x7f39a8163d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58410/0 (socket says 172.21.15.114:58410) 2024-05-27T09:34:21.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.345+0000 7f39a7e00640 1 -- 172.21.15.114:0/1163908142 >> v1:172.21.15.163:6789/0 conn(0x7f39a8073260 legacy=0x7f39a815a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:21.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.345+0000 7f39a7e00640 1 -- 172.21.15.114:0/1163908142 >> v1:172.21.15.114:6789/0 conn(0x7f39a80c2990 legacy=0x7f39a8163d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:21.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.345+0000 7f39a7400640 1 -- 172.21.15.114:0/1163908142 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2939218788 0 0) 0x7f39a8073c80 con 0x7f39a8073260 2024-05-27T09:34:21.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.345+0000 7f39a7e00640 1 -- 172.21.15.114:0/1163908142 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a8073f60 con 0x7f39a81593b0 2024-05-27T09:34:21.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.346+0000 7f39a7e00640 1 --2- 172.21.15.114:0/1163908142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39a81593b0 0x7f39a8142720 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3998009870 tx=0x7f399802a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6328bf2083d18a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:21.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.346+0000 7f39a7400640 1 -- 172.21.15.114:0/1163908142 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998037070 con 0x7f39a81593b0 2024-05-27T09:34:21.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.346+0000 7f39a7400640 1 -- 172.21.15.114:0/1163908142 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3998032640 con 0x7f39a81593b0 2024-05-27T09:34:21.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.346+0000 7f39a7400640 1 -- 172.21.15.114:0/1163908142 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998032a10 con 0x7f39a81593b0 2024-05-27T09:34:21.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.347+0000 7f39af200640 1 -- 172.21.15.114:0/1163908142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39a81593b0 msgr2=0x7f39a8142720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:21.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.347+0000 7f39af200640 1 --2- 172.21.15.114:0/1163908142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39a81593b0 0x7f39a8142720 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3998009870 tx=0x7f399802a3c0 comp rx=0 tx=0).stop 2024-05-27T09:34:21.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.347+0000 7f39af200640 1 -- 172.21.15.114:0/1163908142 shutdown_connections 2024-05-27T09:34:21.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.347+0000 7f39af200640 1 --2- 172.21.15.114:0/1163908142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39a81593b0 0x7f39a8142720 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:21.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.347+0000 7f39af200640 1 -- 172.21.15.114:0/1163908142 >> 172.21.15.114:0/1163908142 conn(0x7f39a806f6e0 msgr2=0x7f39a8070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:21.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.347+0000 7f39af200640 1 -- 172.21.15.114:0/1163908142 shutdown_connections 2024-05-27T09:34:21.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.347+0000 7f39af200640 1 -- 172.21.15.114:0/1163908142 wait complete. 2024-05-27T09:34:21.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.349+0000 7f39af200640 1 Processor -- start 2024-05-27T09:34:21.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.349+0000 7f39af200640 1 -- start start 2024-05-27T09:34:21.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.351+0000 7f39af200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39a8073260 0x7f39a81f6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:21.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.351+0000 7f39af200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f39a80c2990 0x7f39a815b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:21.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.351+0000 7f39af200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39a81593b0 0x7f39a813cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:21.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.351+0000 7f39af200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f39a8074800 con 0x7f39a8073260 2024-05-27T09:34:21.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.351+0000 7f39af200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f39a8074520 con 0x7f39a81593b0 2024-05-27T09:34:21.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.351+0000 7f39af200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f39a8073670 con 0x7f39a80c2990 2024-05-27T09:34:21.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.351+0000 7f39a7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39a8073260 0x7f39a81f6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:21.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.351+0000 7f39a7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39a8073260 0x7f39a81f6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60972/0 (socket says 172.21.15.114:60972) 2024-05-27T09:34:21.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.351+0000 7f39a7e00640 1 -- 172.21.15.114:0/1902084604 learned_addr learned my addr 172.21.15.114:0/1902084604 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:34:21.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.351+0000 7f39ad200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39a81593b0 0x7f39a813cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:21.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.351+0000 7f399f400640 1 --2- 172.21.15.114:0/1902084604 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f39a80c2990 0x7f39a815b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:21.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.352+0000 7f39a7e00640 1 -- 172.21.15.114:0/1902084604 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f39a80c2990 msgr2=0x7f39a815b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:21.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.352+0000 7f39a7e00640 1 --2- 172.21.15.114:0/1902084604 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f39a80c2990 0x7f39a815b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:21.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.352+0000 7f39a7e00640 1 -- 172.21.15.114:0/1902084604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39a81593b0 msgr2=0x7f39a813cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:21.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.352+0000 7f39a7e00640 1 --2- 172.21.15.114:0/1902084604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39a81593b0 0x7f39a813cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:21.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.352+0000 7f39a7e00640 1 -- 172.21.15.114:0/1902084604 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a815b6d0 con 0x7f39a8073260 2024-05-27T09:34:21.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.352+0000 7f39a7e00640 1 --2- 172.21.15.114:0/1902084604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39a8073260 0x7f39a81f6ac0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3998009870 tx=0x7f39980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:21.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.352+0000 7f39a4c00640 1 -- 172.21.15.114:0/1902084604 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998041020 con 0x7f39a8073260 2024-05-27T09:34:21.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.352+0000 7f39a4c00640 1 -- 172.21.15.114:0/1902084604 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3998030990 con 0x7f39a8073260 2024-05-27T09:34:21.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.352+0000 7f39af200640 1 -- 172.21.15.114:0/1902084604 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a8141370 con 0x7f39a8073260 2024-05-27T09:34:21.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.352+0000 7f39a4c00640 1 -- 172.21.15.114:0/1902084604 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998037040 con 0x7f39a8073260 2024-05-27T09:34:21.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.352+0000 7f39af200640 1 -- 172.21.15.114:0/1902084604 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a8141930 con 0x7f39a8073260 2024-05-27T09:34:21.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.353+0000 7f39af200640 1 -- 172.21.15.114:0/1902084604 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a8071a40 con 0x7f39a8073260 2024-05-27T09:34:21.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.355+0000 7f39a4c00640 1 -- 172.21.15.114:0/1902084604 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3998044020 con 0x7f39a8073260 2024-05-27T09:34:21.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.356+0000 7f39a4c00640 1 --2- 172.21.15.114:0/1902084604 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f39840762d0 0x7f39840787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:21.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.356+0000 7f39a4c00640 1 -- 172.21.15.114:0/1902084604 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6082+0+0 (secure 0 0 0) 0x7f39980b8160 con 0x7f39a8073260 2024-05-27T09:34:21.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.357+0000 7f399f400640 1 --2- 172.21.15.114:0/1902084604 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f39840762d0 0x7f39840787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:21.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.357+0000 7f399f400640 1 --2- 172.21.15.114:0/1902084604 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f39840762d0 0x7f39840787e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f39a81f3c70 tx=0x7f3990009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:21.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.361+0000 7f39a4c00640 1 -- 172.21.15.114:0/1902084604 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f39a8071a40 con 0x7f39a8073260 2024-05-27T09:34:21.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.606+0000 7f39af200640 1 -- 172.21.15.114:0/1902084604 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39a806a380 con 0x7f39a8073260 2024-05-27T09:34:21.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.607+0000 7f39a4c00640 1 -- 172.21.15.114:0/1902084604 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+6060 (secure 0 0 0) 0x7f399804b030 con 0x7f39a8073260 2024-05-27T09:34:21.609 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:34:21.609 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:18.130196+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:reconnect","state_seq":23,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4875,"qdb_cluster":[4828,4875]},"id":1}]} 2024-05-27T09:34:21.609 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 33 2024-05-27T09:34:21.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.610+0000 7f39af200640 1 -- 172.21.15.114:0/1902084604 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f39840762d0 msgr2=0x7f39840787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:21.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.610+0000 7f39af200640 1 --2- 172.21.15.114:0/1902084604 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f39840762d0 0x7f39840787e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f39a81f3c70 tx=0x7f3990009040 comp rx=0 tx=0).stop 2024-05-27T09:34:21.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.610+0000 7f39af200640 1 -- 172.21.15.114:0/1902084604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39a8073260 msgr2=0x7f39a81f6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:21.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.610+0000 7f39af200640 1 --2- 172.21.15.114:0/1902084604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39a8073260 0x7f39a81f6ac0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3998009870 tx=0x7f39980046c0 comp rx=0 tx=0).stop 2024-05-27T09:34:21.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.610+0000 7f39af200640 1 -- 172.21.15.114:0/1902084604 shutdown_connections 2024-05-27T09:34:21.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.610+0000 7f39af200640 1 --2- 172.21.15.114:0/1902084604 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f39840762d0 0x7f39840787e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:21.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.610+0000 7f39af200640 1 --2- 172.21.15.114:0/1902084604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39a81593b0 0x7f39a813cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:21.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.610+0000 7f39af200640 1 --2- 172.21.15.114:0/1902084604 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f39a80c2990 0x7f39a815b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:21.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.610+0000 7f39af200640 1 --2- 172.21.15.114:0/1902084604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f39a8073260 0x7f39a81f6ac0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:21.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.610+0000 7f39af200640 1 -- 172.21.15.114:0/1902084604 >> 172.21.15.114:0/1902084604 conn(0x7f39a806f6e0 msgr2=0x7f39a8157ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:21.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.610+0000 7f39af200640 1 -- 172.21.15.114:0/1902084604 shutdown_connections 2024-05-27T09:34:21.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:21.610+0000 7f39af200640 1 -- 172.21.15.114:0/1902084604 wait complete. 2024-05-27T09:34:21.638 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:34:23.639 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:34:23.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.925+0000 7f4cff600640 1 Processor -- start 2024-05-27T09:34:23.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.925+0000 7f4cff600640 1 -- start start 2024-05-27T09:34:23.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.927+0000 7f4cff600640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4cf8074a60 0x7f4cf8154820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:23.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.927+0000 7f4cff600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cf8073260 0x7f4cf8073670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:23.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.928+0000 7f4cff600640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cf815a810 con 0x7f4cf80ca2d0 2024-05-27T09:34:23.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.928+0000 7f4cff600640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f4cf815db60 con 0x7f4cf8074a60 2024-05-27T09:34:23.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.928+0000 7f4cff600640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf815aaf0 con 0x7f4cf8073260 2024-05-27T09:34:23.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.928+0000 7f4cfd600640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f4cf80ca2d0 0x7f4cf80ca6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:58416/0 (socket says 172.21.15.114:58416) 2024-05-27T09:34:23.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.928+0000 7f4cfcc00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f4cf8074a60 0x7f4cf8154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:23.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.928+0000 7f4cfd600640 1 -- 172.21.15.114:0/2458694598 learned_addr learned my addr 172.21.15.114:0/2458694598 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:34:23.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.928+0000 7f4cefe00640 1 --2- 172.21.15.114:0/2458694598 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cf8073260 0x7f4cf8073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:23.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.929+0000 7f4cfcc00640 1 -- 172.21.15.114:0/2458694598 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cf8073260 msgr2=0x7f4cf8073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:34:23.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.929+0000 7f4cfcc00640 1 --2- 172.21.15.114:0/2458694598 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cf8073260 0x7f4cf8073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:23.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.929+0000 7f4cfcc00640 1 -- 172.21.15.114:0/2458694598 >> v1:172.21.15.114:6789/0 conn(0x7f4cf80ca2d0 legacy=0x7f4cf80ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:23.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.929+0000 7f4cfcc00640 1 -- 172.21.15.114:0/2458694598 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf815add0 con 0x7f4cf8074a60 2024-05-27T09:34:23.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.929+0000 7f4cf7e00640 1 -- 172.21.15.114:0/2458694598 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1854521132 0 0) 0x7f4cf815a810 con 0x7f4cf80ca2d0 2024-05-27T09:34:23.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.930+0000 7f4cfcc00640 1 --2- 172.21.15.114:0/2458694598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4cf8074a60 0x7f4cf8154820 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80097a0 tx=0x7f4ce802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8112d645f350b15 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:23.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.930+0000 7f4cf7e00640 1 -- 172.21.15.114:0/2458694598 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8037070 con 0x7f4cf8074a60 2024-05-27T09:34:23.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.930+0000 7f4cf7e00640 1 -- 172.21.15.114:0/2458694598 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce8032440 con 0x7f4cf8074a60 2024-05-27T09:34:23.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.930+0000 7f4cf7e00640 1 -- 172.21.15.114:0/2458694598 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8032810 con 0x7f4cf8074a60 2024-05-27T09:34:23.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.931+0000 7f4cff600640 1 -- 172.21.15.114:0/2458694598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4cf8074a60 msgr2=0x7f4cf8154820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:23.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.931+0000 7f4cff600640 1 --2- 172.21.15.114:0/2458694598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4cf8074a60 0x7f4cf8154820 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80097a0 tx=0x7f4ce802a210 comp rx=0 tx=0).stop 2024-05-27T09:34:23.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.931+0000 7f4cff600640 1 -- 172.21.15.114:0/2458694598 shutdown_connections 2024-05-27T09:34:23.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.931+0000 7f4cff600640 1 --2- 172.21.15.114:0/2458694598 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cf8073260 0x7f4cf8073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:23.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.931+0000 7f4cff600640 1 --2- 172.21.15.114:0/2458694598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4cf8074a60 0x7f4cf8154820 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:23.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.931+0000 7f4cff600640 1 -- 172.21.15.114:0/2458694598 >> 172.21.15.114:0/2458694598 conn(0x7f4cf806f6e0 msgr2=0x7f4cf8070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:23.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.931+0000 7f4cff600640 1 -- 172.21.15.114:0/2458694598 shutdown_connections 2024-05-27T09:34:23.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.931+0000 7f4cff600640 1 -- 172.21.15.114:0/2458694598 wait complete. 2024-05-27T09:34:23.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.933+0000 7f4cff600640 1 Processor -- start 2024-05-27T09:34:23.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.933+0000 7f4cff600640 1 -- start start 2024-05-27T09:34:23.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.935+0000 7f4cff600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cf8073260 0x7f4cf8150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:23.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.935+0000 7f4cff600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4cf8074a60 0x7f4cf81f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:23.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.935+0000 7f4cff600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cf80ca2d0 0x7f4cf8144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:23.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.935+0000 7f4cff600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf815a4e0 con 0x7f4cf8074a60 2024-05-27T09:34:23.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.935+0000 7f4cff600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf815b390 con 0x7f4cf8073260 2024-05-27T09:34:23.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.935+0000 7f4cff600640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf815dff0 con 0x7f4cf80ca2d0 2024-05-27T09:34:23.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.935+0000 7f4cefe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4cf8074a60 0x7f4cf81f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:23.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.935+0000 7f4cefe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4cf8074a60 0x7f4cf81f44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60986/0 (socket says 172.21.15.114:60986) 2024-05-27T09:34:23.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.935+0000 7f4cefe00640 1 -- 172.21.15.114:0/2936352685 learned_addr learned my addr 172.21.15.114:0/2936352685 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:34:23.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.935+0000 7f4cfd600640 1 --2- 172.21.15.114:0/2936352685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cf80ca2d0 0x7f4cf8144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:23.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.935+0000 7f4cfcc00640 1 --2- 172.21.15.114:0/2936352685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cf8073260 0x7f4cf8150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:23.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.936+0000 7f4cefe00640 1 -- 172.21.15.114:0/2936352685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cf80ca2d0 msgr2=0x7f4cf8144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:23.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.936+0000 7f4cefe00640 1 --2- 172.21.15.114:0/2936352685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cf80ca2d0 0x7f4cf8144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:23.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.936+0000 7f4cefe00640 1 -- 172.21.15.114:0/2936352685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cf8073260 msgr2=0x7f4cf8150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:23.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.936+0000 7f4cefe00640 1 --2- 172.21.15.114:0/2936352685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cf8073260 0x7f4cf8150290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:23.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.936+0000 7f4cefe00640 1 -- 172.21.15.114:0/2936352685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf80c7b10 con 0x7f4cf8074a60 2024-05-27T09:34:23.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.936+0000 7f4cfcc00640 1 --2- 172.21.15.114:0/2936352685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cf8073260 0x7f4cf8150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:34:23.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.936+0000 7f4cefe00640 1 --2- 172.21.15.114:0/2936352685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4cf8074a60 0x7f4cf81f44e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000e8a0 tx=0x7f4ce000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:23.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.936+0000 7f4cf5600640 1 -- 172.21.15.114:0/2936352685 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0017070 con 0x7f4cf8074a60 2024-05-27T09:34:23.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.936+0000 7f4cf5600640 1 -- 172.21.15.114:0/2936352685 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce000c940 con 0x7f4cf8074a60 2024-05-27T09:34:23.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.936+0000 7f4cf5600640 1 -- 172.21.15.114:0/2936352685 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce000cd30 con 0x7f4cf8074a60 2024-05-27T09:34:23.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.936+0000 7f4cff600640 1 -- 172.21.15.114:0/2936352685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf81f4a20 con 0x7f4cf8074a60 2024-05-27T09:34:23.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.937+0000 7f4cff600640 1 -- 172.21.15.114:0/2936352685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf80c7550 con 0x7f4cf8074a60 2024-05-27T09:34:23.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.938+0000 7f4cff600640 1 -- 172.21.15.114:0/2936352685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cf80692a0 con 0x7f4cf8074a60 2024-05-27T09:34:23.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.939+0000 7f4cf5600640 1 -- 172.21.15.114:0/2936352685 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4ce001a070 con 0x7f4cf8074a60 2024-05-27T09:34:23.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.940+0000 7f4cf5600640 1 --2- 172.21.15.114:0/2936352685 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4cd40762d0 0x7f4cd40787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:23.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.940+0000 7f4cf5600640 1 -- 172.21.15.114:0/2936352685 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6082+0+0 (secure 0 0 0) 0x7f4ce0097a50 con 0x7f4cf8074a60 2024-05-27T09:34:23.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.940+0000 7f4cfcc00640 1 --2- 172.21.15.114:0/2936352685 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4cd40762d0 0x7f4cd40787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:23.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.941+0000 7f4cfcc00640 1 --2- 172.21.15.114:0/2936352685 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4cd40762d0 0x7f4cd40787e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8034000 tx=0x7f4ce802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:23.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:23.945+0000 7f4cf5600640 1 -- 172.21.15.114:0/2936352685 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4cf80692a0 con 0x7f4cf8074a60 2024-05-27T09:34:24.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:24.193+0000 7f4cff600640 1 -- 172.21.15.114:0/2936352685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4cf806a380 con 0x7f4cf8074a60 2024-05-27T09:34:24.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:24.194+0000 7f4cf5600640 1 -- 172.21.15.114:0/2936352685 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+6057 (secure 0 0 0) 0x7f4ce0060ab0 con 0x7f4cf8074a60 2024-05-27T09:34:24.195 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:34:24.195 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:24.085684+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:rejoin","state_seq":25,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4875,"qdb_cluster":[4875,4828]},"id":1}]} 2024-05-27T09:34:24.195 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 34 2024-05-27T09:34:24.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:24.196+0000 7f4cff600640 1 -- 172.21.15.114:0/2936352685 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4cd40762d0 msgr2=0x7f4cd40787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:24.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:24.196+0000 7f4cff600640 1 --2- 172.21.15.114:0/2936352685 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4cd40762d0 0x7f4cd40787e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8034000 tx=0x7f4ce802e000 comp rx=0 tx=0).stop 2024-05-27T09:34:24.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:24.197+0000 7f4cff600640 1 -- 172.21.15.114:0/2936352685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4cf8074a60 msgr2=0x7f4cf81f44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:24.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:24.197+0000 7f4cff600640 1 --2- 172.21.15.114:0/2936352685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4cf8074a60 0x7f4cf81f44e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000e8a0 tx=0x7f4ce000ed60 comp rx=0 tx=0).stop 2024-05-27T09:34:24.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:24.197+0000 7f4cff600640 1 -- 172.21.15.114:0/2936352685 shutdown_connections 2024-05-27T09:34:24.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:24.197+0000 7f4cff600640 1 --2- 172.21.15.114:0/2936352685 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4cd40762d0 0x7f4cd40787e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:24.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:24.197+0000 7f4cff600640 1 --2- 172.21.15.114:0/2936352685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cf80ca2d0 0x7f4cf8144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:24.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:24.197+0000 7f4cff600640 1 --2- 172.21.15.114:0/2936352685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4cf8074a60 0x7f4cf81f44e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:24.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:24.197+0000 7f4cff600640 1 --2- 172.21.15.114:0/2936352685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cf8073260 0x7f4cf8150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:24.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:24.197+0000 7f4cff600640 1 -- 172.21.15.114:0/2936352685 >> 172.21.15.114:0/2936352685 conn(0x7f4cf806f6e0 msgr2=0x7f4cf8147570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:24.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:24.197+0000 7f4cff600640 1 -- 172.21.15.114:0/2936352685 shutdown_connections 2024-05-27T09:34:24.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:24.197+0000 7f4cff600640 1 -- 172.21.15.114:0/2936352685 wait complete. 2024-05-27T09:34:24.221 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:34:26.222 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:34:26.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.502+0000 7fef89000640 1 Processor -- start 2024-05-27T09:34:26.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.502+0000 7fef89000640 1 -- start start 2024-05-27T09:34:26.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.504+0000 7fef89000640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fef841534c0 0x7fef840c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:26.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.504+0000 7fef89000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef840c8630 0x7fef84074140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:26.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.504+0000 7fef89000640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef840c96d0 con 0x7fef840c8220 2024-05-27T09:34:26.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.504+0000 7fef89000640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fef840c2f40 con 0x7fef841534c0 2024-05-27T09:34:26.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.504+0000 7fef89000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fef84159190 con 0x7fef840c8630 2024-05-27T09:34:26.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.504+0000 7fef83400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fef841534c0 0x7fef840c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:26.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.504+0000 7fef7aa00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fef840c8220 0x7fef84159900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:51596/0 (socket says 172.21.15.114:51596) 2024-05-27T09:34:26.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.504+0000 7fef7aa00640 1 -- 172.21.15.114:0/2036991894 learned_addr learned my addr 172.21.15.114:0/2036991894 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:34:26.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.505+0000 7fef83e00640 1 --2- 172.21.15.114:0/2036991894 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef840c8630 0x7fef84074140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:26.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.505+0000 7fef83400640 1 -- 172.21.15.114:0/2036991894 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef840c8630 msgr2=0x7fef84074140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:34:26.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.505+0000 7fef83400640 1 --2- 172.21.15.114:0/2036991894 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef840c8630 0x7fef84074140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:26.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.505+0000 7fef83400640 1 -- 172.21.15.114:0/2036991894 >> v1:172.21.15.114:6789/0 conn(0x7fef840c8220 legacy=0x7fef84159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:26.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.505+0000 7fef83400640 1 -- 172.21.15.114:0/2036991894 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef840c9970 con 0x7fef841534c0 2024-05-27T09:34:26.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.506+0000 7fef83400640 1 --2- 172.21.15.114:0/2036991894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef841534c0 0x7fef840c2a00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fef740097a0 tx=0x7fef7402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31b01279769576b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:26.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.506+0000 7fef82a00640 1 -- 172.21.15.114:0/2036991894 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74037070 con 0x7fef841534c0 2024-05-27T09:34:26.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.506+0000 7fef82a00640 1 -- 172.21.15.114:0/2036991894 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef74032440 con 0x7fef841534c0 2024-05-27T09:34:26.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.506+0000 7fef82a00640 1 -- 172.21.15.114:0/2036991894 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74032810 con 0x7fef841534c0 2024-05-27T09:34:26.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.507+0000 7fef89000640 1 -- 172.21.15.114:0/2036991894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef841534c0 msgr2=0x7fef840c2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:26.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.507+0000 7fef89000640 1 --2- 172.21.15.114:0/2036991894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef841534c0 0x7fef840c2a00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fef740097a0 tx=0x7fef7402a180 comp rx=0 tx=0).stop 2024-05-27T09:34:26.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.507+0000 7fef89000640 1 -- 172.21.15.114:0/2036991894 shutdown_connections 2024-05-27T09:34:26.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.507+0000 7fef89000640 1 --2- 172.21.15.114:0/2036991894 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef840c8630 0x7fef84074140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:26.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.507+0000 7fef89000640 1 --2- 172.21.15.114:0/2036991894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef841534c0 0x7fef840c2a00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:26.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.507+0000 7fef89000640 1 -- 172.21.15.114:0/2036991894 >> 172.21.15.114:0/2036991894 conn(0x7fef840ca320 msgr2=0x7fef840ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:26.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.507+0000 7fef89000640 1 -- 172.21.15.114:0/2036991894 shutdown_connections 2024-05-27T09:34:26.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.507+0000 7fef89000640 1 -- 172.21.15.114:0/2036991894 wait complete. 2024-05-27T09:34:26.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.510+0000 7fef89000640 1 Processor -- start 2024-05-27T09:34:26.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.510+0000 7fef89000640 1 -- start start 2024-05-27T09:34:26.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.511+0000 7fef89000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef840c8220 0x7fef84156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:26.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.511+0000 7fef89000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef840c8630 0x7fef840cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:26.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.511+0000 7fef89000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef841534c0 0x7fef840abe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:26.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.511+0000 7fef89000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fef840c9f30 con 0x7fef840c8220 2024-05-27T09:34:26.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.511+0000 7fef89000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fef841538d0 con 0x7fef840c8630 2024-05-27T09:34:26.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.511+0000 7fef89000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fef841593e0 con 0x7fef841534c0 2024-05-27T09:34:26.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.511+0000 7fef83400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef840c8220 0x7fef84156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:26.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.511+0000 7fef83400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef840c8220 0x7fef84156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57134/0 (socket says 172.21.15.114:57134) 2024-05-27T09:34:26.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.511+0000 7fef83400640 1 -- 172.21.15.114:0/86889633 learned_addr learned my addr 172.21.15.114:0/86889633 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:34:26.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.512+0000 7fef83e00640 1 --2- 172.21.15.114:0/86889633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef841534c0 0x7fef840abe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:26.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.512+0000 7fef7aa00640 1 --2- 172.21.15.114:0/86889633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef840c8630 0x7fef840cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:26.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.512+0000 7fef83400640 1 -- 172.21.15.114:0/86889633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef841534c0 msgr2=0x7fef840abe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:26.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.512+0000 7fef83400640 1 --2- 172.21.15.114:0/86889633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef841534c0 0x7fef840abe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:26.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.512+0000 7fef83400640 1 -- 172.21.15.114:0/86889633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef840c8630 msgr2=0x7fef840cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:26.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.512+0000 7fef83400640 1 --2- 172.21.15.114:0/86889633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef840c8630 0x7fef840cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:26.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.512+0000 7fef83400640 1 -- 172.21.15.114:0/86889633 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef840ac340 con 0x7fef840c8220 2024-05-27T09:34:26.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.512+0000 7fef83e00640 1 --2- 172.21.15.114:0/86889633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef841534c0 0x7fef840abe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:34:26.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.512+0000 7fef83400640 1 --2- 172.21.15.114:0/86889633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef840c8220 0x7fef84156f30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fef740098c0 tx=0x7fef740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:26.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.513+0000 7fef7be00640 1 -- 172.21.15.114:0/86889633 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74037040 con 0x7fef840c8220 2024-05-27T09:34:26.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.513+0000 7fef7be00640 1 -- 172.21.15.114:0/86889633 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef74042070 con 0x7fef840c8220 2024-05-27T09:34:26.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.513+0000 7fef7be00640 1 -- 172.21.15.114:0/86889633 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74030920 con 0x7fef840c8220 2024-05-27T09:34:26.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.513+0000 7fef89000640 1 -- 172.21.15.114:0/86889633 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef841f3530 con 0x7fef840c8220 2024-05-27T09:34:26.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.513+0000 7fef89000640 1 -- 172.21.15.114:0/86889633 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef840ab600 con 0x7fef840c8220 2024-05-27T09:34:26.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.514+0000 7fef89000640 1 -- 172.21.15.114:0/86889633 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef84050860 con 0x7fef840c8220 2024-05-27T09:34:26.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.515+0000 7fef7be00640 1 -- 172.21.15.114:0/86889633 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fef74030ac0 con 0x7fef840c8220 2024-05-27T09:34:26.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.517+0000 7fef7be00640 1 --2- 172.21.15.114:0/86889633 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fef600762d0 0x7fef600787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:26.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.517+0000 7fef7be00640 1 -- 172.21.15.114:0/86889633 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6082+0+0 (secure 0 0 0) 0x7fef740b8090 con 0x7fef840c8220 2024-05-27T09:34:26.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.517+0000 7fef7aa00640 1 --2- 172.21.15.114:0/86889633 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fef600762d0 0x7fef600787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:26.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.518+0000 7fef7aa00640 1 --2- 172.21.15.114:0/86889633 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fef600762d0 0x7fef600787e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0012e0 tx=0x7fef6c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:26.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.522+0000 7fef7be00640 1 -- 172.21.15.114:0/86889633 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fef740810f0 con 0x7fef840c8220 2024-05-27T09:34:26.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.761+0000 7fef89000640 1 -- 172.21.15.114:0/86889633 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef840c40f0 con 0x7fef840c8220 2024-05-27T09:34:26.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.762+0000 7fef7be00640 1 -- 172.21.15.114:0/86889633 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+6057 (secure 0 0 0) 0x7fef740853f0 con 0x7fef840c8220 2024-05-27T09:34:26.764 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:34:26.764 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:24.085684+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:rejoin","state_seq":25,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4875,"qdb_cluster":[4875,4828]},"id":1}]} 2024-05-27T09:34:26.764 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 34 2024-05-27T09:34:26.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.765+0000 7fef89000640 1 -- 172.21.15.114:0/86889633 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fef600762d0 msgr2=0x7fef600787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:26.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.765+0000 7fef89000640 1 --2- 172.21.15.114:0/86889633 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fef600762d0 0x7fef600787e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0012e0 tx=0x7fef6c009040 comp rx=0 tx=0).stop 2024-05-27T09:34:26.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.765+0000 7fef89000640 1 -- 172.21.15.114:0/86889633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef840c8220 msgr2=0x7fef84156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:26.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.765+0000 7fef89000640 1 --2- 172.21.15.114:0/86889633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef840c8220 0x7fef84156f30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fef740098c0 tx=0x7fef740046c0 comp rx=0 tx=0).stop 2024-05-27T09:34:26.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.765+0000 7fef89000640 1 -- 172.21.15.114:0/86889633 shutdown_connections 2024-05-27T09:34:26.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.765+0000 7fef89000640 1 --2- 172.21.15.114:0/86889633 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fef600762d0 0x7fef600787e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:26.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.765+0000 7fef89000640 1 --2- 172.21.15.114:0/86889633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef841534c0 0x7fef840abe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:26.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.765+0000 7fef89000640 1 --2- 172.21.15.114:0/86889633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef840c8630 0x7fef840cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:26.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.765+0000 7fef89000640 1 --2- 172.21.15.114:0/86889633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef840c8220 0x7fef84156f30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:26.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.765+0000 7fef89000640 1 -- 172.21.15.114:0/86889633 >> 172.21.15.114:0/86889633 conn(0x7fef840ca320 msgr2=0x7fef84073a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:26.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.765+0000 7fef89000640 1 -- 172.21.15.114:0/86889633 shutdown_connections 2024-05-27T09:34:26.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:26.765+0000 7fef89000640 1 -- 172.21.15.114:0/86889633 wait complete. 2024-05-27T09:34:26.790 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:34:28.790 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:34:29.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.072+0000 7f0fc2200640 1 Processor -- start 2024-05-27T09:34:29.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.072+0000 7f0fc2200640 1 -- start start 2024-05-27T09:34:29.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.073+0000 7f0fc2200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0fbc074140 0x7f0fbc074550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:29.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.073+0000 7f0fc2200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fbc0c96d0 con 0x7f0fbc1534c0 2024-05-27T09:34:29.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.073+0000 7f0fc2200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fbc0c8930 con 0x7f0fbc0c2a00 2024-05-27T09:34:29.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.073+0000 7f0fc2200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f0fbc159190 con 0x7f0fbc074140 2024-05-27T09:34:29.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.073+0000 7f0fc0e00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f0fbc1534c0 0x7f0fbc0c8220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:51606/0 (socket says 172.21.15.114:51606) 2024-05-27T09:34:29.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.073+0000 7f0fc0e00640 1 -- 172.21.15.114:0/2283195663 learned_addr learned my addr 172.21.15.114:0/2283195663 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:34:29.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.073+0000 7f0fc1800640 1 --2- 172.21.15.114:0/2283195663 >> v2:172.21.15.163:3300/0 conn(0x7f0fbc074140 0x7f0fbc074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:29.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.074+0000 7f0fbb400640 1 -- 172.21.15.114:0/2283195663 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1343662113 0 0) 0x7f0fbc0c96d0 con 0x7f0fbc1534c0 2024-05-27T09:34:29.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.074+0000 7f0fc1800640 1 -- 172.21.15.114:0/2283195663 >> v1:172.21.15.163:6789/0 conn(0x7f0fbc0c2a00 legacy=0x7f0fbc159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:29.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.074+0000 7f0fc1800640 1 -- 172.21.15.114:0/2283195663 >> v1:172.21.15.114:6789/0 conn(0x7f0fbc1534c0 legacy=0x7f0fbc0c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:29.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.074+0000 7f0fc1800640 1 -- 172.21.15.114:0/2283195663 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc0c9970 con 0x7f0fbc074140 2024-05-27T09:34:29.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.075+0000 7f0fc1800640 1 --2- 172.21.15.114:0/2283195663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fbc074140 0x7f0fbc074550 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4009d90 tx=0x7f0fb402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df7437798ffeecf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:29.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.075+0000 7f0fbb400640 1 -- 172.21.15.114:0/2283195663 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4007740 con 0x7f0fbc074140 2024-05-27T09:34:29.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.075+0000 7f0fbb400640 1 -- 172.21.15.114:0/2283195663 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb40078e0 con 0x7f0fbc074140 2024-05-27T09:34:29.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.075+0000 7f0fbb400640 1 -- 172.21.15.114:0/2283195663 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4007cb0 con 0x7f0fbc074140 2024-05-27T09:34:29.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.076+0000 7f0fc2200640 1 -- 172.21.15.114:0/2283195663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fbc074140 msgr2=0x7f0fbc074550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:29.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.076+0000 7f0fc2200640 1 --2- 172.21.15.114:0/2283195663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fbc074140 0x7f0fbc074550 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4009d90 tx=0x7f0fb402a6f0 comp rx=0 tx=0).stop 2024-05-27T09:34:29.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.076+0000 7f0fc2200640 1 -- 172.21.15.114:0/2283195663 shutdown_connections 2024-05-27T09:34:29.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.076+0000 7f0fc2200640 1 --2- 172.21.15.114:0/2283195663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fbc074140 0x7f0fbc074550 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:29.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.076+0000 7f0fc2200640 1 -- 172.21.15.114:0/2283195663 >> 172.21.15.114:0/2283195663 conn(0x7f0fbc0ca320 msgr2=0x7f0fbc0ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:29.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.076+0000 7f0fc2200640 1 -- 172.21.15.114:0/2283195663 shutdown_connections 2024-05-27T09:34:29.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.076+0000 7f0fc2200640 1 -- 172.21.15.114:0/2283195663 wait complete. 2024-05-27T09:34:29.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.078+0000 7f0fc2200640 1 Processor -- start 2024-05-27T09:34:29.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.079+0000 7f0fc2200640 1 -- start start 2024-05-27T09:34:29.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.080+0000 7f0fc2200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fbc074140 0x7f0fbc1f2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:29.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.080+0000 7f0fc2200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fbc0c2a00 0x7f0fbc06f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:29.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.080+0000 7f0fc2200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0fbc1534c0 0x7f0fbc14e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:29.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.080+0000 7f0fc2200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc0c9f30 con 0x7f0fbc074140 2024-05-27T09:34:29.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.080+0000 7f0fc2200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc1538d0 con 0x7f0fbc0c2a00 2024-05-27T09:34:29.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.080+0000 7f0fc2200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f0fbc1593e0 con 0x7f0fbc1534c0 2024-05-27T09:34:29.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.080+0000 7f0fc0e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fbc074140 0x7f0fbc1f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:29.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.080+0000 7f0fc0e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fbc074140 0x7f0fbc1f2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57140/0 (socket says 172.21.15.114:57140) 2024-05-27T09:34:29.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.080+0000 7f0fc0e00640 1 -- 172.21.15.114:0/2810200410 learned_addr learned my addr 172.21.15.114:0/2810200410 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:34:29.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.080+0000 7f0fbbe00640 1 --2- 172.21.15.114:0/2810200410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fbc0c2a00 0x7f0fbc06f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:29.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.080+0000 7f0fc1800640 1 --2- 172.21.15.114:0/2810200410 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0fbc1534c0 0x7f0fbc14e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:29.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.081+0000 7f0fc0e00640 1 -- 172.21.15.114:0/2810200410 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0fbc1534c0 msgr2=0x7f0fbc14e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:29.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.081+0000 7f0fc0e00640 1 --2- 172.21.15.114:0/2810200410 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0fbc1534c0 0x7f0fbc14e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:29.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.081+0000 7f0fc0e00640 1 -- 172.21.15.114:0/2810200410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fbc0c2a00 msgr2=0x7f0fbc06f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:29.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.081+0000 7f0fc0e00640 1 --2- 172.21.15.114:0/2810200410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fbc0c2a00 0x7f0fbc06f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:29.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.081+0000 7f0fc0e00640 1 -- 172.21.15.114:0/2810200410 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc157b00 con 0x7f0fbc074140 2024-05-27T09:34:29.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.081+0000 7f0fc0e00640 1 --2- 172.21.15.114:0/2810200410 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fbc074140 0x7f0fbc1f2720 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000c3d0 tx=0x7f0fb000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:29.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.081+0000 7f0fb8c00640 1 -- 172.21.15.114:0/2810200410 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb00074e0 con 0x7f0fbc074140 2024-05-27T09:34:29.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.082+0000 7f0fb8c00640 1 -- 172.21.15.114:0/2810200410 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb00047e0 con 0x7f0fbc074140 2024-05-27T09:34:29.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.082+0000 7f0fb8c00640 1 -- 172.21.15.114:0/2810200410 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0004c10 con 0x7f0fbc074140 2024-05-27T09:34:29.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.082+0000 7f0fc2200640 1 -- 172.21.15.114:0/2810200410 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fbc1f5ee0 con 0x7f0fbc074140 2024-05-27T09:34:29.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.082+0000 7f0fc2200640 1 -- 172.21.15.114:0/2810200410 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fbc1f64a0 con 0x7f0fbc074140 2024-05-27T09:34:29.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.083+0000 7f0fc2200640 1 -- 172.21.15.114:0/2810200410 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fbc050860 con 0x7f0fbc074140 2024-05-27T09:34:29.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.084+0000 7f0fb8c00640 1 -- 172.21.15.114:0/2810200410 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0fb0011070 con 0x7f0fbc074140 2024-05-27T09:34:29.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.085+0000 7f0fb8c00640 1 --2- 172.21.15.114:0/2810200410 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0f9c076300 0x7f0f9c078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:29.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.085+0000 7f0fb8c00640 1 -- 172.21.15.114:0/2810200410 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6082+0+0 (secure 0 0 0) 0x7f0fb0097160 con 0x7f0fbc074140 2024-05-27T09:34:29.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.085+0000 7f0fbbe00640 1 --2- 172.21.15.114:0/2810200410 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0f9c076300 0x7f0f9c078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:29.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.086+0000 7f0fbbe00640 1 --2- 172.21.15.114:0/2810200410 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0f9c076300 0x7f0f9c078810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0b2340 tx=0x7f0fa8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:29.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.090+0000 7f0fb8c00640 1 -- 172.21.15.114:0/2810200410 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0fb00602f0 con 0x7f0fbc074140 2024-05-27T09:34:29.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.334+0000 7f0fc2200640 1 -- 172.21.15.114:0/2810200410 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0fbc0c40f0 con 0x7f0fbc074140 2024-05-27T09:34:29.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.335+0000 7f0fb8c00640 1 -- 172.21.15.114:0/2810200410 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+6057 (secure 0 0 0) 0x7f0fb00645f0 con 0x7f0fbc074140 2024-05-27T09:34:29.337 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:34:29.337 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:24.085684+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:rejoin","state_seq":25,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4875,"qdb_cluster":[4875,4828]},"id":1}]} 2024-05-27T09:34:29.337 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 34 2024-05-27T09:34:29.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.338+0000 7f0fc2200640 1 -- 172.21.15.114:0/2810200410 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0f9c076300 msgr2=0x7f0f9c078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:29.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.338+0000 7f0fc2200640 1 --2- 172.21.15.114:0/2810200410 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0f9c076300 0x7f0f9c078810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0b2340 tx=0x7f0fa8009040 comp rx=0 tx=0).stop 2024-05-27T09:34:29.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.338+0000 7f0fc2200640 1 -- 172.21.15.114:0/2810200410 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fbc074140 msgr2=0x7f0fbc1f2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:29.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.338+0000 7f0fc2200640 1 --2- 172.21.15.114:0/2810200410 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fbc074140 0x7f0fbc1f2720 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000c3d0 tx=0x7f0fb000c890 comp rx=0 tx=0).stop 2024-05-27T09:34:29.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.338+0000 7f0fc2200640 1 -- 172.21.15.114:0/2810200410 shutdown_connections 2024-05-27T09:34:29.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.338+0000 7f0fc2200640 1 --2- 172.21.15.114:0/2810200410 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0f9c076300 0x7f0f9c078810 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:29.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.338+0000 7f0fc2200640 1 --2- 172.21.15.114:0/2810200410 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0fbc1534c0 0x7f0fbc14e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:29.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.338+0000 7f0fc2200640 1 --2- 172.21.15.114:0/2810200410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fbc0c2a00 0x7f0fbc06f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:29.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.338+0000 7f0fc2200640 1 --2- 172.21.15.114:0/2810200410 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fbc074140 0x7f0fbc1f2720 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:29.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.338+0000 7f0fc2200640 1 -- 172.21.15.114:0/2810200410 >> 172.21.15.114:0/2810200410 conn(0x7f0fbc0ca320 msgr2=0x7f0fbc0c72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:29.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.338+0000 7f0fc2200640 1 -- 172.21.15.114:0/2810200410 shutdown_connections 2024-05-27T09:34:29.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:29.338+0000 7f0fc2200640 1 -- 172.21.15.114:0/2810200410 wait complete. 2024-05-27T09:34:29.365 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:34:31.366 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:34:31.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.645+0000 7f1bc6800640 1 Processor -- start 2024-05-27T09:34:31.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.645+0000 7f1bc6800640 1 -- start start 2024-05-27T09:34:31.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.647+0000 7f1bc6800640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1bc01547d0 0x7f1bc00c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:31.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.647+0000 7f1bc6800640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bc006fe50 con 0x7f1bc00741f0 2024-05-27T09:34:31.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.647+0000 7f1bc6800640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bc00700f0 con 0x7f1bc0074840 2024-05-27T09:34:31.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.647+0000 7f1bc6800640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f1bc00703d0 con 0x7f1bc01547d0 2024-05-27T09:34:31.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.648+0000 7f1bc5400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f1bc00741f0 0x7f1bc006edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:51608/0 (socket says 172.21.15.114:51608) 2024-05-27T09:34:31.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.648+0000 7f1bc5400640 1 -- 172.21.15.114:0/1048749249 learned_addr learned my addr 172.21.15.114:0/1048749249 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:34:31.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.648+0000 7f1bc4a00640 1 --2- 172.21.15.114:0/1048749249 >> v2:172.21.15.163:3300/0 conn(0x7f1bc01547d0 0x7f1bc00c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:31.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.648+0000 7f1bbfe00640 1 -- 172.21.15.114:0/1048749249 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2963553163 0 0) 0x7f1bc006fe50 con 0x7f1bc00741f0 2024-05-27T09:34:31.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.648+0000 7f1bbfe00640 1 -- 172.21.15.114:0/1048749249 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ba8003520 con 0x7f1bc00741f0 2024-05-27T09:34:31.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.648+0000 7f1bc4a00640 1 -- 172.21.15.114:0/1048749249 >> v1:172.21.15.163:6789/0 conn(0x7f1bc0074840 legacy=0x7f1bc0164b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:31.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.648+0000 7f1bbfe00640 1 -- 172.21.15.114:0/1048749249 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4267681477 0 0) 0x7f1bc00700f0 con 0x7f1bc0074840 2024-05-27T09:34:31.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.648+0000 7f1bc4a00640 1 -- 172.21.15.114:0/1048749249 >> v1:172.21.15.114:6789/0 conn(0x7f1bc00741f0 legacy=0x7f1bc006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:31.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.649+0000 7f1bc4a00640 1 -- 172.21.15.114:0/1048749249 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc00731d0 con 0x7f1bc01547d0 2024-05-27T09:34:31.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.650+0000 7f1bc4a00640 1 --2- 172.21.15.114:0/1048749249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bc01547d0 0x7f1bc00c63a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0097a0 tx=0x7f1bac02a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd92b2bc9de1d567 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:31.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.650+0000 7f1bbfe00640 1 -- 172.21.15.114:0/1048749249 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac037070 con 0x7f1bc01547d0 2024-05-27T09:34:31.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.650+0000 7f1bbfe00640 1 -- 172.21.15.114:0/1048749249 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bac032640 con 0x7f1bc01547d0 2024-05-27T09:34:31.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.650+0000 7f1bbfe00640 1 -- 172.21.15.114:0/1048749249 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac032a10 con 0x7f1bc01547d0 2024-05-27T09:34:31.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.651+0000 7f1bc6800640 1 -- 172.21.15.114:0/1048749249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bc01547d0 msgr2=0x7f1bc00c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:31.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.651+0000 7f1bc6800640 1 --2- 172.21.15.114:0/1048749249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bc01547d0 0x7f1bc00c63a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0097a0 tx=0x7f1bac02a2c0 comp rx=0 tx=0).stop 2024-05-27T09:34:31.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.651+0000 7f1bc6800640 1 -- 172.21.15.114:0/1048749249 shutdown_connections 2024-05-27T09:34:31.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.651+0000 7f1bc6800640 1 --2- 172.21.15.114:0/1048749249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bc01547d0 0x7f1bc00c63a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:31.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.651+0000 7f1bc6800640 1 -- 172.21.15.114:0/1048749249 >> 172.21.15.114:0/1048749249 conn(0x7f1bc00ca320 msgr2=0x7f1bc00ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:31.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.651+0000 7f1bc6800640 1 -- 172.21.15.114:0/1048749249 shutdown_connections 2024-05-27T09:34:31.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.651+0000 7f1bc6800640 1 -- 172.21.15.114:0/1048749249 wait complete. 2024-05-27T09:34:31.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.653+0000 7f1bc6800640 1 Processor -- start 2024-05-27T09:34:31.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.653+0000 7f1bc6800640 1 -- start start 2024-05-27T09:34:31.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.655+0000 7f1bc6800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bc00741f0 0x7f1bc01f3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:31.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.655+0000 7f1bc6800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bc0074840 0x7f1bc0152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:31.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.655+0000 7f1bc6800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1bc01547d0 0x7f1bc0146ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:31.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.655+0000 7f1bc6800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc006f500 con 0x7f1bc01547d0 2024-05-27T09:34:31.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.655+0000 7f1bc6800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc0073790 con 0x7f1bc00741f0 2024-05-27T09:34:31.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.655+0000 7f1bc6800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f1bc006f680 con 0x7f1bc0074840 2024-05-27T09:34:31.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.655+0000 7f1bc5e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1bc01547d0 0x7f1bc0146ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:31.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.655+0000 7f1bc4a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bc0074840 0x7f1bc0152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:31.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.655+0000 7f1bc5400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bc00741f0 0x7f1bc01f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:31.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.655+0000 7f1bc5e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1bc01547d0 0x7f1bc0146ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57146/0 (socket says 172.21.15.114:57146) 2024-05-27T09:34:31.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.655+0000 7f1bc5e00640 1 -- 172.21.15.114:0/1318489720 learned_addr learned my addr 172.21.15.114:0/1318489720 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:34:31.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.656+0000 7f1bc4a00640 1 -- 172.21.15.114:0/1318489720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bc00741f0 msgr2=0x7f1bc01f3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:31.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.656+0000 7f1bc4a00640 1 --2- 172.21.15.114:0/1318489720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bc00741f0 0x7f1bc01f3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:31.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.656+0000 7f1bc4a00640 1 -- 172.21.15.114:0/1318489720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1bc01547d0 msgr2=0x7f1bc0146ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:31.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.656+0000 7f1bc4a00640 1 --2- 172.21.15.114:0/1318489720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1bc01547d0 0x7f1bc0146ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:31.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.656+0000 7f1bc4a00640 1 -- 172.21.15.114:0/1318489720 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc01f3cc0 con 0x7f1bc0074840 2024-05-27T09:34:31.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.656+0000 7f1bc5e00640 1 --2- 172.21.15.114:0/1318489720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1bc01547d0 0x7f1bc0146ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:34:31.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.656+0000 7f1bc4a00640 1 --2- 172.21.15.114:0/1318489720 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bc0074840 0x7f1bc0152260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1bac031000 tx=0x7f1bac0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:31.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.656+0000 7f1bbd600640 1 -- 172.21.15.114:0/1318489720 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac041070 con 0x7f1bc0074840 2024-05-27T09:34:31.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.657+0000 7f1bbd600640 1 -- 172.21.15.114:0/1318489720 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bac004020 con 0x7f1bc0074840 2024-05-27T09:34:31.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.657+0000 7f1bbd600640 1 -- 172.21.15.114:0/1318489720 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac037040 con 0x7f1bc0074840 2024-05-27T09:34:31.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.657+0000 7f1bc6800640 1 -- 172.21.15.114:0/1318489720 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bc01f02d0 con 0x7f1bc0074840 2024-05-27T09:34:31.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.657+0000 7f1bc6800640 1 -- 172.21.15.114:0/1318489720 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bc01f0890 con 0x7f1bc0074840 2024-05-27T09:34:31.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.658+0000 7f1bc6800640 1 -- 172.21.15.114:0/1318489720 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bc01581b0 con 0x7f1bc0074840 2024-05-27T09:34:31.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.659+0000 7f1bbd600640 1 -- 172.21.15.114:0/1318489720 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1bac030770 con 0x7f1bc0074840 2024-05-27T09:34:31.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.660+0000 7f1bbd600640 1 --2- 172.21.15.114:0/1318489720 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1ba00762d0 0x7f1ba00787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:31.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.660+0000 7f1bbd600640 1 -- 172.21.15.114:0/1318489720 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(48..48 src has 1..48) ==== 6082+0+0 (secure 0 0 0) 0x7f1bac0b8900 con 0x7f1bc0074840 2024-05-27T09:34:31.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.660+0000 7f1bc5400640 1 --2- 172.21.15.114:0/1318489720 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1ba00762d0 0x7f1ba00787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:31.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.661+0000 7f1bc5400640 1 --2- 172.21.15.114:0/1318489720 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1ba00762d0 0x7f1ba00787e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4005880 tx=0x7f1bb4006aa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:31.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.664+0000 7f1bbd600640 1 -- 172.21.15.114:0/1318489720 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1bac081c20 con 0x7f1bc0074840 2024-05-27T09:34:31.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.905+0000 7f1bc6800640 1 -- 172.21.15.114:0/1318489720 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1bc014e580 con 0x7f1bc0074840 2024-05-27T09:34:31.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.906+0000 7f1bbd600640 1 -- 172.21.15.114:0/1318489720 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+6062 (secure 0 0 0) 0x7f1bac085f20 con 0x7f1bc0074840 2024-05-27T09:34:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:34:31.907 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:30.127436+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[4828,4875,5086]},"id":1}]} 2024-05-27T09:34:31.907 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 35 2024-05-27T09:34:31.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.909+0000 7f1bc6800640 1 -- 172.21.15.114:0/1318489720 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1ba00762d0 msgr2=0x7f1ba00787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:31.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.909+0000 7f1bc6800640 1 --2- 172.21.15.114:0/1318489720 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1ba00762d0 0x7f1ba00787e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4005880 tx=0x7f1bb4006aa0 comp rx=0 tx=0).stop 2024-05-27T09:34:31.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.909+0000 7f1bc6800640 1 -- 172.21.15.114:0/1318489720 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bc0074840 msgr2=0x7f1bc0152260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:31.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.909+0000 7f1bc6800640 1 --2- 172.21.15.114:0/1318489720 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bc0074840 0x7f1bc0152260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1bac031000 tx=0x7f1bac0046c0 comp rx=0 tx=0).stop 2024-05-27T09:34:31.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.909+0000 7f1bc6800640 1 -- 172.21.15.114:0/1318489720 shutdown_connections 2024-05-27T09:34:31.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.909+0000 7f1bc6800640 1 --2- 172.21.15.114:0/1318489720 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f1ba00762d0 0x7f1ba00787e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:31.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.909+0000 7f1bc6800640 1 --2- 172.21.15.114:0/1318489720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1bc01547d0 0x7f1bc0146ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:31.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.909+0000 7f1bc6800640 1 --2- 172.21.15.114:0/1318489720 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bc0074840 0x7f1bc0152260 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:31.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.909+0000 7f1bc6800640 1 --2- 172.21.15.114:0/1318489720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bc00741f0 0x7f1bc01f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:31.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.909+0000 7f1bc6800640 1 -- 172.21.15.114:0/1318489720 >> 172.21.15.114:0/1318489720 conn(0x7f1bc00ca320 msgr2=0x7f1bc0159380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:31.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.909+0000 7f1bc6800640 1 -- 172.21.15.114:0/1318489720 shutdown_connections 2024-05-27T09:34:31.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:31.909+0000 7f1bc6800640 1 -- 172.21.15.114:0/1318489720 wait complete. 2024-05-27T09:34:31.934 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=0, replacing gid=4300 2024-05-27T09:34:31.935 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 0 secs before reviving mds.e 2024-05-27T09:34:31.935 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2024-05-27T09:34:31.935 INFO:tasks.ceph.mds.e:Restarting daemon 2024-05-27T09:34:31.935 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-05-27T09:34:31.939 INFO:tasks.ceph.mds.e:Started 2024-05-27T09:34:31.940 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:34:32.147 INFO:tasks.ceph.mds.e.smithi114.stdout:starting mds.e at 2024-05-27T09:34:32.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.178+0000 7f5d47e00640 1 Processor -- start 2024-05-27T09:34:32.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.178+0000 7f5d47e00640 1 -- start start 2024-05-27T09:34:32.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.180+0000 7f5d47e00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5d48156980 0x7f5d48160330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:32.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.180+0000 7f5d47e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5d48160870 0x7f5d480cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:32.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.180+0000 7f5d47e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d480cc890 0x7f5d480cb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:32.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.180+0000 7f5d47e00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f5d4806ed40 con 0x7f5d48160870 2024-05-27T09:34:32.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.180+0000 7f5d47e00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f5d4806e1c0 con 0x7f5d48156980 2024-05-27T09:34:32.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.180+0000 7f5d47e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4806e4a0 con 0x7f5d480cc890 2024-05-27T09:34:32.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.180+0000 7f5d3e000640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5d48160870 0x7f5d480cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:32.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.181+0000 7f5d3e000640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5d48160870 0x7f5d480cc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57186/0 (socket says 172.21.15.114:57186) 2024-05-27T09:34:32.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.181+0000 7f5d3e000640 1 -- 172.21.15.114:0/4109372720 learned_addr learned my addr 172.21.15.114:0/4109372720 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:34:32.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.181+0000 7f5d46a00640 1 --2- 172.21.15.114:0/4109372720 >> v2:172.21.15.163:3300/0 conn(0x7f5d48156980 0x7f5d48160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:32.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.181+0000 7f5d47400640 1 --2- 172.21.15.114:0/4109372720 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d480cc890 0x7f5d480cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:32.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.181+0000 7f5d3e000640 1 -- 172.21.15.114:0/4109372720 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d480cc890 msgr2=0x7f5d480cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:34:32.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.181+0000 7f5d3e000640 1 --2- 172.21.15.114:0/4109372720 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d480cc890 0x7f5d480cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:32.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.181+0000 7f5d3e000640 1 -- 172.21.15.114:0/4109372720 >> v2:172.21.15.163:3300/0 conn(0x7f5d48156980 msgr2=0x7f5d48160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:34:32.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.181+0000 7f5d3e000640 1 --2- 172.21.15.114:0/4109372720 >> v2:172.21.15.163:3300/0 conn(0x7f5d48156980 0x7f5d48160330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:32.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.181+0000 7f5d3e000640 1 -- 172.21.15.114:0/4109372720 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4806e780 con 0x7f5d48160870 2024-05-27T09:34:32.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.182+0000 7f5d3e000640 1 --2- 172.21.15.114:0/4109372720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d48160870 0x7f5d480cc350 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5d480d08f0 tx=0x7f5d3002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d164d8b72217e5f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:32.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.182+0000 7f5d46000640 1 -- 172.21.15.114:0/4109372720 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30037070 con 0x7f5d48160870 2024-05-27T09:34:32.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.182+0000 7f5d46000640 1 -- 172.21.15.114:0/4109372720 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d30032440 con 0x7f5d48160870 2024-05-27T09:34:32.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.182+0000 7f5d46000640 1 -- 172.21.15.114:0/4109372720 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30032810 con 0x7f5d48160870 2024-05-27T09:34:32.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.183+0000 7f5d47e00640 1 -- 172.21.15.114:0/4109372720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d48160870 msgr2=0x7f5d480cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:32.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.183+0000 7f5d47e00640 1 --2- 172.21.15.114:0/4109372720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d48160870 0x7f5d480cc350 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5d480d08f0 tx=0x7f5d3002a140 comp rx=0 tx=0).stop 2024-05-27T09:34:32.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.183+0000 7f5d47e00640 1 -- 172.21.15.114:0/4109372720 shutdown_connections 2024-05-27T09:34:32.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.183+0000 7f5d47e00640 1 --2- 172.21.15.114:0/4109372720 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d480cc890 0x7f5d480cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:32.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.183+0000 7f5d47e00640 1 --2- 172.21.15.114:0/4109372720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d48160870 0x7f5d480cc350 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:32.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.183+0000 7f5d47e00640 1 --2- 172.21.15.114:0/4109372720 >> v2:172.21.15.163:3300/0 conn(0x7f5d48156980 0x7f5d48160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:32.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.183+0000 7f5d47e00640 1 -- 172.21.15.114:0/4109372720 >> 172.21.15.114:0/4109372720 conn(0x7f5d48071460 msgr2=0x7f5d48068600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:32.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.183+0000 7f5d47e00640 1 -- 172.21.15.114:0/4109372720 shutdown_connections 2024-05-27T09:34:32.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.183+0000 7f5d47e00640 1 -- 172.21.15.114:0/4109372720 wait complete. 2024-05-27T09:34:32.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.185+0000 7f5d47e00640 1 Processor -- start 2024-05-27T09:34:32.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.185+0000 7f5d47e00640 1 -- start start 2024-05-27T09:34:32.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.187+0000 7f5d47e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d480cc890 0x7f5d481f66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:32.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.187+0000 7f5d47e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d48156980 0x7f5d4814afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:32.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.187+0000 7f5d47e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d48160870 0x7f5d4814a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:32.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.187+0000 7f5d47e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5d48156d90 con 0x7f5d480cc890 2024-05-27T09:34:32.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.187+0000 7f5d47e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4815fe10 con 0x7f5d48160870 2024-05-27T09:34:32.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.187+0000 7f5d47e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f5d48156f10 con 0x7f5d48156980 2024-05-27T09:34:32.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.187+0000 7f5d46a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d480cc890 0x7f5d481f66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:32.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.187+0000 7f5d46a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d480cc890 0x7f5d481f66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57198/0 (socket says 172.21.15.114:57198) 2024-05-27T09:34:32.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.187+0000 7f5d3e000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d48156980 0x7f5d4814afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:32.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.187+0000 7f5d46a00640 1 -- 172.21.15.114:0/3347984654 learned_addr learned my addr 172.21.15.114:0/3347984654 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:34:32.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.187+0000 7f5d47400640 1 --2- 172.21.15.114:0/3347984654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d48160870 0x7f5d4814a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:32.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.188+0000 7f5d46a00640 1 -- 172.21.15.114:0/3347984654 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d48156980 msgr2=0x7f5d4814afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:32.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.188+0000 7f5d46a00640 1 --2- 172.21.15.114:0/3347984654 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d48156980 0x7f5d4814afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:32.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.188+0000 7f5d46a00640 1 -- 172.21.15.114:0/3347984654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d48160870 msgr2=0x7f5d4814a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:32.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.188+0000 7f5d46a00640 1 --2- 172.21.15.114:0/3347984654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d48160870 0x7f5d4814a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:32.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.188+0000 7f5d46a00640 1 -- 172.21.15.114:0/3347984654 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d48156100 con 0x7f5d480cc890 2024-05-27T09:34:32.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.188+0000 7f5d46a00640 1 --2- 172.21.15.114:0/3347984654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d480cc890 0x7f5d481f66b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800e870 tx=0x7f5d3800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:32.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.188+0000 7f5d3f400640 1 -- 172.21.15.114:0/3347984654 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38017070 con 0x7f5d480cc890 2024-05-27T09:34:32.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.188+0000 7f5d3f400640 1 -- 172.21.15.114:0/3347984654 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d3800c8e0 con 0x7f5d480cc890 2024-05-27T09:34:32.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.188+0000 7f5d3f400640 1 -- 172.21.15.114:0/3347984654 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3800ccd0 con 0x7f5d480cc890 2024-05-27T09:34:32.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.188+0000 7f5d47e00640 1 -- 172.21.15.114:0/3347984654 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d480b3410 con 0x7f5d480cc890 2024-05-27T09:34:32.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.188+0000 7f5d47e00640 1 -- 172.21.15.114:0/3347984654 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d480b39d0 con 0x7f5d480cc890 2024-05-27T09:34:32.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.189+0000 7f5d47e00640 1 -- 172.21.15.114:0/3347984654 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d4806d5d0 con 0x7f5d480cc890 2024-05-27T09:34:32.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.191+0000 7f5d3f400640 1 -- 172.21.15.114:0/3347984654 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5d3801a070 con 0x7f5d480cc890 2024-05-27T09:34:32.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.191+0000 7f5d3f400640 1 --2- 172.21.15.114:0/3347984654 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5d240762d0 0x7f5d240787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:32.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.192+0000 7f5d3f400640 1 -- 172.21.15.114:0/3347984654 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6082+0+0 (secure 0 0 0) 0x7f5d38097820 con 0x7f5d480cc890 2024-05-27T09:34:32.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.192+0000 7f5d3e000640 1 --2- 172.21.15.114:0/3347984654 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5d240762d0 0x7f5d240787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:32.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.193+0000 7f5d3e000640 1 --2- 172.21.15.114:0/3347984654 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5d240762d0 0x7f5d240787e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5d30034000 tx=0x7f5d3002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:32.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.197+0000 7f5d3f400640 1 -- 172.21.15.114:0/3347984654 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5d38060a40 con 0x7f5d480cc890 2024-05-27T09:34:32.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.429+0000 7f5d47e00640 1 -- 172.21.15.114:0/3347984654 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d48050860 con 0x7f5d480cc890 2024-05-27T09:34:32.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.430+0000 7f5d3f400640 1 -- 172.21.15.114:0/3347984654 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+6062 (secure 0 0 0) 0x7f5d38064d40 con 0x7f5d480cc890 2024-05-27T09:34:32.430 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:34:32.430 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:30.127436+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[4828,4875,5086]},"id":1}]} 2024-05-27T09:34:32.431 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 35 2024-05-27T09:34:32.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.433+0000 7f5d47e00640 1 -- 172.21.15.114:0/3347984654 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5d240762d0 msgr2=0x7f5d240787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:32.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.433+0000 7f5d47e00640 1 --2- 172.21.15.114:0/3347984654 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5d240762d0 0x7f5d240787e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5d30034000 tx=0x7f5d3002e000 comp rx=0 tx=0).stop 2024-05-27T09:34:32.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.433+0000 7f5d47e00640 1 -- 172.21.15.114:0/3347984654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d480cc890 msgr2=0x7f5d481f66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:32.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.433+0000 7f5d47e00640 1 --2- 172.21.15.114:0/3347984654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d480cc890 0x7f5d481f66b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800e870 tx=0x7f5d3800ed30 comp rx=0 tx=0).stop 2024-05-27T09:34:32.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.433+0000 7f5d47e00640 1 -- 172.21.15.114:0/3347984654 shutdown_connections 2024-05-27T09:34:32.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.433+0000 7f5d47e00640 1 --2- 172.21.15.114:0/3347984654 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5d240762d0 0x7f5d240787e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:32.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.433+0000 7f5d47e00640 1 --2- 172.21.15.114:0/3347984654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d48160870 0x7f5d4814a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:32.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.433+0000 7f5d47e00640 1 --2- 172.21.15.114:0/3347984654 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d48156980 0x7f5d4814afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:32.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.433+0000 7f5d47e00640 1 --2- 172.21.15.114:0/3347984654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5d480cc890 0x7f5d481f66b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:32.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.433+0000 7f5d47e00640 1 -- 172.21.15.114:0/3347984654 >> 172.21.15.114:0/3347984654 conn(0x7f5d48071460 msgr2=0x7f5d48151830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:32.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.433+0000 7f5d47e00640 1 -- 172.21.15.114:0/3347984654 shutdown_connections 2024-05-27T09:34:32.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:32.433+0000 7f5d47e00640 1 -- 172.21.15.114:0/3347984654 wait complete. 2024-05-27T09:34:32.457 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is in active, standby or standby-replay 2024-05-27T09:34:34.459 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:34:34.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.734+0000 7f93d7000640 1 Processor -- start 2024-05-27T09:34:34.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.734+0000 7f93d7000640 1 -- start start 2024-05-27T09:34:34.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.736+0000 7f93d7000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93d00741f0 0x7f93d00c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:34.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.736+0000 7f93d7000640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f93d01547d0 0x7f93d015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:34.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.736+0000 7f93d7000640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93d006fe50 con 0x7f93d0074840 2024-05-27T09:34:34.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.736+0000 7f93d7000640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f93d00703d0 con 0x7f93d01547d0 2024-05-27T09:34:34.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.736+0000 7f93d7000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f93d015b260 con 0x7f93d00741f0 2024-05-27T09:34:34.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.737+0000 7f93d5c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93d00741f0 0x7f93d00c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:34.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.737+0000 7f93d6600640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f93d0074840 0x7f93d006edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:51622/0 (socket says 172.21.15.114:51622) 2024-05-27T09:34:34.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.737+0000 7f93d6600640 1 -- 172.21.15.114:0/3082151574 learned_addr learned my addr 172.21.15.114:0/3082151574 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:34:34.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.737+0000 7f93d5200640 1 --2- 172.21.15.114:0/3082151574 >> v2:172.21.15.114:3300/0 conn(0x7f93d01547d0 0x7f93d015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:34.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.737+0000 7f93d5200640 1 -- 172.21.15.114:0/3082151574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93d00741f0 msgr2=0x7f93d00c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:34:34.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.737+0000 7f93d5200640 1 --2- 172.21.15.114:0/3082151574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93d00741f0 0x7f93d00c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:34.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.737+0000 7f93d5200640 1 -- 172.21.15.114:0/3082151574 >> v1:172.21.15.114:6789/0 conn(0x7f93d0074840 legacy=0x7f93d006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:34.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.737+0000 7f93d5200640 1 -- 172.21.15.114:0/3082151574 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d00700f0 con 0x7f93d01547d0 2024-05-27T09:34:34.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.737+0000 7f93cfe00640 1 -- 172.21.15.114:0/3082151574 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3007343789 0 0) 0x7f93d006fe50 con 0x7f93d0074840 2024-05-27T09:34:34.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.738+0000 7f93d5200640 1 --2- 172.21.15.114:0/3082151574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93d01547d0 0x7f93d015ad20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f93bc0097a0 tx=0x7f93bc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5a6b8aed2e464 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:34.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.738+0000 7f93cfe00640 1 -- 172.21.15.114:0/3082151574 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc037070 con 0x7f93d01547d0 2024-05-27T09:34:34.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.738+0000 7f93cfe00640 1 -- 172.21.15.114:0/3082151574 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93bc032440 con 0x7f93d01547d0 2024-05-27T09:34:34.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.738+0000 7f93cfe00640 1 -- 172.21.15.114:0/3082151574 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc032810 con 0x7f93d01547d0 2024-05-27T09:34:34.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.739+0000 7f93d7000640 1 -- 172.21.15.114:0/3082151574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93d01547d0 msgr2=0x7f93d015ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:34.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.739+0000 7f93d7000640 1 --2- 172.21.15.114:0/3082151574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93d01547d0 0x7f93d015ad20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f93bc0097a0 tx=0x7f93bc02a210 comp rx=0 tx=0).stop 2024-05-27T09:34:34.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.739+0000 7f93d7000640 1 -- 172.21.15.114:0/3082151574 shutdown_connections 2024-05-27T09:34:34.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.739+0000 7f93d7000640 1 --2- 172.21.15.114:0/3082151574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93d01547d0 0x7f93d015ad20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:34.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.739+0000 7f93d7000640 1 --2- 172.21.15.114:0/3082151574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93d00741f0 0x7f93d00c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:34.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.739+0000 7f93d7000640 1 -- 172.21.15.114:0/3082151574 >> 172.21.15.114:0/3082151574 conn(0x7f93d00ca320 msgr2=0x7f93d00ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:34.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.739+0000 7f93d7000640 1 -- 172.21.15.114:0/3082151574 shutdown_connections 2024-05-27T09:34:34.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.739+0000 7f93d7000640 1 -- 172.21.15.114:0/3082151574 wait complete. 2024-05-27T09:34:34.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.742+0000 7f93d7000640 1 Processor -- start 2024-05-27T09:34:34.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.742+0000 7f93d7000640 1 -- start start 2024-05-27T09:34:34.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.743+0000 7f93d7000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93d00741f0 0x7f93d01f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:34.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.743+0000 7f93d7000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93d0074840 0x7f93d00cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:34.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.743+0000 7f93d7000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93d01547d0 0x7f93d0156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:34.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.743+0000 7f93d7000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f93d0073790 con 0x7f93d00741f0 2024-05-27T09:34:34.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.743+0000 7f93d7000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f93d00734b0 con 0x7f93d01547d0 2024-05-27T09:34:34.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.743+0000 7f93d7000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f93d006f500 con 0x7f93d0074840 2024-05-27T09:34:34.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.743+0000 7f93d5c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93d00741f0 0x7f93d01f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:34.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.743+0000 7f93d5c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93d00741f0 0x7f93d01f5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57216/0 (socket says 172.21.15.114:57216) 2024-05-27T09:34:34.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.744+0000 7f93d5c00640 1 -- 172.21.15.114:0/1176228947 learned_addr learned my addr 172.21.15.114:0/1176228947 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:34:34.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.744+0000 7f93d5200640 1 --2- 172.21.15.114:0/1176228947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93d0074840 0x7f93d00cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:34.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.744+0000 7f93d6600640 1 --2- 172.21.15.114:0/1176228947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93d01547d0 0x7f93d0156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:34.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.744+0000 7f93d5c00640 1 -- 172.21.15.114:0/1176228947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93d0074840 msgr2=0x7f93d00cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:34.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.744+0000 7f93d5c00640 1 --2- 172.21.15.114:0/1176228947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93d0074840 0x7f93d00cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:34.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.744+0000 7f93d5c00640 1 -- 172.21.15.114:0/1176228947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93d01547d0 msgr2=0x7f93d0156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:34.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.744+0000 7f93d5c00640 1 --2- 172.21.15.114:0/1176228947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93d01547d0 0x7f93d0156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:34.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.744+0000 7f93d5c00640 1 -- 172.21.15.114:0/1176228947 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d01f6310 con 0x7f93d00741f0 2024-05-27T09:34:34.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.744+0000 7f93d5c00640 1 --2- 172.21.15.114:0/1176228947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93d00741f0 0x7f93d01f5dd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f93c400e870 tx=0x7f93c400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:34.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.744+0000 7f93cd600640 1 -- 172.21.15.114:0/1176228947 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4017070 con 0x7f93d00741f0 2024-05-27T09:34:34.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.744+0000 7f93cd600640 1 -- 172.21.15.114:0/1176228947 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93c400c940 con 0x7f93d00741f0 2024-05-27T09:34:34.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.744+0000 7f93cd600640 1 -- 172.21.15.114:0/1176228947 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c400cd30 con 0x7f93d00741f0 2024-05-27T09:34:34.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.745+0000 7f93d7000640 1 -- 172.21.15.114:0/1176228947 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d00c2970 con 0x7f93d00741f0 2024-05-27T09:34:34.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.745+0000 7f93d7000640 1 -- 172.21.15.114:0/1176228947 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d00c38b0 con 0x7f93d00741f0 2024-05-27T09:34:34.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.746+0000 7f93d7000640 1 -- 172.21.15.114:0/1176228947 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d0152d80 con 0x7f93d00741f0 2024-05-27T09:34:34.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.747+0000 7f93cd600640 1 -- 172.21.15.114:0/1176228947 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f93c401a070 con 0x7f93d00741f0 2024-05-27T09:34:34.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.748+0000 7f93cd600640 1 --2- 172.21.15.114:0/1176228947 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f93b00762d0 0x7f93b00787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:34.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.748+0000 7f93cd600640 1 -- 172.21.15.114:0/1176228947 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6082+0+0 (secure 0 0 0) 0x7f93c40979c0 con 0x7f93d00741f0 2024-05-27T09:34:34.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.748+0000 7f93d5200640 1 --2- 172.21.15.114:0/1176228947 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f93b00762d0 0x7f93b00787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:34.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.749+0000 7f93d5200640 1 --2- 172.21.15.114:0/1176228947 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f93b00762d0 0x7f93b00787e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f93bc034000 tx=0x7f93bc02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:34.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.753+0000 7f93cd600640 1 -- 172.21.15.114:0/1176228947 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f93c4060ab0 con 0x7f93d00741f0 2024-05-27T09:34:34.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.994+0000 7f93d7000640 1 -- 172.21.15.114:0/1176228947 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93d00b3710 con 0x7f93d00741f0 2024-05-27T09:34:34.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.995+0000 7f93cd600640 1 -- 172.21.15.114:0/1176228947 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+6890 (secure 0 0 0) 0x7f93c4064db0 con 0x7f93d00741f0 2024-05-27T09:34:34.996 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:34:34.996 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5160,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:30.127436+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086,4875,4828]},"id":1}]} 2024-05-27T09:34:34.996 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 36 2024-05-27T09:34:34.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.998+0000 7f93d7000640 1 -- 172.21.15.114:0/1176228947 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f93b00762d0 msgr2=0x7f93b00787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:34.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.998+0000 7f93d7000640 1 --2- 172.21.15.114:0/1176228947 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f93b00762d0 0x7f93b00787e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f93bc034000 tx=0x7f93bc02e000 comp rx=0 tx=0).stop 2024-05-27T09:34:34.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.998+0000 7f93d7000640 1 -- 172.21.15.114:0/1176228947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93d00741f0 msgr2=0x7f93d01f5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:34.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.998+0000 7f93d7000640 1 --2- 172.21.15.114:0/1176228947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93d00741f0 0x7f93d01f5dd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f93c400e870 tx=0x7f93c400ed30 comp rx=0 tx=0).stop 2024-05-27T09:34:34.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.998+0000 7f93d7000640 1 -- 172.21.15.114:0/1176228947 shutdown_connections 2024-05-27T09:34:34.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.998+0000 7f93d7000640 1 --2- 172.21.15.114:0/1176228947 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f93b00762d0 0x7f93b00787e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:34.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.998+0000 7f93d7000640 1 --2- 172.21.15.114:0/1176228947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93d01547d0 0x7f93d0156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:34.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.998+0000 7f93d7000640 1 --2- 172.21.15.114:0/1176228947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93d0074840 0x7f93d00cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:34.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.998+0000 7f93d7000640 1 --2- 172.21.15.114:0/1176228947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93d00741f0 0x7f93d01f5dd0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:34.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.998+0000 7f93d7000640 1 -- 172.21.15.114:0/1176228947 >> 172.21.15.114:0/1176228947 conn(0x7f93d00ca320 msgr2=0x7f93d014fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:34.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.999+0000 7f93d7000640 1 -- 172.21.15.114:0/1176228947 shutdown_connections 2024-05-27T09:34:34.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:34:34.999+0000 7f93d7000640 1 -- 172.21.15.114:0/1176228947 wait complete. 2024-05-27T09:34:35.022 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2024-05-27T09:34:35.023 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 45 secs before thrashing 2024-05-27T09:34:35.398 INFO:tasks.workunit.client.0.smithi114.stderr:error: RPC failed; curl 56 OpenSSL SSL_read: error:0A000126:SSL routines::unexpected eof while reading, errno 0 2024-05-27T09:34:35.399 INFO:tasks.workunit.client.0.smithi114.stderr:fetch-pack: unexpected disconnect while reading sideband packet 2024-05-27T09:34:35.399 INFO:tasks.workunit.client.0.smithi114.stderr:fatal: early EOF 2024-05-27T09:34:35.410 INFO:tasks.workunit.client.0.smithi114.stderr:fatal: fetch-pack: invalid index-pack output 2024-05-27T09:34:35.710 INFO:tasks.workunit.client.0.smithi114.stderr:+ retry 2024-05-27T09:34:35.710 INFO:tasks.workunit.client.0.smithi114.stderr:+ rm -rf ceph 2024-05-27T09:34:35.713 INFO:tasks.workunit.client.0.smithi114.stderr:+ timeout 3600 git clone https://git.ceph.com/ceph.git 2024-05-27T09:34:35.722 INFO:tasks.workunit.client.0.smithi114.stderr:Cloning into 'ceph'... 2024-05-27T09:34:36.021 INFO:tasks.ceph.mds.c.smithi114.stderr:2024-05-27T09:34:36.020+0000 7fd2b8600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-27T09:34:36.021 INFO:tasks.ceph.mds.c.smithi114.stderr:2024-05-27T09:34:36.020+0000 7fd2b8600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-27T09:34:36.021 INFO:tasks.ceph.mds.c.smithi114.stderr:2024-05-27T09:34:36.020+0000 7fd2b8600640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-27T09:34:49.426 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:34:49.435 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:35:19.544 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:35:19.553 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:35:20.024 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:35:20.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.279+0000 7f955dc00640 1 Processor -- start 2024-05-27T09:35:20.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.279+0000 7f955dc00640 1 -- start start 2024-05-27T09:35:20.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.281+0000 7f955dc00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f95580c8780 0x7f955815fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:20.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.281+0000 7f955dc00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95580b46e0 0x7f9558166fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:20.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.281+0000 7f955dc00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95580cd1e0 con 0x7f95580b75b0 2024-05-27T09:35:20.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.281+0000 7f955dc00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f95580cd4c0 con 0x7f95580c8780 2024-05-27T09:35:20.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.281+0000 7f955dc00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f95580c7f80 con 0x7f95580b46e0 2024-05-27T09:35:20.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.282+0000 7f9557e00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f95580c8780 0x7f955815fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:20.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.282+0000 7f955d200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95580b46e0 0x7f9558166fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:20.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.282+0000 7f9557e00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f95580c8780 0x7f955815fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:60374/0 (socket says 172.21.15.114:60374) 2024-05-27T09:35:20.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.282+0000 7f9557e00640 1 -- 172.21.15.114:0/650854191 learned_addr learned my addr 172.21.15.114:0/650854191 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:20.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.282+0000 7f9557e00640 1 -- 172.21.15.114:0/650854191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95580b46e0 msgr2=0x7f9558166fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:35:20.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.282+0000 7f9557e00640 1 --2- 172.21.15.114:0/650854191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95580b46e0 0x7f9558166fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:20.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.282+0000 7f9557e00640 1 -- 172.21.15.114:0/650854191 >> v1:172.21.15.163:6789/0 conn(0x7f95580b75b0 legacy=0x7f95580d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:20.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.282+0000 7f9557e00640 1 -- 172.21.15.114:0/650854191 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9558160160 con 0x7f95580c8780 2024-05-27T09:35:20.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.282+0000 7f955d200640 1 --2- 172.21.15.114:0/650854191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95580b46e0 0x7f9558166fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:35:20.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.283+0000 7f9557e00640 1 --2- 172.21.15.114:0/650854191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95580c8780 0x7f955815fc20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9548009870 tx=0x7f954802a2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39798449cbd56a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:20.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.284+0000 7f9556a00640 1 -- 172.21.15.114:0/650854191 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9548037070 con 0x7f95580c8780 2024-05-27T09:35:20.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.284+0000 7f9556a00640 1 -- 172.21.15.114:0/650854191 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9548032640 con 0x7f95580c8780 2024-05-27T09:35:20.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.284+0000 7f9556a00640 1 -- 172.21.15.114:0/650854191 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9548032a10 con 0x7f95580c8780 2024-05-27T09:35:20.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.284+0000 7f955dc00640 1 -- 172.21.15.114:0/650854191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95580c8780 msgr2=0x7f955815fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:20.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.284+0000 7f955dc00640 1 --2- 172.21.15.114:0/650854191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95580c8780 0x7f955815fc20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9548009870 tx=0x7f954802a2e0 comp rx=0 tx=0).stop 2024-05-27T09:35:20.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.284+0000 7f955dc00640 1 -- 172.21.15.114:0/650854191 shutdown_connections 2024-05-27T09:35:20.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.284+0000 7f955dc00640 1 --2- 172.21.15.114:0/650854191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95580b46e0 0x7f9558166fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:20.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.284+0000 7f955dc00640 1 --2- 172.21.15.114:0/650854191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95580c8780 0x7f955815fc20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:20.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.284+0000 7f955dc00640 1 -- 172.21.15.114:0/650854191 >> 172.21.15.114:0/650854191 conn(0x7f955806e040 msgr2=0x7f955806e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:20.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.284+0000 7f955dc00640 1 -- 172.21.15.114:0/650854191 shutdown_connections 2024-05-27T09:35:20.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.285+0000 7f955dc00640 1 -- 172.21.15.114:0/650854191 wait complete. 2024-05-27T09:35:20.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.287+0000 7f955dc00640 1 Processor -- start 2024-05-27T09:35:20.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.287+0000 7f955dc00640 1 -- start start 2024-05-27T09:35:20.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.288+0000 7f955dc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95580b46e0 0x7f95580ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:20.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.288+0000 7f955dc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95580b75b0 0x7f95580cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:20.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.288+0000 7f955dc00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95580c8780 0x7f9558150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:20.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.288+0000 7f955dc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f95580c6800 con 0x7f95580b75b0 2024-05-27T09:35:20.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.288+0000 7f955dc00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f95580cdf30 con 0x7f95580b46e0 2024-05-27T09:35:20.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.288+0000 7f955dc00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f95580c98f0 con 0x7f95580c8780 2024-05-27T09:35:20.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.288+0000 7f9557400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95580b75b0 0x7f95580cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:20.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.289+0000 7f9557400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95580b75b0 0x7f95580cb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46032/0 (socket says 172.21.15.114:46032) 2024-05-27T09:35:20.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.289+0000 7f9557400640 1 -- 172.21.15.114:0/914241194 learned_addr learned my addr 172.21.15.114:0/914241194 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:20.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.289+0000 7f9557e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95580b46e0 0x7f95580ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:20.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.289+0000 7f955d200640 1 --2- 172.21.15.114:0/914241194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95580c8780 0x7f9558150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:20.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.289+0000 7f9557400640 1 -- 172.21.15.114:0/914241194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95580c8780 msgr2=0x7f9558150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:20.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.289+0000 7f9557400640 1 --2- 172.21.15.114:0/914241194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95580c8780 0x7f9558150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:20.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.289+0000 7f9557400640 1 -- 172.21.15.114:0/914241194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95580b46e0 msgr2=0x7f95580ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:20.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.289+0000 7f9557400640 1 --2- 172.21.15.114:0/914241194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95580b46e0 0x7f95580ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:20.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.289+0000 7f9557400640 1 -- 172.21.15.114:0/914241194 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955815d430 con 0x7f95580b75b0 2024-05-27T09:35:20.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.289+0000 7f9557e00640 1 --2- 172.21.15.114:0/914241194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95580b46e0 0x7f95580ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:35:20.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.289+0000 7f9557400640 1 --2- 172.21.15.114:0/914241194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95580b75b0 0x7f95580cb3f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f954400c370 tx=0x7f954400c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:20.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.290+0000 7f954fe00640 1 -- 172.21.15.114:0/914241194 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95440052d0 con 0x7f95580b75b0 2024-05-27T09:35:20.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.290+0000 7f954fe00640 1 -- 172.21.15.114:0/914241194 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9544007020 con 0x7f95580b75b0 2024-05-27T09:35:20.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.290+0000 7f955dc00640 1 -- 172.21.15.114:0/914241194 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955815d710 con 0x7f95580b75b0 2024-05-27T09:35:20.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.290+0000 7f954fe00640 1 -- 172.21.15.114:0/914241194 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95440054b0 con 0x7f95580b75b0 2024-05-27T09:35:20.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.290+0000 7f955dc00640 1 -- 172.21.15.114:0/914241194 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95580ca690 con 0x7f95580b75b0 2024-05-27T09:35:20.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.291+0000 7f955dc00640 1 -- 172.21.15.114:0/914241194 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f955815d180 con 0x7f95580b75b0 2024-05-27T09:35:20.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.294+0000 7f954fe00640 1 -- 172.21.15.114:0/914241194 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9544007410 con 0x7f95580b75b0 2024-05-27T09:35:20.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.295+0000 7f954fe00640 1 --2- 172.21.15.114:0/914241194 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9538076300 0x7f9538078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:20.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.295+0000 7f954fe00640 1 -- 172.21.15.114:0/914241194 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6082+0+0 (secure 0 0 0) 0x7f9544096a10 con 0x7f95580b75b0 2024-05-27T09:35:20.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.295+0000 7f9557e00640 1 --2- 172.21.15.114:0/914241194 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9538076300 0x7f9538078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:20.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.296+0000 7f9557e00640 1 --2- 172.21.15.114:0/914241194 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9538076300 0x7f9538078810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9548034000 tx=0x7f954802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:20.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.300+0000 7f954fe00640 1 -- 172.21.15.114:0/914241194 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f954405fc50 con 0x7f95580b75b0 2024-05-27T09:35:20.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.574+0000 7f955dc00640 1 -- 172.21.15.114:0/914241194 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9558050860 con 0x7f95580b75b0 2024-05-27T09:35:20.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.575+0000 7f954fe00640 1 -- 172.21.15.114:0/914241194 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+6890 (secure 0 0 0) 0x7f9544068290 con 0x7f95580b75b0 2024-05-27T09:35:20.576 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:35:20.576 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5160,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:30.127436+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086,4875,4828]},"id":1}]} 2024-05-27T09:35:20.576 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 36 2024-05-27T09:35:20.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.578+0000 7f955dc00640 1 -- 172.21.15.114:0/914241194 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9538076300 msgr2=0x7f9538078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:20.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.578+0000 7f955dc00640 1 --2- 172.21.15.114:0/914241194 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9538076300 0x7f9538078810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9548034000 tx=0x7f954802e000 comp rx=0 tx=0).stop 2024-05-27T09:35:20.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.578+0000 7f955dc00640 1 -- 172.21.15.114:0/914241194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95580b75b0 msgr2=0x7f95580cb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:20.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.578+0000 7f955dc00640 1 --2- 172.21.15.114:0/914241194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95580b75b0 0x7f95580cb3f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f954400c370 tx=0x7f954400c770 comp rx=0 tx=0).stop 2024-05-27T09:35:20.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.578+0000 7f955dc00640 1 -- 172.21.15.114:0/914241194 shutdown_connections 2024-05-27T09:35:20.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.578+0000 7f955dc00640 1 --2- 172.21.15.114:0/914241194 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9538076300 0x7f9538078810 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:20.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.578+0000 7f955dc00640 1 --2- 172.21.15.114:0/914241194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95580c8780 0x7f9558150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:20.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.578+0000 7f955dc00640 1 --2- 172.21.15.114:0/914241194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95580b75b0 0x7f95580cb3f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:20.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.578+0000 7f955dc00640 1 --2- 172.21.15.114:0/914241194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95580b46e0 0x7f95580ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:20.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.578+0000 7f955dc00640 1 -- 172.21.15.114:0/914241194 >> 172.21.15.114:0/914241194 conn(0x7f955806e040 msgr2=0x7f9558075da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:20.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.579+0000 7f955dc00640 1 -- 172.21.15.114:0/914241194 shutdown_connections 2024-05-27T09:35:20.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:20.579+0000 7f955dc00640 1 -- 172.21.15.114:0/914241194 wait complete. 2024-05-27T09:35:20.612 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=2) 2024-05-27T09:35:20.613 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-05-27T09:35:20.613 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:35:20.626 INFO:tasks.ceph.mds.f:Stopped 2024-05-27T09:35:20.627 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:35:21.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.114+0000 7febed200640 1 Processor -- start 2024-05-27T09:35:21.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.114+0000 7febed200640 1 -- start start 2024-05-27T09:35:21.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.119+0000 7febed200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7febe80c44b0 0x7febe80c2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:21.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.119+0000 7febed200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7febe80c8c30 0x7febe80c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:21.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.119+0000 7febed200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febe80c97b0 con 0x7febe80cae40 2024-05-27T09:35:21.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.119+0000 7febed200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7febe80c9580 con 0x7febe80c8c30 2024-05-27T09:35:21.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.119+0000 7febe6a00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7febe80c44b0 0x7febe80c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:21.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.119+0000 7febed200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7febe80c7660 con 0x7febe80c44b0 2024-05-27T09:35:21.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.119+0000 7febe6a00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7febe80c44b0 0x7febe80c2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:60386/0 (socket says 172.21.15.114:60386) 2024-05-27T09:35:21.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.119+0000 7febe6a00640 1 -- 172.21.15.114:0/3731083128 learned_addr learned my addr 172.21.15.114:0/3731083128 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:21.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.119+0000 7febe7e00640 1 --2- 172.21.15.114:0/3731083128 >> v2:172.21.15.114:3300/0 conn(0x7febe80c8c30 0x7febe80c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:21.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.119+0000 7febe6a00640 1 -- 172.21.15.114:0/3731083128 >> v2:172.21.15.114:3300/0 conn(0x7febe80c8c30 msgr2=0x7febe80c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:35:21.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.125+0000 7febe6a00640 1 --2- 172.21.15.114:0/3731083128 >> v2:172.21.15.114:3300/0 conn(0x7febe80c8c30 0x7febe80c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:21.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.125+0000 7febe6a00640 1 -- 172.21.15.114:0/3731083128 >> v1:172.21.15.114:6789/0 conn(0x7febe80cae40 legacy=0x7febe81527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:35:21.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.125+0000 7febe6a00640 1 -- 172.21.15.114:0/3731083128 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe80c7940 con 0x7febe80c44b0 2024-05-27T09:35:21.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.129+0000 7febe7e00640 1 --2- 172.21.15.114:0/3731083128 >> v2:172.21.15.114:3300/0 conn(0x7febe80c8c30 0x7febe80c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:35:21.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.130+0000 7febe6a00640 1 --2- 172.21.15.114:0/3731083128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febe80c44b0 0x7febe80c2840 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7febe81561a0 tx=0x7febe002a750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b268b925290637e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:21.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.131+0000 7febe6000640 1 -- 172.21.15.114:0/3731083128 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0036070 con 0x7febe80c44b0 2024-05-27T09:35:21.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.131+0000 7febe6000640 1 -- 172.21.15.114:0/3731083128 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe0031600 con 0x7febe80c44b0 2024-05-27T09:35:21.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.133+0000 7febed200640 1 -- 172.21.15.114:0/3731083128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febe80c44b0 msgr2=0x7febe80c2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:21.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.134+0000 7febe6000640 1 -- 172.21.15.114:0/3731083128 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0036070 con 0x7febe80c44b0 2024-05-27T09:35:21.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.135+0000 7febed200640 1 --2- 172.21.15.114:0/3731083128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febe80c44b0 0x7febe80c2840 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7febe81561a0 tx=0x7febe002a750 comp rx=0 tx=0).stop 2024-05-27T09:35:21.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.136+0000 7febed200640 1 -- 172.21.15.114:0/3731083128 shutdown_connections 2024-05-27T09:35:21.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.136+0000 7febed200640 1 --2- 172.21.15.114:0/3731083128 >> v2:172.21.15.114:3300/0 conn(0x7febe80c8c30 0x7febe80c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:21.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.136+0000 7febed200640 1 --2- 172.21.15.114:0/3731083128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febe80c44b0 0x7febe80c2840 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:21.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.136+0000 7febed200640 1 -- 172.21.15.114:0/3731083128 >> 172.21.15.114:0/3731083128 conn(0x7febe80ca320 msgr2=0x7febe80ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:21.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.139+0000 7febed200640 1 -- 172.21.15.114:0/3731083128 shutdown_connections 2024-05-27T09:35:21.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.139+0000 7febed200640 1 -- 172.21.15.114:0/3731083128 wait complete. 2024-05-27T09:35:21.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.147+0000 7febed200640 1 Processor -- start 2024-05-27T09:35:21.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.147+0000 7febed200640 1 -- start start 2024-05-27T09:35:21.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.152+0000 7febed200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febe80c44b0 0x7febe80c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:21.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.152+0000 7febed200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febe80c8c30 0x7febe80c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:21.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.153+0000 7febe6a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febe80c8c30 0x7febe80c4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:21.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.153+0000 7febe6a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febe80c8c30 0x7febe80c4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:33948/0 (socket says 172.21.15.114:33948) 2024-05-27T09:35:21.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.153+0000 7febe6a00640 1 -- 172.21.15.114:0/1674296674 learned_addr learned my addr 172.21.15.114:0/1674296674 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:21.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.153+0000 7febe7400640 1 --2- 172.21.15.114:0/1674296674 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febe80c44b0 0x7febe80c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:21.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.154+0000 7febed200640 1 --2- 172.21.15.114:0/1674296674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febe815bf40 0x7febe814b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:21.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.155+0000 7febe7e00640 1 --2- 172.21.15.114:0/1674296674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febe815bf40 0x7febe814b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:21.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.155+0000 7febed200640 1 -- 172.21.15.114:0/1674296674 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7febe80c2d80 con 0x7febe815bf40 2024-05-27T09:35:21.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.155+0000 7febed200640 1 -- 172.21.15.114:0/1674296674 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7febe80c9d00 con 0x7febe80c44b0 2024-05-27T09:35:21.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.155+0000 7febed200640 1 -- 172.21.15.114:0/1674296674 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7febe80c7c20 con 0x7febe80c8c30 2024-05-27T09:35:21.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.156+0000 7febe7400640 1 -- 172.21.15.114:0/1674296674 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febe80c8c30 msgr2=0x7febe80c4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:21.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.157+0000 7febe7400640 1 --2- 172.21.15.114:0/1674296674 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febe80c8c30 0x7febe80c4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:21.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.157+0000 7febe7400640 1 -- 172.21.15.114:0/1674296674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febe815bf40 msgr2=0x7febe814b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:21.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.158+0000 7febe7400640 1 --2- 172.21.15.114:0/1674296674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febe815bf40 0x7febe814b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:21.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.158+0000 7febe7400640 1 -- 172.21.15.114:0/1674296674 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe80c5310 con 0x7febe80c44b0 2024-05-27T09:35:21.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.158+0000 7febe6a00640 1 --2- 172.21.15.114:0/1674296674 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febe80c8c30 0x7febe80c4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:35:21.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.159+0000 7febe7e00640 1 --2- 172.21.15.114:0/1674296674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febe815bf40 0x7febe814b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:35:21.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.160+0000 7febe7400640 1 --2- 172.21.15.114:0/1674296674 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febe80c44b0 0x7febe80c6a10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7febe80ce7a0 tx=0x7febd800c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:21.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.160+0000 7febdf400640 1 -- 172.21.15.114:0/1674296674 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8017070 con 0x7febe80c44b0 2024-05-27T09:35:21.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.160+0000 7febdf400640 1 -- 172.21.15.114:0/1674296674 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febd8003610 con 0x7febe80c44b0 2024-05-27T09:35:21.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.163+0000 7febed200640 1 -- 172.21.15.114:0/1674296674 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe81ee4b0 con 0x7febe80c44b0 2024-05-27T09:35:21.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.163+0000 7febed200640 1 -- 172.21.15.114:0/1674296674 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe81eea70 con 0x7febe80c44b0 2024-05-27T09:35:21.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.164+0000 7febdf400640 1 -- 172.21.15.114:0/1674296674 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8003a00 con 0x7febe80c44b0 2024-05-27T09:35:21.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.170+0000 7febed200640 1 -- 172.21.15.114:0/1674296674 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febe8050860 con 0x7febe80c44b0 2024-05-27T09:35:21.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.170+0000 7febdf400640 1 -- 172.21.15.114:0/1674296674 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7febd801a070 con 0x7febe80c44b0 2024-05-27T09:35:21.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.174+0000 7febdf400640 1 --2- 172.21.15.114:0/1674296674 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7febc80763a0 0x7febc80788b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:21.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.174+0000 7febdf400640 1 -- 172.21.15.114:0/1674296674 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6082+0+0 (secure 0 0 0) 0x7febd8096990 con 0x7febe80c44b0 2024-05-27T09:35:21.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.180+0000 7febdf400640 1 -- 172.21.15.114:0/1674296674 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7febd805fb00 con 0x7febe80c44b0 2024-05-27T09:35:21.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.181+0000 7febe6a00640 1 --2- 172.21.15.114:0/1674296674 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7febc80763a0 0x7febc80788b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:21.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.182+0000 7febe6a00640 1 --2- 172.21.15.114:0/1674296674 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7febc80763a0 0x7febc80788b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7febe0009850 tx=0x7febe0009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:21.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.637+0000 7febed200640 1 -- 172.21.15.114:0/1674296674 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febe8074c10 con 0x7febe80c44b0 2024-05-27T09:35:21.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.638+0000 7febdf400640 1 -- 172.21.15.114:0/1674296674 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+6890 (secure 0 0 0) 0x7febd8063e00 con 0x7febe80c44b0 2024-05-27T09:35:21.638 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:35:21.638 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5160,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:30.127436+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086,4875,4828]},"id":1}]} 2024-05-27T09:35:21.639 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 36 2024-05-27T09:35:21.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.640+0000 7febed200640 1 -- 172.21.15.114:0/1674296674 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7febc80763a0 msgr2=0x7febc80788b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:21.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.640+0000 7febed200640 1 --2- 172.21.15.114:0/1674296674 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7febc80763a0 0x7febc80788b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7febe0009850 tx=0x7febe0009000 comp rx=0 tx=0).stop 2024-05-27T09:35:21.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.640+0000 7febed200640 1 -- 172.21.15.114:0/1674296674 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febe80c44b0 msgr2=0x7febe80c6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:21.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.640+0000 7febed200640 1 --2- 172.21.15.114:0/1674296674 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febe80c44b0 0x7febe80c6a10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7febe80ce7a0 tx=0x7febd800c870 comp rx=0 tx=0).stop 2024-05-27T09:35:21.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.640+0000 7febed200640 1 -- 172.21.15.114:0/1674296674 shutdown_connections 2024-05-27T09:35:21.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.640+0000 7febed200640 1 --2- 172.21.15.114:0/1674296674 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7febc80763a0 0x7febc80788b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:21.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.640+0000 7febed200640 1 --2- 172.21.15.114:0/1674296674 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febe815bf40 0x7febe814b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:21.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.640+0000 7febed200640 1 --2- 172.21.15.114:0/1674296674 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febe80c8c30 0x7febe80c4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:21.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.640+0000 7febed200640 1 --2- 172.21.15.114:0/1674296674 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febe80c44b0 0x7febe80c6a10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:21.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.640+0000 7febed200640 1 -- 172.21.15.114:0/1674296674 >> 172.21.15.114:0/1674296674 conn(0x7febe80ca320 msgr2=0x7febe80ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:21.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.640+0000 7febed200640 1 -- 172.21.15.114:0/1674296674 shutdown_connections 2024-05-27T09:35:21.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:21.640+0000 7febed200640 1 -- 172.21.15.114:0/1674296674 wait complete. 2024-05-27T09:35:21.664 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-05-27T09:35:23.665 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:35:23.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.954+0000 7fa1c6a00640 1 Processor -- start 2024-05-27T09:35:23.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.955+0000 7fa1c6a00640 1 -- start start 2024-05-27T09:35:23.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.956+0000 7fa1c6a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa1c00741f0 0x7fa1c00c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:23.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.956+0000 7fa1c6a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1c01547d0 0x7fa1c015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:23.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.956+0000 7fa1c6a00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa1c0074840 0x7fa1c006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:23.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.956+0000 7fa1c6a00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa1c00703d0 con 0x7fa1c00741f0 2024-05-27T09:35:23.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.956+0000 7fa1c6a00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fa1c015b260 con 0x7fa1c0074840 2024-05-27T09:35:23.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.956+0000 7fa1c6a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c006fe50 con 0x7fa1c01547d0 2024-05-27T09:35:23.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.957+0000 7fa1c6000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa1c0074840 0x7fa1c006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:23.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.957+0000 7fa1c4c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1c01547d0 0x7fa1c015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:23.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.957+0000 7fa1c5600640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa1c00741f0 0x7fa1c00c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:23.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.957+0000 7fa1c6000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa1c0074840 0x7fa1c006edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:60408/0 (socket says 172.21.15.114:60408) 2024-05-27T09:35:23.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.957+0000 7fa1c6000640 1 -- 172.21.15.114:0/3365797971 learned_addr learned my addr 172.21.15.114:0/3365797971 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:23.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.957+0000 7fa1c4c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1c01547d0 0x7fa1c015ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:33962/0 (socket says 172.21.15.114:33962) 2024-05-27T09:35:23.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.957+0000 7fa1c6000640 1 -- 172.21.15.114:0/3365797971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1c01547d0 msgr2=0x7fa1c015ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:35:23.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.957+0000 7fa1c6000640 1 --2- 172.21.15.114:0/3365797971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1c01547d0 0x7fa1c015ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:23.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.957+0000 7fa1c6000640 1 -- 172.21.15.114:0/3365797971 >> v2:172.21.15.114:3300/0 conn(0x7fa1c00741f0 msgr2=0x7fa1c00c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:35:23.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.957+0000 7fa1c6000640 1 --2- 172.21.15.114:0/3365797971 >> v2:172.21.15.114:3300/0 conn(0x7fa1c00741f0 0x7fa1c00c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:23.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.957+0000 7fa1c6000640 1 -- 172.21.15.114:0/3365797971 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c00700f0 con 0x7fa1c0074840 2024-05-27T09:35:23.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.957+0000 7fa1c4c00640 1 --2- 172.21.15.114:0/3365797971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1c01547d0 0x7fa1c015ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:35:23.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.958+0000 7fa1c6000640 1 --2- 172.21.15.114:0/3365797971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1c0074840 0x7fa1c006edf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa1c00b31e0 tx=0x7fa1b802a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2c31377ff5ccd3e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:23.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.959+0000 7fa1bfe00640 1 -- 172.21.15.114:0/3365797971 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8036070 con 0x7fa1c0074840 2024-05-27T09:35:23.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.959+0000 7fa1bfe00640 1 -- 172.21.15.114:0/3365797971 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1b8031600 con 0x7fa1c0074840 2024-05-27T09:35:23.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.959+0000 7fa1bfe00640 1 -- 172.21.15.114:0/3365797971 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b80319d0 con 0x7fa1c0074840 2024-05-27T09:35:23.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.959+0000 7fa1c6a00640 1 -- 172.21.15.114:0/3365797971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1c0074840 msgr2=0x7fa1c006edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:23.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.959+0000 7fa1c6a00640 1 --2- 172.21.15.114:0/3365797971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1c0074840 0x7fa1c006edf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa1c00b31e0 tx=0x7fa1b802a730 comp rx=0 tx=0).stop 2024-05-27T09:35:23.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.959+0000 7fa1c6a00640 1 -- 172.21.15.114:0/3365797971 shutdown_connections 2024-05-27T09:35:23.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.959+0000 7fa1c6a00640 1 --2- 172.21.15.114:0/3365797971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1c0074840 0x7fa1c006edf0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:23.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.959+0000 7fa1c6a00640 1 --2- 172.21.15.114:0/3365797971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1c01547d0 0x7fa1c015ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:23.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.959+0000 7fa1c6a00640 1 --2- 172.21.15.114:0/3365797971 >> v2:172.21.15.114:3300/0 conn(0x7fa1c00741f0 0x7fa1c00c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:23.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.959+0000 7fa1c6a00640 1 -- 172.21.15.114:0/3365797971 >> 172.21.15.114:0/3365797971 conn(0x7fa1c00ca320 msgr2=0x7fa1c00ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:23.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.960+0000 7fa1c6a00640 1 -- 172.21.15.114:0/3365797971 shutdown_connections 2024-05-27T09:35:23.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.960+0000 7fa1c6a00640 1 -- 172.21.15.114:0/3365797971 wait complete. 2024-05-27T09:35:23.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.962+0000 7fa1c6a00640 1 Processor -- start 2024-05-27T09:35:23.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.962+0000 7fa1c6a00640 1 -- start start 2024-05-27T09:35:23.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.964+0000 7fa1c6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1c00741f0 0x7fa1c0155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:23.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.964+0000 7fa1c6a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1c0074840 0x7fa1c00cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:23.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.964+0000 7fa1c6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1c01547d0 0x7fa1c014a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:23.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.964+0000 7fa1c6a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c0073790 con 0x7fa1c00741f0 2024-05-27T09:35:23.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.964+0000 7fa1c6a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c00734b0 con 0x7fa1c01547d0 2024-05-27T09:35:23.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.964+0000 7fa1c6a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c0073cb0 con 0x7fa1c0074840 2024-05-27T09:35:23.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.964+0000 7fa1c5600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1c00741f0 0x7fa1c0155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:23.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.964+0000 7fa1c5600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1c00741f0 0x7fa1c0155c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46056/0 (socket says 172.21.15.114:46056) 2024-05-27T09:35:23.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.964+0000 7fa1c6000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1c01547d0 0x7fa1c014a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:23.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.964+0000 7fa1c5600640 1 -- 172.21.15.114:0/640858113 learned_addr learned my addr 172.21.15.114:0/640858113 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:23.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.964+0000 7fa1c4c00640 1 --2- 172.21.15.114:0/640858113 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1c0074840 0x7fa1c00cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:23.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.965+0000 7fa1c6000640 1 -- 172.21.15.114:0/640858113 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1c0074840 msgr2=0x7fa1c00cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:23.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.965+0000 7fa1c6000640 1 --2- 172.21.15.114:0/640858113 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1c0074840 0x7fa1c00cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:23.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.965+0000 7fa1c6000640 1 -- 172.21.15.114:0/640858113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1c00741f0 msgr2=0x7fa1c0155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:23.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.965+0000 7fa1c6000640 1 --2- 172.21.15.114:0/640858113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1c00741f0 0x7fa1c0155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:23.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.965+0000 7fa1c6000640 1 -- 172.21.15.114:0/640858113 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c01561b0 con 0x7fa1c01547d0 2024-05-27T09:35:23.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.965+0000 7fa1c4c00640 1 --2- 172.21.15.114:0/640858113 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1c0074840 0x7fa1c00cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:35:23.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.965+0000 7fa1c6000640 1 --2- 172.21.15.114:0/640858113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1c01547d0 0x7fa1c014a120 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8030000 tx=0x7fa1b8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:23.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.965+0000 7fa1bd600640 1 -- 172.21.15.114:0/640858113 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b803f020 con 0x7fa1c01547d0 2024-05-27T09:35:23.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.965+0000 7fa1bd600640 1 -- 172.21.15.114:0/640858113 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1b8037040 con 0x7fa1c01547d0 2024-05-27T09:35:23.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.966+0000 7fa1c6a00640 1 -- 172.21.15.114:0/640858113 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c0159fb0 con 0x7fa1c01547d0 2024-05-27T09:35:23.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.966+0000 7fa1bd600640 1 -- 172.21.15.114:0/640858113 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8036070 con 0x7fa1c01547d0 2024-05-27T09:35:23.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.966+0000 7fa1c6a00640 1 -- 172.21.15.114:0/640858113 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c01508a0 con 0x7fa1c01547d0 2024-05-27T09:35:23.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.967+0000 7fa1c6a00640 1 -- 172.21.15.114:0/640858113 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1c0159d20 con 0x7fa1c01547d0 2024-05-27T09:35:23.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.968+0000 7fa1bd600640 1 -- 172.21.15.114:0/640858113 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa1b8042020 con 0x7fa1c01547d0 2024-05-27T09:35:23.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.969+0000 7fa1bd600640 1 --2- 172.21.15.114:0/640858113 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa1a0076300 0x7fa1a0078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:23.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.969+0000 7fa1bd600640 1 -- 172.21.15.114:0/640858113 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6082+0+0 (secure 0 0 0) 0x7fa1b80b8210 con 0x7fa1c01547d0 2024-05-27T09:35:23.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.969+0000 7fa1c5600640 1 --2- 172.21.15.114:0/640858113 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa1a0076300 0x7fa1a0078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:23.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.970+0000 7fa1c5600640 1 --2- 172.21.15.114:0/640858113 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa1a0076300 0x7fa1a0078810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009ae0 tx=0x7fa1b4009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:23.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:23.972+0000 7fa1bd600640 1 -- 172.21.15.114:0/640858113 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa1b8049070 con 0x7fa1c01547d0 2024-05-27T09:35:24.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:24.233+0000 7fa1c6a00640 1 -- 172.21.15.114:0/640858113 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1c014e8e0 con 0x7fa1c01547d0 2024-05-27T09:35:24.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:24.234+0000 7fa1bd600640 1 -- 172.21.15.114:0/640858113 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+6890 (secure 0 0 0) 0x7fa1b8081410 con 0x7fa1c01547d0 2024-05-27T09:35:24.235 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:35:24.235 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5160,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:30.127436+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086,4875,4828]},"id":1}]} 2024-05-27T09:35:24.235 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 36 2024-05-27T09:35:24.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:24.237+0000 7fa1c6a00640 1 -- 172.21.15.114:0/640858113 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa1a0076300 msgr2=0x7fa1a0078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:24.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:24.237+0000 7fa1c6a00640 1 --2- 172.21.15.114:0/640858113 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa1a0076300 0x7fa1a0078810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009ae0 tx=0x7fa1b4009190 comp rx=0 tx=0).stop 2024-05-27T09:35:24.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:24.237+0000 7fa1c6a00640 1 -- 172.21.15.114:0/640858113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1c01547d0 msgr2=0x7fa1c014a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:24.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:24.237+0000 7fa1c6a00640 1 --2- 172.21.15.114:0/640858113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1c01547d0 0x7fa1c014a120 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8030000 tx=0x7fa1b8004a40 comp rx=0 tx=0).stop 2024-05-27T09:35:24.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:24.237+0000 7fa1c6a00640 1 -- 172.21.15.114:0/640858113 shutdown_connections 2024-05-27T09:35:24.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:24.237+0000 7fa1c6a00640 1 --2- 172.21.15.114:0/640858113 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa1a0076300 0x7fa1a0078810 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:24.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:24.237+0000 7fa1c6a00640 1 --2- 172.21.15.114:0/640858113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1c01547d0 0x7fa1c014a120 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:24.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:24.237+0000 7fa1c6a00640 1 --2- 172.21.15.114:0/640858113 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1c0074840 0x7fa1c00cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:24.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:24.237+0000 7fa1c6a00640 1 --2- 172.21.15.114:0/640858113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa1c00741f0 0x7fa1c0155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:24.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:24.237+0000 7fa1c6a00640 1 -- 172.21.15.114:0/640858113 >> 172.21.15.114:0/640858113 conn(0x7fa1c00ca320 msgr2=0x7fa1c0151580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:24.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:24.237+0000 7fa1c6a00640 1 -- 172.21.15.114:0/640858113 shutdown_connections 2024-05-27T09:35:24.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:24.237+0000 7fa1c6a00640 1 -- 172.21.15.114:0/640858113 wait complete. 2024-05-27T09:35:24.262 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-05-27T09:35:26.263 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:35:26.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.516+0000 7feb74400640 1 Processor -- start 2024-05-27T09:35:26.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.517+0000 7feb74400640 1 -- start start 2024-05-27T09:35:26.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.518+0000 7feb74400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb6c154820 0x7feb6c0ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:26.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.518+0000 7feb74400640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb6c0cac20 con 0x7feb6c074a60 2024-05-27T09:35:26.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.518+0000 7feb74400640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb6c15a810 con 0x7feb6c0ca810 2024-05-27T09:35:26.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.518+0000 7feb74400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7feb6c154c30 con 0x7feb6c154820 2024-05-27T09:35:26.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.519+0000 7feb71a00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7feb6c074a60 0x7feb6c073260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:37836/0 (socket says 172.21.15.114:37836) 2024-05-27T09:35:26.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.519+0000 7feb71a00640 1 -- 172.21.15.114:0/4241548541 learned_addr learned my addr 172.21.15.114:0/4241548541 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:35:26.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.519+0000 7feb71000640 1 --2- 172.21.15.114:0/4241548541 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb6c154820 0x7feb6c0ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:26.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.519+0000 7feb6be00640 1 -- 172.21.15.114:0/4241548541 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2214544151 0 0) 0x7feb6c0cac20 con 0x7feb6c074a60 2024-05-27T09:35:26.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.519+0000 7feb6be00640 1 -- 172.21.15.114:0/4241548541 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb54003520 con 0x7feb6c074a60 2024-05-27T09:35:26.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.519+0000 7feb71000640 1 -- 172.21.15.114:0/4241548541 >> v1:172.21.15.163:6789/0 conn(0x7feb6c0ca810 legacy=0x7feb6c15dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:26.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.519+0000 7feb6be00640 1 -- 172.21.15.114:0/4241548541 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 338531609 0 0) 0x7feb6c15a810 con 0x7feb6c0ca810 2024-05-27T09:35:26.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.519+0000 7feb71000640 1 -- 172.21.15.114:0/4241548541 >> v1:172.21.15.114:6789/0 conn(0x7feb6c074a60 legacy=0x7feb6c073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:26.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.519+0000 7feb71000640 1 -- 172.21.15.114:0/4241548541 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c15aaf0 con 0x7feb6c154820 2024-05-27T09:35:26.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.521+0000 7feb71000640 1 --2- 172.21.15.114:0/4241548541 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb6c154820 0x7feb6c0ca2d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feb580097d0 tx=0x7feb5802a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a88781858e757b26 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:26.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.521+0000 7feb6be00640 1 -- 172.21.15.114:0/4241548541 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58037070 con 0x7feb6c154820 2024-05-27T09:35:26.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.521+0000 7feb6be00640 1 -- 172.21.15.114:0/4241548541 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb58032440 con 0x7feb6c154820 2024-05-27T09:35:26.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.521+0000 7feb6be00640 1 -- 172.21.15.114:0/4241548541 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb580327f0 con 0x7feb6c154820 2024-05-27T09:35:26.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.522+0000 7feb74400640 1 -- 172.21.15.114:0/4241548541 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb6c154820 msgr2=0x7feb6c0ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:26.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.522+0000 7feb74400640 1 --2- 172.21.15.114:0/4241548541 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb6c154820 0x7feb6c0ca2d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feb580097d0 tx=0x7feb5802a290 comp rx=0 tx=0).stop 2024-05-27T09:35:26.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.522+0000 7feb74400640 1 -- 172.21.15.114:0/4241548541 shutdown_connections 2024-05-27T09:35:26.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.522+0000 7feb74400640 1 --2- 172.21.15.114:0/4241548541 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb6c154820 0x7feb6c0ca2d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:26.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.522+0000 7feb74400640 1 -- 172.21.15.114:0/4241548541 >> 172.21.15.114:0/4241548541 conn(0x7feb6c06f6e0 msgr2=0x7feb6c070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:26.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.522+0000 7feb74400640 1 -- 172.21.15.114:0/4241548541 shutdown_connections 2024-05-27T09:35:26.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.522+0000 7feb74400640 1 -- 172.21.15.114:0/4241548541 wait complete. 2024-05-27T09:35:26.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.524+0000 7feb74400640 1 Processor -- start 2024-05-27T09:35:26.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.524+0000 7feb74400640 1 -- start start 2024-05-27T09:35:26.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.526+0000 7feb74400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb6c074a60 0x7feb6c140060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:26.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.526+0000 7feb74400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb6c0ca810 0x7feb6c1405a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:26.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.526+0000 7feb74400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb6c154820 0x7feb6c152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:26.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.526+0000 7feb74400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c15b0b0 con 0x7feb6c074a60 2024-05-27T09:35:26.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.526+0000 7feb74400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c15db60 con 0x7feb6c0ca810 2024-05-27T09:35:26.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.526+0000 7feb74400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7feb6c15b390 con 0x7feb6c154820 2024-05-27T09:35:26.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.526+0000 7feb71a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb6c074a60 0x7feb6c140060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:26.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.526+0000 7feb71a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb6c074a60 0x7feb6c140060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36222/0 (socket says 172.21.15.114:36222) 2024-05-27T09:35:26.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.526+0000 7feb71a00640 1 -- 172.21.15.114:0/4165871886 learned_addr learned my addr 172.21.15.114:0/4165871886 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:26.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.526+0000 7feb71000640 1 --2- 172.21.15.114:0/4165871886 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb6c0ca810 0x7feb6c1405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:26.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.526+0000 7feb72400640 1 --2- 172.21.15.114:0/4165871886 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb6c154820 0x7feb6c152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:26.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.527+0000 7feb71a00640 1 -- 172.21.15.114:0/4165871886 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb6c154820 msgr2=0x7feb6c152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:26.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.527+0000 7feb71a00640 1 --2- 172.21.15.114:0/4165871886 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb6c154820 0x7feb6c152120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:26.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.527+0000 7feb71a00640 1 -- 172.21.15.114:0/4165871886 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb6c0ca810 msgr2=0x7feb6c1405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:26.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.527+0000 7feb71a00640 1 --2- 172.21.15.114:0/4165871886 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb6c0ca810 0x7feb6c1405a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:26.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.527+0000 7feb71a00640 1 -- 172.21.15.114:0/4165871886 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c0ce740 con 0x7feb6c074a60 2024-05-27T09:35:26.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.527+0000 7feb71000640 1 --2- 172.21.15.114:0/4165871886 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb6c0ca810 0x7feb6c1405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:35:26.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.527+0000 7feb71a00640 1 --2- 172.21.15.114:0/4165871886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb6c074a60 0x7feb6c140060 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7feb6000c410 tx=0x7feb6000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:26.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.527+0000 7feb69600640 1 -- 172.21.15.114:0/4165871886 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60017070 con 0x7feb6c074a60 2024-05-27T09:35:26.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.527+0000 7feb69600640 1 -- 172.21.15.114:0/4165871886 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb60005ce0 con 0x7feb6c074a60 2024-05-27T09:35:26.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.527+0000 7feb74400640 1 -- 172.21.15.114:0/4165871886 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6c0c6db0 con 0x7feb6c074a60 2024-05-27T09:35:26.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.527+0000 7feb69600640 1 -- 172.21.15.114:0/4165871886 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb600072d0 con 0x7feb6c074a60 2024-05-27T09:35:26.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.527+0000 7feb74400640 1 -- 172.21.15.114:0/4165871886 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6c0c7370 con 0x7feb6c074a60 2024-05-27T09:35:26.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.529+0000 7feb74400640 1 -- 172.21.15.114:0/4165871886 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6c06a380 con 0x7feb6c074a60 2024-05-27T09:35:26.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.530+0000 7feb69600640 1 -- 172.21.15.114:0/4165871886 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7feb6001a070 con 0x7feb6c074a60 2024-05-27T09:35:26.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.531+0000 7feb69600640 1 --2- 172.21.15.114:0/4165871886 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7feb4c076300 0x7feb4c078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:26.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.531+0000 7feb69600640 1 -- 172.21.15.114:0/4165871886 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6082+0+0 (secure 0 0 0) 0x7feb60096bf0 con 0x7feb6c074a60 2024-05-27T09:35:26.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.531+0000 7feb71000640 1 --2- 172.21.15.114:0/4165871886 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7feb4c076300 0x7feb4c078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:26.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.532+0000 7feb71000640 1 --2- 172.21.15.114:0/4165871886 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7feb4c076300 0x7feb4c078810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7feb58034000 tx=0x7feb5802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:26.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.536+0000 7feb69600640 1 -- 172.21.15.114:0/4165871886 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7feb6005fc50 con 0x7feb6c074a60 2024-05-27T09:35:26.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.799+0000 7feb74400640 1 -- 172.21.15.114:0/4165871886 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb6c14e930 con 0x7feb6c074a60 2024-05-27T09:35:26.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.800+0000 7feb69600640 1 -- 172.21.15.114:0/4165871886 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+6890 (secure 0 0 0) 0x7feb60063f50 con 0x7feb6c074a60 2024-05-27T09:35:26.801 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:35:26.801 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5160,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:30.127436+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086,4875,4828]},"id":1}]} 2024-05-27T09:35:26.801 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 36 2024-05-27T09:35:26.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.803+0000 7feb74400640 1 -- 172.21.15.114:0/4165871886 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7feb4c076300 msgr2=0x7feb4c078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:26.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.803+0000 7feb74400640 1 --2- 172.21.15.114:0/4165871886 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7feb4c076300 0x7feb4c078810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7feb58034000 tx=0x7feb5802e000 comp rx=0 tx=0).stop 2024-05-27T09:35:26.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.803+0000 7feb74400640 1 -- 172.21.15.114:0/4165871886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb6c074a60 msgr2=0x7feb6c140060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:26.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.803+0000 7feb74400640 1 --2- 172.21.15.114:0/4165871886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb6c074a60 0x7feb6c140060 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7feb6000c410 tx=0x7feb6000c8d0 comp rx=0 tx=0).stop 2024-05-27T09:35:26.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.803+0000 7feb74400640 1 -- 172.21.15.114:0/4165871886 shutdown_connections 2024-05-27T09:35:26.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.803+0000 7feb74400640 1 --2- 172.21.15.114:0/4165871886 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7feb4c076300 0x7feb4c078810 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:26.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.803+0000 7feb74400640 1 --2- 172.21.15.114:0/4165871886 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb6c154820 0x7feb6c152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:26.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.803+0000 7feb74400640 1 --2- 172.21.15.114:0/4165871886 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb6c0ca810 0x7feb6c1405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:26.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.803+0000 7feb74400640 1 --2- 172.21.15.114:0/4165871886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb6c074a60 0x7feb6c140060 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:26.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.803+0000 7feb74400640 1 -- 172.21.15.114:0/4165871886 >> 172.21.15.114:0/4165871886 conn(0x7feb6c06f6e0 msgr2=0x7feb6c071460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:26.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.803+0000 7feb74400640 1 -- 172.21.15.114:0/4165871886 shutdown_connections 2024-05-27T09:35:26.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:26.803+0000 7feb74400640 1 -- 172.21.15.114:0/4165871886 wait complete. 2024-05-27T09:35:26.827 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-05-27T09:35:28.827 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:35:29.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.129+0000 7f55a0e00640 1 Processor -- start 2024-05-27T09:35:29.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.130+0000 7f55a0e00640 1 -- start start 2024-05-27T09:35:29.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.132+0000 7f55a0e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f559c153140 0x7f559c0b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:29.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.132+0000 7f55a0e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f559c0cdb50 0x7f559c0ce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:29.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.133+0000 7f55a0e00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f559c15c930 con 0x7f559c0cee50 2024-05-27T09:35:29.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.133+0000 7f55a0e00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f559c15cc10 con 0x7f559c0cdb50 2024-05-27T09:35:29.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.133+0000 7f55a0e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f559c15cef0 con 0x7f559c153140 2024-05-27T09:35:29.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.133+0000 7f559aa00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f559c0cdb50 0x7f559c0ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:29.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.133+0000 7f559aa00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f559c0cdb50 0x7f559c0ce910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36224/0 (socket says 172.21.15.114:36224) 2024-05-27T09:35:29.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.133+0000 7f559be00640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f559c0cee50 0x7f559c0ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:46292/0 (socket says 172.21.15.114:46292) 2024-05-27T09:35:29.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.133+0000 7f559aa00640 1 -- 172.21.15.114:0/599680829 learned_addr learned my addr 172.21.15.114:0/599680829 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:29.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.133+0000 7f559b400640 1 --2- 172.21.15.114:0/599680829 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f559c153140 0x7f559c0b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:29.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.134+0000 7f559b400640 1 -- 172.21.15.114:0/599680829 >> v2:172.21.15.114:3300/0 conn(0x7f559c0cdb50 msgr2=0x7f559c0ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:35:29.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.134+0000 7f559b400640 1 --2- 172.21.15.114:0/599680829 >> v2:172.21.15.114:3300/0 conn(0x7f559c0cdb50 0x7f559c0ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:29.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.134+0000 7f559b400640 1 -- 172.21.15.114:0/599680829 >> v1:172.21.15.163:6789/0 conn(0x7f559c0cee50 legacy=0x7f559c0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:29.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.134+0000 7f559b400640 1 -- 172.21.15.114:0/599680829 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559c15d1d0 con 0x7f559c153140 2024-05-27T09:35:29.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.135+0000 7f559b400640 1 --2- 172.21.15.114:0/599680829 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f559c153140 0x7f559c0b45c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f558c0097a0 tx=0x7f558c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf7d4e2199371f81 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:29.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.135+0000 7f559a000640 1 -- 172.21.15.114:0/599680829 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c037070 con 0x7f559c153140 2024-05-27T09:35:29.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.135+0000 7f559a000640 1 -- 172.21.15.114:0/599680829 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f558c032440 con 0x7f559c153140 2024-05-27T09:35:29.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.135+0000 7f559a000640 1 -- 172.21.15.114:0/599680829 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c032810 con 0x7f559c153140 2024-05-27T09:35:29.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.136+0000 7f55a0e00640 1 -- 172.21.15.114:0/599680829 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f559c153140 msgr2=0x7f559c0b45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:29.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.136+0000 7f55a0e00640 1 --2- 172.21.15.114:0/599680829 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f559c153140 0x7f559c0b45c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f558c0097a0 tx=0x7f558c02a210 comp rx=0 tx=0).stop 2024-05-27T09:35:29.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.136+0000 7f55a0e00640 1 -- 172.21.15.114:0/599680829 shutdown_connections 2024-05-27T09:35:29.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.136+0000 7f55a0e00640 1 --2- 172.21.15.114:0/599680829 >> v2:172.21.15.114:3300/0 conn(0x7f559c0cdb50 0x7f559c0ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:29.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.136+0000 7f55a0e00640 1 --2- 172.21.15.114:0/599680829 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f559c153140 0x7f559c0b45c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:29.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.136+0000 7f55a0e00640 1 -- 172.21.15.114:0/599680829 >> 172.21.15.114:0/599680829 conn(0x7f559c08d960 msgr2=0x7f559c08dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:29.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.136+0000 7f55a0e00640 1 -- 172.21.15.114:0/599680829 shutdown_connections 2024-05-27T09:35:29.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.137+0000 7f55a0e00640 1 -- 172.21.15.114:0/599680829 wait complete. 2024-05-27T09:35:29.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.139+0000 7f55a0e00640 1 Processor -- start 2024-05-27T09:35:29.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.139+0000 7f55a0e00640 1 -- start start 2024-05-27T09:35:29.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.140+0000 7f55a0e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f559c0cdb50 0x7f559c0d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:29.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.140+0000 7f55a0e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f559c0cee50 0x7f559c1e5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:29.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.140+0000 7f55a0e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f559c153140 0x7f559c0cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:29.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.141+0000 7f55a0e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f559c0cd660 con 0x7f559c0cdb50 2024-05-27T09:35:29.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.141+0000 7f55a0e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f559c0cd4e0 con 0x7f559c153140 2024-05-27T09:35:29.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.141+0000 7f55a0e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f559c0cc250 con 0x7f559c0cee50 2024-05-27T09:35:29.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.141+0000 7f559aa00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f559c0cee50 0x7f559c1e5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:29.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.141+0000 7f559b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f559c0cdb50 0x7f559c0d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:29.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.141+0000 7f559aa00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f559c0cee50 0x7f559c1e5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:44262/0 (socket says 172.21.15.114:44262) 2024-05-27T09:35:29.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.141+0000 7f559aa00640 1 -- 172.21.15.114:0/3003827186 learned_addr learned my addr 172.21.15.114:0/3003827186 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:29.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.141+0000 7f559be00640 1 --2- 172.21.15.114:0/3003827186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f559c153140 0x7f559c0cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:29.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.141+0000 7f559aa00640 1 -- 172.21.15.114:0/3003827186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f559c153140 msgr2=0x7f559c0cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:29.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.141+0000 7f559aa00640 1 --2- 172.21.15.114:0/3003827186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f559c153140 0x7f559c0cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:29.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.141+0000 7f559aa00640 1 -- 172.21.15.114:0/3003827186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f559c0cdb50 msgr2=0x7f559c0d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:29.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.141+0000 7f559aa00640 1 --2- 172.21.15.114:0/3003827186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f559c0cdb50 0x7f559c0d10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:29.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.141+0000 7f559aa00640 1 -- 172.21.15.114:0/3003827186 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559c0d15f0 con 0x7f559c0cee50 2024-05-27T09:35:29.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.142+0000 7f559aa00640 1 --2- 172.21.15.114:0/3003827186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f559c0cee50 0x7f559c1e5ae0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5588009f10 tx=0x7f558800c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:29.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.142+0000 7f5593400640 1 -- 172.21.15.114:0/3003827186 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5588023070 con 0x7f559c0cee50 2024-05-27T09:35:29.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.142+0000 7f5593400640 1 -- 172.21.15.114:0/3003827186 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5588015bd0 con 0x7f559c0cee50 2024-05-27T09:35:29.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.142+0000 7f55a0e00640 1 -- 172.21.15.114:0/3003827186 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559c1f2e30 con 0x7f559c0cee50 2024-05-27T09:35:29.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.142+0000 7f5593400640 1 -- 172.21.15.114:0/3003827186 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5588016050 con 0x7f559c0cee50 2024-05-27T09:35:29.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.142+0000 7f55a0e00640 1 -- 172.21.15.114:0/3003827186 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f559c1f33f0 con 0x7f559c0cee50 2024-05-27T09:35:29.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.144+0000 7f55a0e00640 1 -- 172.21.15.114:0/3003827186 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f559c050860 con 0x7f559c0cee50 2024-05-27T09:35:29.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.146+0000 7f5593400640 1 -- 172.21.15.114:0/3003827186 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5588026070 con 0x7f559c0cee50 2024-05-27T09:35:29.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.148+0000 7f5593400640 1 --2- 172.21.15.114:0/3003827186 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f557c076300 0x7f557c078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:29.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.148+0000 7f5593400640 1 -- 172.21.15.114:0/3003827186 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(48..48 src has 1..48) ==== 6082+0+0 (secure 0 0 0) 0x7f558809faf0 con 0x7f559c0cee50 2024-05-27T09:35:29.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.148+0000 7f559b400640 1 --2- 172.21.15.114:0/3003827186 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f557c076300 0x7f557c078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:29.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.149+0000 7f559b400640 1 --2- 172.21.15.114:0/3003827186 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f557c076300 0x7f557c078810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f558c034000 tx=0x7f558c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:29.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.150+0000 7f5593400640 1 -- 172.21.15.114:0/3003827186 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5588046020 con 0x7f559c0cee50 2024-05-27T09:35:29.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.401+0000 7f55a0e00640 1 -- 172.21.15.114:0/3003827186 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f559c15a7f0 con 0x7f559c0cee50 2024-05-27T09:35:29.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.402+0000 7f5593400640 1 -- 172.21.15.114:0/3003827186 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+6890 (secure 0 0 0) 0x7f5588068c60 con 0x7f559c0cee50 2024-05-27T09:35:29.403 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:35:29.403 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5160,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:30.127436+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086,4875,4828]},"id":1}]} 2024-05-27T09:35:29.404 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 36 2024-05-27T09:35:29.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.405+0000 7f55a0e00640 1 -- 172.21.15.114:0/3003827186 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f557c076300 msgr2=0x7f557c078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:29.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.405+0000 7f55a0e00640 1 --2- 172.21.15.114:0/3003827186 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f557c076300 0x7f557c078810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f558c034000 tx=0x7f558c02e000 comp rx=0 tx=0).stop 2024-05-27T09:35:29.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.405+0000 7f55a0e00640 1 -- 172.21.15.114:0/3003827186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f559c0cee50 msgr2=0x7f559c1e5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:29.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.405+0000 7f55a0e00640 1 --2- 172.21.15.114:0/3003827186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f559c0cee50 0x7f559c1e5ae0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5588009f10 tx=0x7f558800c520 comp rx=0 tx=0).stop 2024-05-27T09:35:29.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.405+0000 7f55a0e00640 1 -- 172.21.15.114:0/3003827186 shutdown_connections 2024-05-27T09:35:29.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.405+0000 7f55a0e00640 1 --2- 172.21.15.114:0/3003827186 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f557c076300 0x7f557c078810 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:29.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.405+0000 7f55a0e00640 1 --2- 172.21.15.114:0/3003827186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f559c153140 0x7f559c0cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:29.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.405+0000 7f55a0e00640 1 --2- 172.21.15.114:0/3003827186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f559c0cee50 0x7f559c1e5ae0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:29.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.405+0000 7f55a0e00640 1 --2- 172.21.15.114:0/3003827186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f559c0cdb50 0x7f559c0d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:29.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.405+0000 7f55a0e00640 1 -- 172.21.15.114:0/3003827186 >> 172.21.15.114:0/3003827186 conn(0x7f559c08d960 msgr2=0x7f559c0c6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:29.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.405+0000 7f55a0e00640 1 -- 172.21.15.114:0/3003827186 shutdown_connections 2024-05-27T09:35:29.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:29.405+0000 7f55a0e00640 1 -- 172.21.15.114:0/3003827186 wait complete. 2024-05-27T09:35:29.429 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-05-27T09:35:31.429 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:35:31.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.696+0000 7fbec6800640 1 Processor -- start 2024-05-27T09:35:31.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.697+0000 7fbec6800640 1 -- start start 2024-05-27T09:35:31.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.699+0000 7fbec6800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbec0153140 0x7fbec00b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:31.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.699+0000 7fbec6800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbec00cdb50 0x7fbec00ce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:31.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.699+0000 7fbec6800640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbec015c930 con 0x7fbec00cee50 2024-05-27T09:35:31.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.700+0000 7fbec6800640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fbec015cc10 con 0x7fbec0153140 2024-05-27T09:35:31.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.700+0000 7fbec6800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fbec015cef0 con 0x7fbec00cdb50 2024-05-27T09:35:31.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.700+0000 7fbec5400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbec0153140 0x7fbec00b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:31.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.700+0000 7fbec5e00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fbec00cee50 0x7fbec00ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:37848/0 (socket says 172.21.15.114:37848) 2024-05-27T09:35:31.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.700+0000 7fbec5e00640 1 -- 172.21.15.114:0/2354305051 learned_addr learned my addr 172.21.15.114:0/2354305051 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:35:31.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.700+0000 7fbec4a00640 1 --2- 172.21.15.114:0/2354305051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbec00cdb50 0x7fbec00ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:31.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.700+0000 7fbebfe00640 1 -- 172.21.15.114:0/2354305051 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 272361695 0 0) 0x7fbec015c930 con 0x7fbec00cee50 2024-05-27T09:35:31.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.700+0000 7fbebfe00640 1 -- 172.21.15.114:0/2354305051 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbea8003520 con 0x7fbec00cee50 2024-05-27T09:35:31.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.701+0000 7fbec5400640 1 -- 172.21.15.114:0/2354305051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbec00cdb50 msgr2=0x7fbec00ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:35:31.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.701+0000 7fbec5400640 1 --2- 172.21.15.114:0/2354305051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbec00cdb50 0x7fbec00ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:31.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.701+0000 7fbec5400640 1 -- 172.21.15.114:0/2354305051 >> v1:172.21.15.114:6789/0 conn(0x7fbec00cee50 legacy=0x7fbec00ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:31.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.701+0000 7fbec5400640 1 -- 172.21.15.114:0/2354305051 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec015d1d0 con 0x7fbec0153140 2024-05-27T09:35:31.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.701+0000 7fbec4a00640 1 --2- 172.21.15.114:0/2354305051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbec00cdb50 0x7fbec00ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:35:31.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.702+0000 7fbec5400640 1 --2- 172.21.15.114:0/2354305051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbec0153140 0x7fbec00b45c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbeb40097a0 tx=0x7fbeb402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c46ce88c829e863 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:31.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.702+0000 7fbebfe00640 1 -- 172.21.15.114:0/2354305051 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4037070 con 0x7fbec0153140 2024-05-27T09:35:31.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.702+0000 7fbebfe00640 1 -- 172.21.15.114:0/2354305051 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeb4032440 con 0x7fbec0153140 2024-05-27T09:35:31.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.702+0000 7fbebfe00640 1 -- 172.21.15.114:0/2354305051 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4032810 con 0x7fbec0153140 2024-05-27T09:35:31.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.703+0000 7fbec6800640 1 -- 172.21.15.114:0/2354305051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbec0153140 msgr2=0x7fbec00b45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:31.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.703+0000 7fbec6800640 1 --2- 172.21.15.114:0/2354305051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbec0153140 0x7fbec00b45c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbeb40097a0 tx=0x7fbeb402a210 comp rx=0 tx=0).stop 2024-05-27T09:35:31.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.703+0000 7fbec6800640 1 -- 172.21.15.114:0/2354305051 shutdown_connections 2024-05-27T09:35:31.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.703+0000 7fbec6800640 1 --2- 172.21.15.114:0/2354305051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbec00cdb50 0x7fbec00ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:31.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.703+0000 7fbec6800640 1 --2- 172.21.15.114:0/2354305051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbec0153140 0x7fbec00b45c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:31.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.703+0000 7fbec6800640 1 -- 172.21.15.114:0/2354305051 >> 172.21.15.114:0/2354305051 conn(0x7fbec008d960 msgr2=0x7fbec008dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:31.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.703+0000 7fbec6800640 1 -- 172.21.15.114:0/2354305051 shutdown_connections 2024-05-27T09:35:31.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.703+0000 7fbec6800640 1 -- 172.21.15.114:0/2354305051 wait complete. 2024-05-27T09:35:31.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.705+0000 7fbec6800640 1 Processor -- start 2024-05-27T09:35:31.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.705+0000 7fbec6800640 1 -- start start 2024-05-27T09:35:31.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.708+0000 7fbec6800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbec00cdb50 0x7fbec0150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:31.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.708+0000 7fbec6800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbec00cee50 0x7fbec015d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:31.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.708+0000 7fbec6800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbec0153140 0x7fbec00cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:31.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.708+0000 7fbec6800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbec00cd660 con 0x7fbec0153140 2024-05-27T09:35:31.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.708+0000 7fbec6800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbec00cd4e0 con 0x7fbec00cee50 2024-05-27T09:35:31.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.708+0000 7fbec6800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fbec00cc250 con 0x7fbec00cdb50 2024-05-27T09:35:31.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.708+0000 7fbec4a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbec00cee50 0x7fbec015d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:31.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.708+0000 7fbec5e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbec0153140 0x7fbec00cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:31.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.708+0000 7fbec5400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbec00cdb50 0x7fbec0150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:31.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.708+0000 7fbec4a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbec00cee50 0x7fbec015d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:34218/0 (socket says 172.21.15.114:34218) 2024-05-27T09:35:31.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.708+0000 7fbec4a00640 1 -- 172.21.15.114:0/2255278561 learned_addr learned my addr 172.21.15.114:0/2255278561 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:31.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.709+0000 7fbec4a00640 1 -- 172.21.15.114:0/2255278561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbec00cdb50 msgr2=0x7fbec0150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:31.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.709+0000 7fbec4a00640 1 --2- 172.21.15.114:0/2255278561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbec00cdb50 0x7fbec0150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:31.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.709+0000 7fbec4a00640 1 -- 172.21.15.114:0/2255278561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbec0153140 msgr2=0x7fbec00cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:31.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.709+0000 7fbec4a00640 1 --2- 172.21.15.114:0/2255278561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbec0153140 0x7fbec00cabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:31.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.709+0000 7fbec4a00640 1 -- 172.21.15.114:0/2255278561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec01520f0 con 0x7fbec00cee50 2024-05-27T09:35:31.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.709+0000 7fbec5400640 1 --2- 172.21.15.114:0/2255278561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbec00cdb50 0x7fbec0150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:35:31.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.709+0000 7fbec4a00640 1 --2- 172.21.15.114:0/2255278561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbec00cee50 0x7fbec015d6b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00e960 tx=0x7fbeac00ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:31.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.709+0000 7fbebd600640 1 -- 172.21.15.114:0/2255278561 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac017070 con 0x7fbec00cee50 2024-05-27T09:35:31.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.709+0000 7fbebd600640 1 -- 172.21.15.114:0/2255278561 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeac00ca00 con 0x7fbec00cee50 2024-05-27T09:35:31.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.709+0000 7fbec6800640 1 -- 172.21.15.114:0/2255278561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec01f5c10 con 0x7fbec00cee50 2024-05-27T09:35:31.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.709+0000 7fbebd600640 1 -- 172.21.15.114:0/2255278561 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac00cdd0 con 0x7fbec00cee50 2024-05-27T09:35:31.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.709+0000 7fbec6800640 1 -- 172.21.15.114:0/2255278561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec01f5eb0 con 0x7fbec00cee50 2024-05-27T09:35:31.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.711+0000 7fbec6800640 1 -- 172.21.15.114:0/2255278561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbec0050860 con 0x7fbec00cee50 2024-05-27T09:35:31.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.712+0000 7fbebd600640 1 -- 172.21.15.114:0/2255278561 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbeac01a070 con 0x7fbec00cee50 2024-05-27T09:35:31.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.713+0000 7fbebd600640 1 --2- 172.21.15.114:0/2255278561 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbea00762d0 0x7fbea00787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:31.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.713+0000 7fbebd600640 1 -- 172.21.15.114:0/2255278561 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6082+0+0 (secure 0 0 0) 0x7fbeac0975f0 con 0x7fbec00cee50 2024-05-27T09:35:31.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.713+0000 7fbec5400640 1 --2- 172.21.15.114:0/2255278561 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbea00762d0 0x7fbea00787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:31.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.714+0000 7fbec5400640 1 --2- 172.21.15.114:0/2255278561 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbea00762d0 0x7fbea00787e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4034000 tx=0x7fbeb402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:31.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.718+0000 7fbebd600640 1 -- 172.21.15.114:0/2255278561 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbeac0606e0 con 0x7fbec00cee50 2024-05-27T09:35:31.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.997+0000 7fbec6800640 1 -- 172.21.15.114:0/2255278561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbec00cd930 con 0x7fbec00cee50 2024-05-27T09:35:32.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:31.999+0000 7fbebd600640 1 -- 172.21.15.114:0/2255278561 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+6890 (secure 0 0 0) 0x7fbeac0649e0 con 0x7fbec00cee50 2024-05-27T09:35:32.000 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:35:32.000 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5160,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:30.127436+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086,4875,4828]},"id":1}]} 2024-05-27T09:35:32.000 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 36 2024-05-27T09:35:32.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:32.002+0000 7fbec6800640 1 -- 172.21.15.114:0/2255278561 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbea00762d0 msgr2=0x7fbea00787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:32.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:32.002+0000 7fbec6800640 1 --2- 172.21.15.114:0/2255278561 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbea00762d0 0x7fbea00787e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4034000 tx=0x7fbeb402e000 comp rx=0 tx=0).stop 2024-05-27T09:35:32.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:32.002+0000 7fbec6800640 1 -- 172.21.15.114:0/2255278561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbec00cee50 msgr2=0x7fbec015d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:32.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:32.002+0000 7fbec6800640 1 --2- 172.21.15.114:0/2255278561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbec00cee50 0x7fbec015d6b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00e960 tx=0x7fbeac00ee20 comp rx=0 tx=0).stop 2024-05-27T09:35:32.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:32.002+0000 7fbec6800640 1 -- 172.21.15.114:0/2255278561 shutdown_connections 2024-05-27T09:35:32.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:32.002+0000 7fbec6800640 1 --2- 172.21.15.114:0/2255278561 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbea00762d0 0x7fbea00787e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:32.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:32.002+0000 7fbec6800640 1 --2- 172.21.15.114:0/2255278561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbec0153140 0x7fbec00cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:32.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:32.002+0000 7fbec6800640 1 --2- 172.21.15.114:0/2255278561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbec00cee50 0x7fbec015d6b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:32.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:32.002+0000 7fbec6800640 1 --2- 172.21.15.114:0/2255278561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbec00cdb50 0x7fbec0150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:32.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:32.002+0000 7fbec6800640 1 -- 172.21.15.114:0/2255278561 >> 172.21.15.114:0/2255278561 conn(0x7fbec008d960 msgr2=0x7fbec00c81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:32.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:32.002+0000 7fbec6800640 1 -- 172.21.15.114:0/2255278561 shutdown_connections 2024-05-27T09:35:32.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:32.002+0000 7fbec6800640 1 -- 172.21.15.114:0/2255278561 wait complete. 2024-05-27T09:35:32.033 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-05-27T09:35:34.034 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:35:34.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.362+0000 7f3ba2800640 1 Processor -- start 2024-05-27T09:35:34.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.362+0000 7f3ba2800640 1 -- start start 2024-05-27T09:35:34.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.365+0000 7f3ba2800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b9c0c8370 0x7f3b9c152f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:34.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.365+0000 7f3ba2800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f3b9c0ca400 0x7f3b9c0ca810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:34.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.365+0000 7f3ba2800640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b9c0c42f0 con 0x7f3b9c0c9170 2024-05-27T09:35:34.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.365+0000 7f3ba2800640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f3b9c0c9b40 con 0x7f3b9c0ca400 2024-05-27T09:35:34.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.365+0000 7f3ba2800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f3b9c0c2b20 con 0x7f3b9c0c8370 2024-05-27T09:35:34.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.365+0000 7f3b9b400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f3b9c0c9170 0x7f3b9c15c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:37854/0 (socket says 172.21.15.114:37854) 2024-05-27T09:35:34.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.365+0000 7f3b9b400640 1 -- 172.21.15.114:0/1754423090 learned_addr learned my addr 172.21.15.114:0/1754423090 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:35:34.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.365+0000 7f3b9be00640 1 --2- 172.21.15.114:0/1754423090 >> v2:172.21.15.114:3300/0 conn(0x7f3b9c0ca400 0x7f3b9c0ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:34.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.366+0000 7f3b9aa00640 1 --2- 172.21.15.114:0/1754423090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b9c0c8370 0x7f3b9c152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:34.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.366+0000 7f3b9be00640 1 -- 172.21.15.114:0/1754423090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b9c0c8370 msgr2=0x7f3b9c152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:35:34.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.366+0000 7f3b9be00640 1 --2- 172.21.15.114:0/1754423090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b9c0c8370 0x7f3b9c152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:34.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.366+0000 7f3b9be00640 1 -- 172.21.15.114:0/1754423090 >> v1:172.21.15.114:6789/0 conn(0x7f3b9c0c9170 legacy=0x7f3b9c15c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:34.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.366+0000 7f3b9be00640 1 -- 172.21.15.114:0/1754423090 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9c0c45d0 con 0x7f3b9c0ca400 2024-05-27T09:35:34.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.367+0000 7f3b9be00640 1 --2- 172.21.15.114:0/1754423090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b9c0ca400 0x7f3b9c0ca810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3b94009d90 tx=0x7f3b9402c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f95d1b38b9f8526 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:34.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.367+0000 7f3b9a000640 1 -- 172.21.15.114:0/1754423090 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b94036020 con 0x7f3b9c0ca400 2024-05-27T09:35:34.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.367+0000 7f3b9a000640 1 -- 172.21.15.114:0/1754423090 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b94032800 con 0x7f3b9c0ca400 2024-05-27T09:35:34.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.368+0000 7f3b9a000640 1 -- 172.21.15.114:0/1754423090 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b94032bd0 con 0x7f3b9c0ca400 2024-05-27T09:35:34.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.368+0000 7f3ba2800640 1 -- 172.21.15.114:0/1754423090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b9c0ca400 msgr2=0x7f3b9c0ca810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:34.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.368+0000 7f3ba2800640 1 --2- 172.21.15.114:0/1754423090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b9c0ca400 0x7f3b9c0ca810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3b94009d90 tx=0x7f3b9402c900 comp rx=0 tx=0).stop 2024-05-27T09:35:34.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.369+0000 7f3ba2800640 1 -- 172.21.15.114:0/1754423090 shutdown_connections 2024-05-27T09:35:34.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.369+0000 7f3ba2800640 1 --2- 172.21.15.114:0/1754423090 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b9c0ca400 0x7f3b9c0ca810 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:34.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.369+0000 7f3ba2800640 1 --2- 172.21.15.114:0/1754423090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b9c0c8370 0x7f3b9c152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:34.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.369+0000 7f3ba2800640 1 -- 172.21.15.114:0/1754423090 >> 172.21.15.114:0/1754423090 conn(0x7f3b9c06f6e0 msgr2=0x7f3b9c070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:34.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.369+0000 7f3ba2800640 1 -- 172.21.15.114:0/1754423090 shutdown_connections 2024-05-27T09:35:34.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.369+0000 7f3ba2800640 1 -- 172.21.15.114:0/1754423090 wait complete. 2024-05-27T09:35:34.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.372+0000 7f3ba2800640 1 Processor -- start 2024-05-27T09:35:34.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.373+0000 7f3ba2800640 1 -- start start 2024-05-27T09:35:34.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.375+0000 7f3ba2800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b9c0c8370 0x7f3b9c1f2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:34.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.375+0000 7f3ba2800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b9c0c9170 0x7f3b9c15b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:34.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.375+0000 7f3ba2800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b9c0ca400 0x7f3b9c148310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:34.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.375+0000 7f3ba2800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c0c9fd0 con 0x7f3b9c0ca400 2024-05-27T09:35:34.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.375+0000 7f3ba2800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c0c97d0 con 0x7f3b9c0c9170 2024-05-27T09:35:34.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.375+0000 7f3ba2800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f3b9c0ca150 con 0x7f3b9c0c8370 2024-05-27T09:35:34.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.375+0000 7f3b9b400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b9c0c8370 0x7f3b9c1f2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:34.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.375+0000 7f3b9b400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b9c0c8370 0x7f3b9c1f2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:44304/0 (socket says 172.21.15.114:44304) 2024-05-27T09:35:34.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.375+0000 7f3b9b400640 1 -- 172.21.15.114:0/1304341351 learned_addr learned my addr 172.21.15.114:0/1304341351 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:34.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.375+0000 7f3b9aa00640 1 --2- 172.21.15.114:0/1304341351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b9c0c9170 0x7f3b9c15b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:34.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.375+0000 7f3b9be00640 1 --2- 172.21.15.114:0/1304341351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b9c0ca400 0x7f3b9c148310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:34.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.376+0000 7f3b9b400640 1 -- 172.21.15.114:0/1304341351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b9c0c9170 msgr2=0x7f3b9c15b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:34.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.376+0000 7f3b9b400640 1 --2- 172.21.15.114:0/1304341351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b9c0c9170 0x7f3b9c15b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:34.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.376+0000 7f3b9b400640 1 -- 172.21.15.114:0/1304341351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b9c0ca400 msgr2=0x7f3b9c148310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:34.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.376+0000 7f3b9b400640 1 --2- 172.21.15.114:0/1304341351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b9c0ca400 0x7f3b9c148310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:34.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.376+0000 7f3b9b400640 1 -- 172.21.15.114:0/1304341351 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9c1f2eb0 con 0x7f3b9c0c8370 2024-05-27T09:35:34.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.376+0000 7f3b9be00640 1 --2- 172.21.15.114:0/1304341351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b9c0ca400 0x7f3b9c148310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:35:34.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.377+0000 7f3b9b400640 1 --2- 172.21.15.114:0/1304341351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b9c0c8370 0x7f3b9c1f2970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c001eb0 tx=0x7f3b8c00c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:34.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.377+0000 7f3b93400640 1 -- 172.21.15.114:0/1304341351 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c017070 con 0x7f3b9c0c8370 2024-05-27T09:35:34.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.377+0000 7f3b93400640 1 -- 172.21.15.114:0/1304341351 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b8c005d00 con 0x7f3b9c0c8370 2024-05-27T09:35:34.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.377+0000 7f3ba2800640 1 -- 172.21.15.114:0/1304341351 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b9c156e80 con 0x7f3b9c0c8370 2024-05-27T09:35:34.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.377+0000 7f3b93400640 1 -- 172.21.15.114:0/1304341351 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c007310 con 0x7f3b9c0c8370 2024-05-27T09:35:34.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.377+0000 7f3ba2800640 1 -- 172.21.15.114:0/1304341351 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b9c153c10 con 0x7f3b9c0c8370 2024-05-27T09:35:34.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.379+0000 7f3ba2800640 1 -- 172.21.15.114:0/1304341351 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b9c071c30 con 0x7f3b9c0c8370 2024-05-27T09:35:34.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.380+0000 7f3b93400640 1 -- 172.21.15.114:0/1304341351 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3b8c01a070 con 0x7f3b9c0c8370 2024-05-27T09:35:34.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.381+0000 7f3b93400640 1 --2- 172.21.15.114:0/1304341351 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f3b7c0762d0 0x7f3b7c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:34.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.381+0000 7f3b93400640 1 -- 172.21.15.114:0/1304341351 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(48..48 src has 1..48) ==== 6082+0+0 (secure 0 0 0) 0x7f3b8c096ac0 con 0x7f3b9c0c8370 2024-05-27T09:35:34.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.381+0000 7f3b9aa00640 1 --2- 172.21.15.114:0/1304341351 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f3b7c0762d0 0x7f3b7c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:34.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.382+0000 7f3b9aa00640 1 --2- 172.21.15.114:0/1304341351 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f3b7c0762d0 0x7f3b7c0787e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0b2950 tx=0x7f3b88008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:34.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.386+0000 7f3b93400640 1 -- 172.21.15.114:0/1304341351 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3b9c071c30 con 0x7f3b9c0c8370 2024-05-27T09:35:34.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.711+0000 7f3ba2800640 1 -- 172.21.15.114:0/1304341351 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b9c06a380 con 0x7f3b9c0c8370 2024-05-27T09:35:34.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.712+0000 7f3b93400640 1 -- 172.21.15.114:0/1304341351 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+6890 (secure 0 0 0) 0x7f3b8c05fc40 con 0x7f3b9c0c8370 2024-05-27T09:35:34.714 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:35:34.714 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5160,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:30.127436+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086,4875,4828]},"id":1}]} 2024-05-27T09:35:34.714 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 36 2024-05-27T09:35:34.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.715+0000 7f3ba2800640 1 -- 172.21.15.114:0/1304341351 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f3b7c0762d0 msgr2=0x7f3b7c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:34.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.715+0000 7f3ba2800640 1 --2- 172.21.15.114:0/1304341351 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f3b7c0762d0 0x7f3b7c0787e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0b2950 tx=0x7f3b88008040 comp rx=0 tx=0).stop 2024-05-27T09:35:34.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.715+0000 7f3ba2800640 1 -- 172.21.15.114:0/1304341351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b9c0c8370 msgr2=0x7f3b9c1f2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:34.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.715+0000 7f3ba2800640 1 --2- 172.21.15.114:0/1304341351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b9c0c8370 0x7f3b9c1f2970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c001eb0 tx=0x7f3b8c00c890 comp rx=0 tx=0).stop 2024-05-27T09:35:34.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.716+0000 7f3ba2800640 1 -- 172.21.15.114:0/1304341351 shutdown_connections 2024-05-27T09:35:34.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.716+0000 7f3ba2800640 1 --2- 172.21.15.114:0/1304341351 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f3b7c0762d0 0x7f3b7c0787e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:34.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.716+0000 7f3ba2800640 1 --2- 172.21.15.114:0/1304341351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3b9c0ca400 0x7f3b9c148310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:34.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.716+0000 7f3ba2800640 1 --2- 172.21.15.114:0/1304341351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b9c0c9170 0x7f3b9c15b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:34.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.716+0000 7f3ba2800640 1 --2- 172.21.15.114:0/1304341351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b9c0c8370 0x7f3b9c1f2970 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:34.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.716+0000 7f3ba2800640 1 -- 172.21.15.114:0/1304341351 >> 172.21.15.114:0/1304341351 conn(0x7f3b9c06f6e0 msgr2=0x7f3b9c149240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:34.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.716+0000 7f3ba2800640 1 -- 172.21.15.114:0/1304341351 shutdown_connections 2024-05-27T09:35:34.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:34.716+0000 7f3ba2800640 1 -- 172.21.15.114:0/1304341351 wait complete. 2024-05-27T09:35:34.759 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-05-27T09:35:36.759 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:35:36.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.993+0000 7fc1f7200640 1 Processor -- start 2024-05-27T09:35:36.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.993+0000 7fc1f7200640 1 -- start start 2024-05-27T09:35:36.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.995+0000 7fc1f7200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1f80cc350 0x7fc1f80cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:36.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.995+0000 7fc1f7200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc1f80cb740 0x7fc1f80cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:36.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.995+0000 7fc1f7200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1f806e1c0 con 0x7fc1f8156980 2024-05-27T09:35:36.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.995+0000 7fc1f7200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fc1f806ed40 con 0x7fc1f80cb740 2024-05-27T09:35:36.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.995+0000 7fc1f7200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fc1f806e4a0 con 0x7fc1f80cc350 2024-05-27T09:35:36.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.996+0000 7fc1f5e00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fc1f8156980 0x7fc1f8160330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:52124/0 (socket says 172.21.15.114:52124) 2024-05-27T09:35:36.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.996+0000 7fc1f5e00640 1 -- 172.21.15.114:0/2566258600 learned_addr learned my addr 172.21.15.114:0/2566258600 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:35:36.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.996+0000 7fc1f5400640 1 --2- 172.21.15.114:0/2566258600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1f80cc350 0x7fc1f80cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:36.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.996+0000 7fc1f6800640 1 --2- 172.21.15.114:0/2566258600 >> v2:172.21.15.163:3300/0 conn(0x7fc1f80cb740 0x7fc1f80cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:36.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.996+0000 7fc1f4a00640 1 -- 172.21.15.114:0/2566258600 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3556570249 0 0) 0x7fc1f806e1c0 con 0x7fc1f8156980 2024-05-27T09:35:36.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.997+0000 7fc1f4a00640 1 -- 172.21.15.114:0/2566258600 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1dc003520 con 0x7fc1f8156980 2024-05-27T09:35:36.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.997+0000 7fc1f5400640 1 -- 172.21.15.114:0/2566258600 >> v2:172.21.15.163:3300/0 conn(0x7fc1f80cb740 msgr2=0x7fc1f80cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:35:36.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.997+0000 7fc1f5400640 1 --2- 172.21.15.114:0/2566258600 >> v2:172.21.15.163:3300/0 conn(0x7fc1f80cb740 0x7fc1f80cbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:36.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.997+0000 7fc1f5400640 1 -- 172.21.15.114:0/2566258600 >> v1:172.21.15.114:6789/0 conn(0x7fc1f8156980 legacy=0x7fc1f8160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:36.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.997+0000 7fc1f5400640 1 -- 172.21.15.114:0/2566258600 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f806e780 con 0x7fc1f80cc350 2024-05-27T09:35:36.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.997+0000 7fc1f6800640 1 --2- 172.21.15.114:0/2566258600 >> v2:172.21.15.163:3300/0 conn(0x7fc1f80cb740 0x7fc1f80cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:35:36.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.998+0000 7fc1f5400640 1 --2- 172.21.15.114:0/2566258600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1f80cc350 0x7fc1f80cc760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40097a0 tx=0x7fc1e402a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f32656eee7ad964 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:37.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.998+0000 7fc1f4a00640 1 -- 172.21.15.114:0/2566258600 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4037070 con 0x7fc1f80cc350 2024-05-27T09:35:37.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.998+0000 7fc1f4a00640 1 -- 172.21.15.114:0/2566258600 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1e402ae40 con 0x7fc1f80cc350 2024-05-27T09:35:37.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.998+0000 7fc1f4a00640 1 -- 172.21.15.114:0/2566258600 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4032660 con 0x7fc1f80cc350 2024-05-27T09:35:37.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.999+0000 7fc1f7200640 1 -- 172.21.15.114:0/2566258600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1f80cc350 msgr2=0x7fc1f80cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:37.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.999+0000 7fc1f7200640 1 --2- 172.21.15.114:0/2566258600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1f80cc350 0x7fc1f80cc760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40097a0 tx=0x7fc1e402a0f0 comp rx=0 tx=0).stop 2024-05-27T09:35:37.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.999+0000 7fc1f7200640 1 -- 172.21.15.114:0/2566258600 shutdown_connections 2024-05-27T09:35:37.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.999+0000 7fc1f7200640 1 --2- 172.21.15.114:0/2566258600 >> v2:172.21.15.163:3300/0 conn(0x7fc1f80cb740 0x7fc1f80cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:37.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:36.999+0000 7fc1f7200640 1 --2- 172.21.15.114:0/2566258600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1f80cc350 0x7fc1f80cc760 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:37.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.000+0000 7fc1f7200640 1 -- 172.21.15.114:0/2566258600 >> 172.21.15.114:0/2566258600 conn(0x7fc1f8071460 msgr2=0x7fc1f8068600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:37.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.000+0000 7fc1f7200640 1 -- 172.21.15.114:0/2566258600 shutdown_connections 2024-05-27T09:35:37.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.000+0000 7fc1f7200640 1 -- 172.21.15.114:0/2566258600 wait complete. 2024-05-27T09:35:37.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.002+0000 7fc1f7200640 1 Processor -- start 2024-05-27T09:35:37.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.002+0000 7fc1f7200640 1 -- start start 2024-05-27T09:35:37.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.004+0000 7fc1f7200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1f80cb740 0x7fc1f8157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:37.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.004+0000 7fc1f7200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1f80cc350 0x7fc1f815d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:37.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.004+0000 7fc1f7200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1f8156980 0x7fc1f80b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:37.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.004+0000 7fc1f7200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f80cc090 con 0x7fc1f8156980 2024-05-27T09:35:37.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.004+0000 7fc1f7200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f815fe10 con 0x7fc1f80cc350 2024-05-27T09:35:37.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.004+0000 7fc1f7200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fc1f8156d90 con 0x7fc1f80cb740 2024-05-27T09:35:37.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.004+0000 7fc1f6800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1f8156980 0x7fc1f80b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:37.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.004+0000 7fc1f6800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1f8156980 0x7fc1f80b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39192/0 (socket says 172.21.15.114:39192) 2024-05-27T09:35:37.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.004+0000 7fc1f6800640 1 -- 172.21.15.114:0/4197675687 learned_addr learned my addr 172.21.15.114:0/4197675687 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:37.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.004+0000 7fc1f5e00640 1 --2- 172.21.15.114:0/4197675687 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1f80cb740 0x7fc1f8157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:37.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.004+0000 7fc1f5400640 1 --2- 172.21.15.114:0/4197675687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1f80cc350 0x7fc1f815d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:37.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.005+0000 7fc1f5e00640 1 -- 172.21.15.114:0/4197675687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1f80cc350 msgr2=0x7fc1f815d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:37.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.005+0000 7fc1f5e00640 1 --2- 172.21.15.114:0/4197675687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1f80cc350 0x7fc1f815d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:37.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.005+0000 7fc1f5e00640 1 -- 172.21.15.114:0/4197675687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1f8156980 msgr2=0x7fc1f80b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:37.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.005+0000 7fc1f5e00640 1 --2- 172.21.15.114:0/4197675687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1f8156980 0x7fc1f80b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:37.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.005+0000 7fc1f5e00640 1 -- 172.21.15.114:0/4197675687 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f8157140 con 0x7fc1f80cb740 2024-05-27T09:35:37.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.005+0000 7fc1f5400640 1 --2- 172.21.15.114:0/4197675687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1f80cc350 0x7fc1f815d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:35:37.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.005+0000 7fc1f5e00640 1 --2- 172.21.15.114:0/4197675687 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1f80cb740 0x7fc1f8157ec0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800c390 tx=0x7fc1e800c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:37.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.005+0000 7fc1ee000640 1 -- 172.21.15.114:0/4197675687 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8017070 con 0x7fc1f80cb740 2024-05-27T09:35:37.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.005+0000 7fc1ee000640 1 -- 172.21.15.114:0/4197675687 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1e8005ce0 con 0x7fc1f80cb740 2024-05-27T09:35:37.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.005+0000 7fc1ee000640 1 -- 172.21.15.114:0/4197675687 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e80072d0 con 0x7fc1f80cb740 2024-05-27T09:35:37.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.005+0000 7fc1f7200640 1 -- 172.21.15.114:0/4197675687 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f8156060 con 0x7fc1f80cb740 2024-05-27T09:35:37.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.006+0000 7fc1f7200640 1 -- 172.21.15.114:0/4197675687 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f81576c0 con 0x7fc1f80cb740 2024-05-27T09:35:37.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.007+0000 7fc1f7200640 1 -- 172.21.15.114:0/4197675687 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1f806d5d0 con 0x7fc1f80cb740 2024-05-27T09:35:37.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.009+0000 7fc1ee000640 1 -- 172.21.15.114:0/4197675687 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc1e801a070 con 0x7fc1f80cb740 2024-05-27T09:35:37.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.010+0000 7fc1ee000640 1 --2- 172.21.15.114:0/4197675687 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc1d80762d0 0x7fc1d80787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:37.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.010+0000 7fc1ee000640 1 -- 172.21.15.114:0/4197675687 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(48..48 src has 1..48) ==== 6082+0+0 (secure 0 0 0) 0x7fc1e8096ac0 con 0x7fc1f80cb740 2024-05-27T09:35:37.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.010+0000 7fc1f5400640 1 --2- 172.21.15.114:0/4197675687 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc1d80762d0 0x7fc1d80787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:37.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.011+0000 7fc1f5400640 1 --2- 172.21.15.114:0/4197675687 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc1d80762d0 0x7fc1d80787e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4034000 tx=0x7fc1e402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:37.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.016+0000 7fc1ee000640 1 -- 172.21.15.114:0/4197675687 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc1e805fb60 con 0x7fc1f80cb740 2024-05-27T09:35:37.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.257+0000 7fc1f7200640 1 -- 172.21.15.114:0/4197675687 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1f8050860 con 0x7fc1f80cb740 2024-05-27T09:35:37.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.258+0000 7fc1ee000640 1 -- 172.21.15.114:0/4197675687 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+6890 (secure 0 0 0) 0x7fc1e8096e20 con 0x7fc1f80cb740 2024-05-27T09:35:37.260 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:35:37.260 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5160,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:30.127436+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086,4875,4828]},"id":1}]} 2024-05-27T09:35:37.260 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 36 2024-05-27T09:35:37.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.260+0000 7fc1f7200640 1 -- 172.21.15.114:0/4197675687 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc1d80762d0 msgr2=0x7fc1d80787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:37.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.260+0000 7fc1f7200640 1 --2- 172.21.15.114:0/4197675687 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc1d80762d0 0x7fc1d80787e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4034000 tx=0x7fc1e402e000 comp rx=0 tx=0).stop 2024-05-27T09:35:37.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.260+0000 7fc1f7200640 1 -- 172.21.15.114:0/4197675687 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1f80cb740 msgr2=0x7fc1f8157ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:37.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.260+0000 7fc1f7200640 1 --2- 172.21.15.114:0/4197675687 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1f80cb740 0x7fc1f8157ec0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800c390 tx=0x7fc1e800c850 comp rx=0 tx=0).stop 2024-05-27T09:35:37.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.260+0000 7fc1f7200640 1 -- 172.21.15.114:0/4197675687 shutdown_connections 2024-05-27T09:35:37.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.260+0000 7fc1f7200640 1 --2- 172.21.15.114:0/4197675687 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc1d80762d0 0x7fc1d80787e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:37.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.260+0000 7fc1f7200640 1 --2- 172.21.15.114:0/4197675687 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1f8156980 0x7fc1f80b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:37.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.260+0000 7fc1f7200640 1 --2- 172.21.15.114:0/4197675687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1f80cc350 0x7fc1f815d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:37.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.260+0000 7fc1f7200640 1 --2- 172.21.15.114:0/4197675687 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1f80cb740 0x7fc1f8157ec0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:37.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.260+0000 7fc1f7200640 1 -- 172.21.15.114:0/4197675687 >> 172.21.15.114:0/4197675687 conn(0x7fc1f8071460 msgr2=0x7fc1f8150790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:37.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.261+0000 7fc1f7200640 1 -- 172.21.15.114:0/4197675687 shutdown_connections 2024-05-27T09:35:37.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:37.261+0000 7fc1f7200640 1 -- 172.21.15.114:0/4197675687 wait complete. 2024-05-27T09:35:37.284 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-05-27T09:35:39.285 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:35:39.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.671+0000 7f5fe4c00640 1 Processor -- start 2024-05-27T09:35:39.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.671+0000 7f5fe4c00640 1 -- start start 2024-05-27T09:35:39.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.673+0000 7f5fe4c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fe01547d0 0x7f5fe00c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:39.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.673+0000 7f5fe4c00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5fe0074840 0x7f5fe015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:39.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.673+0000 7f5fe4c00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fe006fe50 con 0x7f5fe00741f0 2024-05-27T09:35:39.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.673+0000 7f5fe4c00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f5fe00703d0 con 0x7f5fe0074840 2024-05-27T09:35:39.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.673+0000 7f5fe4c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe015b260 con 0x7f5fe01547d0 2024-05-27T09:35:39.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.673+0000 7f5fdf400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f5fe00741f0 0x7f5fe006edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:52126/0 (socket says 172.21.15.114:52126) 2024-05-27T09:35:39.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.673+0000 7f5fdf400640 1 -- 172.21.15.114:0/2188590588 learned_addr learned my addr 172.21.15.114:0/2188590588 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:35:39.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.674+0000 7f5fdfe00640 1 --2- 172.21.15.114:0/2188590588 >> v2:172.21.15.114:3300/0 conn(0x7f5fe0074840 0x7f5fe015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:39.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.674+0000 7f5fdea00640 1 --2- 172.21.15.114:0/2188590588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fe01547d0 0x7f5fe00c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:39.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.674+0000 7f5fde000640 1 -- 172.21.15.114:0/2188590588 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1429143087 0 0) 0x7f5fe006fe50 con 0x7f5fe00741f0 2024-05-27T09:35:39.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.674+0000 7f5fde000640 1 -- 172.21.15.114:0/2188590588 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5fc4003520 con 0x7f5fe00741f0 2024-05-27T09:35:39.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.674+0000 7f5fdea00640 1 -- 172.21.15.114:0/2188590588 >> v2:172.21.15.114:3300/0 conn(0x7f5fe0074840 msgr2=0x7f5fe015ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:35:39.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.674+0000 7f5fdea00640 1 --2- 172.21.15.114:0/2188590588 >> v2:172.21.15.114:3300/0 conn(0x7f5fe0074840 0x7f5fe015ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:39.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.674+0000 7f5fdea00640 1 -- 172.21.15.114:0/2188590588 >> v1:172.21.15.114:6789/0 conn(0x7f5fe00741f0 legacy=0x7f5fe006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:39.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.674+0000 7f5fdea00640 1 -- 172.21.15.114:0/2188590588 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe00700f0 con 0x7f5fe01547d0 2024-05-27T09:35:39.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.676+0000 7f5fdea00640 1 --2- 172.21.15.114:0/2188590588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fe01547d0 0x7f5fe00c63a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0097a0 tx=0x7f5fcc02a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8668118c1b120be1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:39.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.676+0000 7f5fde000640 1 -- 172.21.15.114:0/2188590588 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc037070 con 0x7f5fe01547d0 2024-05-27T09:35:39.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.676+0000 7f5fde000640 1 -- 172.21.15.114:0/2188590588 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fcc032440 con 0x7f5fe01547d0 2024-05-27T09:35:39.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.676+0000 7f5fde000640 1 -- 172.21.15.114:0/2188590588 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc0327f0 con 0x7f5fe01547d0 2024-05-27T09:35:39.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.677+0000 7f5fe4c00640 1 -- 172.21.15.114:0/2188590588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fe01547d0 msgr2=0x7f5fe00c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:39.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.677+0000 7f5fe4c00640 1 --2- 172.21.15.114:0/2188590588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fe01547d0 0x7f5fe00c63a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0097a0 tx=0x7f5fcc02a230 comp rx=0 tx=0).stop 2024-05-27T09:35:39.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.677+0000 7f5fe4c00640 1 -- 172.21.15.114:0/2188590588 shutdown_connections 2024-05-27T09:35:39.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.677+0000 7f5fe4c00640 1 --2- 172.21.15.114:0/2188590588 >> v2:172.21.15.114:3300/0 conn(0x7f5fe0074840 0x7f5fe015ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:39.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.678+0000 7f5fe4c00640 1 --2- 172.21.15.114:0/2188590588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fe01547d0 0x7f5fe00c63a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:39.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.678+0000 7f5fe4c00640 1 -- 172.21.15.114:0/2188590588 >> 172.21.15.114:0/2188590588 conn(0x7f5fe00ca320 msgr2=0x7f5fe00ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:39.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.678+0000 7f5fe4c00640 1 -- 172.21.15.114:0/2188590588 shutdown_connections 2024-05-27T09:35:39.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.678+0000 7f5fe4c00640 1 -- 172.21.15.114:0/2188590588 wait complete. 2024-05-27T09:35:39.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.681+0000 7f5fe4c00640 1 Processor -- start 2024-05-27T09:35:39.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.681+0000 7f5fe4c00640 1 -- start start 2024-05-27T09:35:39.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.684+0000 7f5fe4c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fe00741f0 0x7f5fe015bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:39.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.684+0000 7f5fe4c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fe0074840 0x7f5fe00c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:39.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.684+0000 7f5fe4c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fe01547d0 0x7f5fe0154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:39.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.684+0000 7f5fe4c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe0073790 con 0x7f5fe01547d0 2024-05-27T09:35:39.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.684+0000 7f5fe4c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe00734b0 con 0x7f5fe0074840 2024-05-27T09:35:39.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.684+0000 7f5fe4c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe006f500 con 0x7f5fe00741f0 2024-05-27T09:35:39.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.684+0000 7f5fdf400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fe00741f0 0x7f5fe015bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:39.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.684+0000 7f5fdf400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fe00741f0 0x7f5fe015bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:34562/0 (socket says 172.21.15.114:34562) 2024-05-27T09:35:39.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.684+0000 7f5fdf400640 1 -- 172.21.15.114:0/2286377283 learned_addr learned my addr 172.21.15.114:0/2286377283 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:39.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.684+0000 7f5fdea00640 1 --2- 172.21.15.114:0/2286377283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fe0074840 0x7f5fe00c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:39.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.684+0000 7f5fdfe00640 1 --2- 172.21.15.114:0/2286377283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fe01547d0 0x7f5fe0154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:39.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.685+0000 7f5fdf400640 1 -- 172.21.15.114:0/2286377283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fe0074840 msgr2=0x7f5fe00c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:39.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.685+0000 7f5fdf400640 1 --2- 172.21.15.114:0/2286377283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fe0074840 0x7f5fe00c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:39.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.685+0000 7f5fdf400640 1 -- 172.21.15.114:0/2286377283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fe01547d0 msgr2=0x7f5fe0154c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:39.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.685+0000 7f5fdf400640 1 --2- 172.21.15.114:0/2286377283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fe01547d0 0x7f5fe0154c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:39.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.685+0000 7f5fdf400640 1 -- 172.21.15.114:0/2286377283 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe015c000 con 0x7f5fe00741f0 2024-05-27T09:35:39.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.685+0000 7f5fdf400640 1 --2- 172.21.15.114:0/2286377283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fe00741f0 0x7f5fe015bac0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000c400 tx=0x7f5fd000c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:39.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.685+0000 7f5fd7400640 1 -- 172.21.15.114:0/2286377283 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0017070 con 0x7f5fe00741f0 2024-05-27T09:35:39.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.685+0000 7f5fd7400640 1 -- 172.21.15.114:0/2286377283 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fd0005ce0 con 0x7f5fe00741f0 2024-05-27T09:35:39.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.685+0000 7f5fd7400640 1 -- 172.21.15.114:0/2286377283 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd00072d0 con 0x7f5fe00741f0 2024-05-27T09:35:39.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.685+0000 7f5fe4c00640 1 -- 172.21.15.114:0/2286377283 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe0157500 con 0x7f5fe00741f0 2024-05-27T09:35:39.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.686+0000 7f5fe4c00640 1 -- 172.21.15.114:0/2286377283 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe0157aa0 con 0x7f5fe00741f0 2024-05-27T09:35:39.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.687+0000 7f5fe4c00640 1 -- 172.21.15.114:0/2286377283 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe0157ca0 con 0x7f5fe00741f0 2024-05-27T09:35:39.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.688+0000 7f5fd7400640 1 -- 172.21.15.114:0/2286377283 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5fd001a070 con 0x7f5fe00741f0 2024-05-27T09:35:39.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.690+0000 7f5fd7400640 1 --2- 172.21.15.114:0/2286377283 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5fc00762d0 0x7f5fc00787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:39.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.690+0000 7f5fd7400640 1 -- 172.21.15.114:0/2286377283 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(48..48 src has 1..48) ==== 6082+0+0 (secure 0 0 0) 0x7f5fd0096b00 con 0x7f5fe00741f0 2024-05-27T09:35:39.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.690+0000 7f5fdea00640 1 --2- 172.21.15.114:0/2286377283 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5fc00762d0 0x7f5fc00787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:39.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.691+0000 7f5fdea00640 1 --2- 172.21.15.114:0/2286377283 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5fc00762d0 0x7f5fc00787e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc034000 tx=0x7f5fcc02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:39.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:39.694+0000 7f5fd7400640 1 -- 172.21.15.114:0/2286377283 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5fe0157ca0 con 0x7f5fe00741f0 2024-05-27T09:35:40.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:40.062+0000 7f5fe4c00640 1 -- 172.21.15.114:0/2286377283 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fe015b400 con 0x7f5fe00741f0 2024-05-27T09:35:40.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:40.063+0000 7f5fd7400640 1 -- 172.21.15.114:0/2286377283 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+6890 (secure 0 0 0) 0x7f5fd0096e20 con 0x7f5fe00741f0 2024-05-27T09:35:40.064 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:35:40.064 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5160,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:34:30.127436+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":4828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4828":{"gid":4828,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.163:6839/3371890103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":3371890103},{"type":"v1","addr":"172.21.15.163:6839","nonce":3371890103}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086,4875,4828]},"id":1}]} 2024-05-27T09:35:40.064 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 36 2024-05-27T09:35:40.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:40.065+0000 7f5fe4c00640 1 -- 172.21.15.114:0/2286377283 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5fc00762d0 msgr2=0x7f5fc00787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:40.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:40.065+0000 7f5fe4c00640 1 --2- 172.21.15.114:0/2286377283 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5fc00762d0 0x7f5fc00787e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc034000 tx=0x7f5fcc02e000 comp rx=0 tx=0).stop 2024-05-27T09:35:40.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:40.065+0000 7f5fe4c00640 1 -- 172.21.15.114:0/2286377283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fe00741f0 msgr2=0x7f5fe015bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:40.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:40.065+0000 7f5fe4c00640 1 --2- 172.21.15.114:0/2286377283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fe00741f0 0x7f5fe015bac0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000c400 tx=0x7f5fd000c8c0 comp rx=0 tx=0).stop 2024-05-27T09:35:40.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:40.065+0000 7f5fe4c00640 1 -- 172.21.15.114:0/2286377283 shutdown_connections 2024-05-27T09:35:40.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:40.065+0000 7f5fe4c00640 1 --2- 172.21.15.114:0/2286377283 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5fc00762d0 0x7f5fc00787e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:40.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:40.065+0000 7f5fe4c00640 1 --2- 172.21.15.114:0/2286377283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5fe01547d0 0x7f5fe0154c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:40.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:40.065+0000 7f5fe4c00640 1 --2- 172.21.15.114:0/2286377283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fe0074840 0x7f5fe00c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:40.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:40.065+0000 7f5fe4c00640 1 --2- 172.21.15.114:0/2286377283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fe00741f0 0x7f5fe015bac0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:40.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:40.065+0000 7f5fe4c00640 1 -- 172.21.15.114:0/2286377283 >> 172.21.15.114:0/2286377283 conn(0x7f5fe00ca320 msgr2=0x7f5fe00c7190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:40.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:40.065+0000 7f5fe4c00640 1 -- 172.21.15.114:0/2286377283 shutdown_connections 2024-05-27T09:35:40.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:40.065+0000 7f5fe4c00640 1 -- 172.21.15.114:0/2286377283 wait complete. 2024-05-27T09:35:40.092 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-05-27T09:35:42.092 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:35:42.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.504+0000 7f4376000640 1 Processor -- start 2024-05-27T09:35:42.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.504+0000 7f4376000640 1 -- start start 2024-05-27T09:35:42.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.508+0000 7f4376000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4370073780 0x7f4370152a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:42.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.508+0000 7f4376000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43700c4720 0x7f437015bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:42.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.509+0000 7f436fe00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f437015c1d0 0x7f43700ca330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:52138/0 (socket says 172.21.15.114:52138) 2024-05-27T09:35:42.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.509+0000 7f436fe00640 1 -- 172.21.15.114:0/400394657 learned_addr learned my addr 172.21.15.114:0/400394657 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:35:42.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.509+0000 7f436ea00640 1 --2- 172.21.15.114:0/400394657 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43700c4720 0x7f437015bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:42.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.509+0000 7f4376000640 1 -- 172.21.15.114:0/400394657 --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43701586a0 con 0x7f437015c1d0 2024-05-27T09:35:42.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.510+0000 7f4376000640 1 -- 172.21.15.114:0/400394657 --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f43700c8590 con 0x7f4370073780 2024-05-27T09:35:42.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.510+0000 7f4376000640 1 -- 172.21.15.114:0/400394657 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f4370073c50 con 0x7f43700c4720 2024-05-27T09:35:42.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.510+0000 7f436ea00640 1 -- 172.21.15.114:0/400394657 >> v2:172.21.15.163:3300/0 conn(0x7f4370073780 msgr2=0x7f4370152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:35:42.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.510+0000 7f436f400640 1 --2- 172.21.15.114:0/400394657 >> v2:172.21.15.163:3300/0 conn(0x7f4370073780 0x7f4370152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:42.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.511+0000 7f436ea00640 1 --2- 172.21.15.114:0/400394657 >> v2:172.21.15.163:3300/0 conn(0x7f4370073780 0x7f4370152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:42.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.511+0000 7f436ea00640 1 -- 172.21.15.114:0/400394657 >> v1:172.21.15.114:6789/0 conn(0x7f437015c1d0 legacy=0x7f43700ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:42.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.511+0000 7f436ea00640 1 -- 172.21.15.114:0/400394657 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4370158960 con 0x7f43700c4720 2024-05-27T09:35:42.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.511+0000 7f436f400640 1 --2- 172.21.15.114:0/400394657 >> v2:172.21.15.163:3300/0 conn(0x7f4370073780 0x7f4370152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:35:42.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.512+0000 7f436ea00640 1 --2- 172.21.15.114:0/400394657 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43700c4720 0x7f437015bc90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f43600097a0 tx=0x7f436002c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bec169781cfc53d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:42.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.514+0000 7f436e000640 1 -- 172.21.15.114:0/400394657 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360039070 con 0x7f43700c4720 2024-05-27T09:35:42.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.515+0000 7f436e000640 1 -- 172.21.15.114:0/400394657 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4360034640 con 0x7f43700c4720 2024-05-27T09:35:42.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.515+0000 7f436e000640 1 -- 172.21.15.114:0/400394657 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360034a10 con 0x7f43700c4720 2024-05-27T09:35:42.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.516+0000 7f4376000640 1 -- 172.21.15.114:0/400394657 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43700c4720 msgr2=0x7f437015bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:42.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.516+0000 7f4376000640 1 --2- 172.21.15.114:0/400394657 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43700c4720 0x7f437015bc90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f43600097a0 tx=0x7f436002c440 comp rx=0 tx=0).stop 2024-05-27T09:35:42.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.519+0000 7f4376000640 1 -- 172.21.15.114:0/400394657 shutdown_connections 2024-05-27T09:35:42.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.519+0000 7f4376000640 1 --2- 172.21.15.114:0/400394657 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43700c4720 0x7f437015bc90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:42.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.519+0000 7f4376000640 1 --2- 172.21.15.114:0/400394657 >> v2:172.21.15.163:3300/0 conn(0x7f4370073780 0x7f4370152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:42.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.519+0000 7f4376000640 1 -- 172.21.15.114:0/400394657 >> 172.21.15.114:0/400394657 conn(0x7f437006f6e0 msgr2=0x7f4370070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:42.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.520+0000 7f4376000640 1 -- 172.21.15.114:0/400394657 shutdown_connections 2024-05-27T09:35:42.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.520+0000 7f4376000640 1 -- 172.21.15.114:0/400394657 wait complete. 2024-05-27T09:35:42.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.524+0000 7f4376000640 1 Processor -- start 2024-05-27T09:35:42.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.524+0000 7f4376000640 1 -- start start 2024-05-27T09:35:42.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.526+0000 7f4376000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4370073780 0x7f43701f4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:42.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.526+0000 7f4376000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43700c4720 0x7f43700ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:42.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.526+0000 7f4376000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f437015c1d0 0x7f43701481c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:42.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.527+0000 7f4376000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4370159200 con 0x7f43700c4720 2024-05-27T09:35:42.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.527+0000 7f4376000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4370158f20 con 0x7f4370073780 2024-05-27T09:35:42.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.527+0000 7f4376000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f43700c4b30 con 0x7f437015c1d0 2024-05-27T09:35:42.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.527+0000 7f436f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4370073780 0x7f43701f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:42.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.527+0000 7f436f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4370073780 0x7f43701f4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:35376/0 (socket says 172.21.15.114:35376) 2024-05-27T09:35:42.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.527+0000 7f436f400640 1 -- 172.21.15.114:0/2211109740 learned_addr learned my addr 172.21.15.114:0/2211109740 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:42.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.527+0000 7f436fe00640 1 --2- 172.21.15.114:0/2211109740 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f437015c1d0 0x7f43701481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:42.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.527+0000 7f436f400640 1 -- 172.21.15.114:0/2211109740 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f437015c1d0 msgr2=0x7f43701481c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:42.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.527+0000 7f436f400640 1 --2- 172.21.15.114:0/2211109740 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f437015c1d0 0x7f43701481c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:42.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.527+0000 7f436f400640 1 -- 172.21.15.114:0/2211109740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43700c4720 msgr2=0x7f43700ca960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T09:35:42.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.527+0000 7f436f400640 1 --2- 172.21.15.114:0/2211109740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43700c4720 0x7f43700ca960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:42.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.527+0000 7f436f400640 1 -- 172.21.15.114:0/2211109740 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43701f47b0 con 0x7f4370073780 2024-05-27T09:35:42.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.528+0000 7f436f400640 1 --2- 172.21.15.114:0/2211109740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4370073780 0x7f43701f4270 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f436800ce30 tx=0x7f436800a650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:42.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.528+0000 7f4367400640 1 -- 172.21.15.114:0/2211109740 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436800f680 con 0x7f4370073780 2024-05-27T09:35:42.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.528+0000 7f4367400640 1 -- 172.21.15.114:0/2211109740 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f436800f820 con 0x7f4370073780 2024-05-27T09:35:42.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.528+0000 7f4367400640 1 -- 172.21.15.114:0/2211109740 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436800fbf0 con 0x7f4370073780 2024-05-27T09:35:42.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.529+0000 7f4376000640 1 -- 172.21.15.114:0/2211109740 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43700afc00 con 0x7f4370073780 2024-05-27T09:35:42.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.529+0000 7f4376000640 1 -- 172.21.15.114:0/2211109740 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43700b2030 con 0x7f4370073780 2024-05-27T09:35:42.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.531+0000 7f4376000640 1 -- 172.21.15.114:0/2211109740 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4370156c60 con 0x7f4370073780 2024-05-27T09:35:42.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.533+0000 7f4367400640 1 -- 172.21.15.114:0/2211109740 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4368018400 con 0x7f4370073780 2024-05-27T09:35:42.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.534+0000 7f4367400640 1 --2- 172.21.15.114:0/2211109740 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f43500762d0 0x7f43500787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:42.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.535+0000 7f4367400640 1 -- 172.21.15.114:0/2211109740 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6168+0+0 (secure 0 0 0) 0x7f4368098510 con 0x7f4370073780 2024-05-27T09:35:42.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.535+0000 7f436ea00640 1 --2- 172.21.15.114:0/2211109740 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f43500762d0 0x7f43500787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:42.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.536+0000 7f436ea00640 1 --2- 172.21.15.114:0/2211109740 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f43500762d0 0x7f43500787e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4360036000 tx=0x7f4360030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:42.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:42.539+0000 7f4367400640 1 -- 172.21.15.114:0/2211109740 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4370156c60 con 0x7f4370073780 2024-05-27T09:35:43.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.031+0000 7f4376000640 1 -- 172.21.15.114:0/2211109740 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f437006a380 con 0x7f4370073780 2024-05-27T09:35:43.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.032+0000 7f4367400640 1 -- 172.21.15.114:0/2211109740 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+6056 (secure 0 0 0) 0x7f4368014030 con 0x7f4370073780 2024-05-27T09:35:43.037 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:35:43.037 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:35:40.437274+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[4875,5086]},"id":1}]} 2024-05-27T09:35:43.038 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 37 2024-05-27T09:35:43.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.036+0000 7f4376000640 1 -- 172.21.15.114:0/2211109740 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f43500762d0 msgr2=0x7f43500787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:43.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.036+0000 7f4376000640 1 --2- 172.21.15.114:0/2211109740 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f43500762d0 0x7f43500787e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4360036000 tx=0x7f4360030000 comp rx=0 tx=0).stop 2024-05-27T09:35:43.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.036+0000 7f4376000640 1 -- 172.21.15.114:0/2211109740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4370073780 msgr2=0x7f43701f4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:43.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.036+0000 7f4376000640 1 --2- 172.21.15.114:0/2211109740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4370073780 0x7f43701f4270 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f436800ce30 tx=0x7f436800a650 comp rx=0 tx=0).stop 2024-05-27T09:35:43.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.037+0000 7f4376000640 1 -- 172.21.15.114:0/2211109740 shutdown_connections 2024-05-27T09:35:43.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.037+0000 7f4376000640 1 --2- 172.21.15.114:0/2211109740 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f43500762d0 0x7f43500787e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:43.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.037+0000 7f4376000640 1 --2- 172.21.15.114:0/2211109740 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f437015c1d0 0x7f43701481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:43.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.037+0000 7f4376000640 1 --2- 172.21.15.114:0/2211109740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43700c4720 0x7f43700ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:43.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.037+0000 7f4376000640 1 --2- 172.21.15.114:0/2211109740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4370073780 0x7f43701f4270 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:43.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.037+0000 7f4376000640 1 -- 172.21.15.114:0/2211109740 >> 172.21.15.114:0/2211109740 conn(0x7f437006f6e0 msgr2=0x7f43700ca070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:43.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.037+0000 7f4376000640 1 -- 172.21.15.114:0/2211109740 shutdown_connections 2024-05-27T09:35:43.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.038+0000 7f4376000640 1 -- 172.21.15.114:0/2211109740 wait complete. 2024-05-27T09:35:43.094 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2024-05-27T09:35:43.094 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-05-27T09:35:43.094 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:35:43.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.511+0000 7fa3ef200640 1 Processor -- start 2024-05-27T09:35:43.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.512+0000 7fa3ef200640 1 -- start start 2024-05-27T09:35:43.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.513+0000 7fa3ef200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa3f00ca5e0 0x7fa3f015d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:43.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.513+0000 7fa3ef200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3f0154ab0 con 0x7fa3f00724c0 2024-05-27T09:35:43.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.513+0000 7fa3ef200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3f00cbba0 con 0x7fa3f015afd0 2024-05-27T09:35:43.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.513+0000 7fa3ef200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fa3f00c9950 con 0x7fa3f00ca5e0 2024-05-27T09:35:43.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.514+0000 7fa3ed400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fa3f00724c0 0x7fa3f01543a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:52154/0 (socket says 172.21.15.114:52154) 2024-05-27T09:35:43.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.514+0000 7fa3ed400640 1 -- 172.21.15.114:0/2933889388 learned_addr learned my addr 172.21.15.114:0/2933889388 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:35:43.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.514+0000 7fa3ede00640 1 --2- 172.21.15.114:0/2933889388 >> v2:172.21.15.163:3300/0 conn(0x7fa3f00ca5e0 0x7fa3f015d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:43.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.515+0000 7fa3ede00640 1 -- 172.21.15.114:0/2933889388 >> v1:172.21.15.163:6789/0 conn(0x7fa3f015afd0 legacy=0x7fa3f00cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:43.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.515+0000 7fa3ede00640 1 -- 172.21.15.114:0/2933889388 >> v1:172.21.15.114:6789/0 conn(0x7fa3f00724c0 legacy=0x7fa3f01543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:43.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.515+0000 7fa3ede00640 1 -- 172.21.15.114:0/2933889388 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f00c93b0 con 0x7fa3f00ca5e0 2024-05-27T09:35:43.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.515+0000 7fa3eca00640 1 -- 172.21.15.114:0/2933889388 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3355976460 0 0) 0x7fa3f0154ab0 con 0x7fa3f00724c0 2024-05-27T09:35:43.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.516+0000 7fa3ede00640 1 --2- 172.21.15.114:0/2933889388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3f00ca5e0 0x7fa3f015d920 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009720 tx=0x7fa3e002a030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c47db9912267c8a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:43.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.516+0000 7fa3eca00640 1 -- 172.21.15.114:0/2933889388 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0037070 con 0x7fa3f00ca5e0 2024-05-27T09:35:43.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.516+0000 7fa3eca00640 1 -- 172.21.15.114:0/2933889388 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e002ad40 con 0x7fa3f00ca5e0 2024-05-27T09:35:43.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.516+0000 7fa3eca00640 1 -- 172.21.15.114:0/2933889388 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0032660 con 0x7fa3f00ca5e0 2024-05-27T09:35:43.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.517+0000 7fa3ef200640 1 -- 172.21.15.114:0/2933889388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3f00ca5e0 msgr2=0x7fa3f015d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:43.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.517+0000 7fa3ef200640 1 --2- 172.21.15.114:0/2933889388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3f00ca5e0 0x7fa3f015d920 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009720 tx=0x7fa3e002a030 comp rx=0 tx=0).stop 2024-05-27T09:35:43.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.517+0000 7fa3ef200640 1 -- 172.21.15.114:0/2933889388 shutdown_connections 2024-05-27T09:35:43.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.517+0000 7fa3ef200640 1 --2- 172.21.15.114:0/2933889388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3f00ca5e0 0x7fa3f015d920 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:43.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.517+0000 7fa3ef200640 1 -- 172.21.15.114:0/2933889388 >> 172.21.15.114:0/2933889388 conn(0x7fa3f006d7a0 msgr2=0x7fa3f006dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:43.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.517+0000 7fa3ef200640 1 -- 172.21.15.114:0/2933889388 shutdown_connections 2024-05-27T09:35:43.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.517+0000 7fa3ef200640 1 -- 172.21.15.114:0/2933889388 wait complete. 2024-05-27T09:35:43.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.520+0000 7fa3ef200640 1 Processor -- start 2024-05-27T09:35:43.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.520+0000 7fa3ef200640 1 -- start start 2024-05-27T09:35:43.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.521+0000 7fa3ef200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3f00724c0 0x7fa3f00c3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:43.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.521+0000 7fa3ef200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f00ca5e0 0x7fa3f01e4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:43.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.521+0000 7fa3ef200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3f015afd0 0x7fa3f00c43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:43.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.521+0000 7fa3ef200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f015a7d0 con 0x7fa3f00ca5e0 2024-05-27T09:35:43.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.521+0000 7fa3ef200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f015a4f0 con 0x7fa3f00724c0 2024-05-27T09:35:43.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.521+0000 7fa3ef200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa3f015aab0 con 0x7fa3f015afd0 2024-05-27T09:35:43.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.522+0000 7fa3ede00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3f00724c0 0x7fa3f00c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:43.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.522+0000 7fa3ee800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3f015afd0 0x7fa3f00c43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:43.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.522+0000 7fa3ede00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3f00724c0 0x7fa3f00c3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:35400/0 (socket says 172.21.15.114:35400) 2024-05-27T09:35:43.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.522+0000 7fa3ede00640 1 -- 172.21.15.114:0/605391883 learned_addr learned my addr 172.21.15.114:0/605391883 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:43.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.522+0000 7fa3ed400640 1 --2- 172.21.15.114:0/605391883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f00ca5e0 0x7fa3f01e4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:43.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.522+0000 7fa3ede00640 1 -- 172.21.15.114:0/605391883 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3f015afd0 msgr2=0x7fa3f00c43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:43.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.522+0000 7fa3ede00640 1 --2- 172.21.15.114:0/605391883 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3f015afd0 0x7fa3f00c43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:43.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.522+0000 7fa3ede00640 1 -- 172.21.15.114:0/605391883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f00ca5e0 msgr2=0x7fa3f01e4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:43.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.522+0000 7fa3ede00640 1 --2- 172.21.15.114:0/605391883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f00ca5e0 0x7fa3f01e4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:43.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.522+0000 7fa3ede00640 1 -- 172.21.15.114:0/605391883 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f01e4cb0 con 0x7fa3f00724c0 2024-05-27T09:35:43.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.522+0000 7fa3ee800640 1 --2- 172.21.15.114:0/605391883 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3f015afd0 0x7fa3f00c43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:35:43.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.522+0000 7fa3ede00640 1 --2- 172.21.15.114:0/605391883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3f00724c0 0x7fa3f00c3950 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0000c00 tx=0x7fa3e00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:43.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.523+0000 7fa3e6000640 1 -- 172.21.15.114:0/605391883 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0037040 con 0x7fa3f00724c0 2024-05-27T09:35:43.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.523+0000 7fa3e6000640 1 -- 172.21.15.114:0/605391883 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e0004030 con 0x7fa3f00724c0 2024-05-27T09:35:43.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.523+0000 7fa3ef200640 1 -- 172.21.15.114:0/605391883 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f01e6720 con 0x7fa3f00724c0 2024-05-27T09:35:43.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.523+0000 7fa3e6000640 1 -- 172.21.15.114:0/605391883 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0004420 con 0x7fa3f00724c0 2024-05-27T09:35:43.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.523+0000 7fa3ef200640 1 -- 172.21.15.114:0/605391883 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f01502b0 con 0x7fa3f00724c0 2024-05-27T09:35:43.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.524+0000 7fa3ef200640 1 -- 172.21.15.114:0/605391883 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f0050860 con 0x7fa3f00724c0 2024-05-27T09:35:43.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.525+0000 7fa3e6000640 1 -- 172.21.15.114:0/605391883 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa3e003a070 con 0x7fa3f00724c0 2024-05-27T09:35:43.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.527+0000 7fa3e6000640 1 --2- 172.21.15.114:0/605391883 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa3d0076300 0x7fa3d0078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:43.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.527+0000 7fa3e6000640 1 -- 172.21.15.114:0/605391883 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6168+0+0 (secure 0 0 0) 0x7fa3e00b7c70 con 0x7fa3f00724c0 2024-05-27T09:35:43.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.527+0000 7fa3ed400640 1 --2- 172.21.15.114:0/605391883 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa3d0076300 0x7fa3d0078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:43.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.528+0000 7fa3ed400640 1 --2- 172.21.15.114:0/605391883 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa3d0076300 0x7fa3d0078810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0157f50 tx=0x7fa3dc007c40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:43.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.531+0000 7fa3e6000640 1 -- 172.21.15.114:0/605391883 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa3e0041020 con 0x7fa3f00724c0 2024-05-27T09:35:43.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.953+0000 7fa3ef200640 1 -- 172.21.15.114:0/605391883 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3f00c8220 con 0x7fa3f00724c0 2024-05-27T09:35:43.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.954+0000 7fa3e6000640 1 -- 172.21.15.114:0/605391883 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+6056 (secure 0 0 0) 0x7fa3e0080da0 con 0x7fa3f00724c0 2024-05-27T09:35:43.956 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:35:43.956 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:35:40.437274+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[4875,5086]},"id":1}]} 2024-05-27T09:35:43.956 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 37 2024-05-27T09:35:43.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.956+0000 7fa3ef200640 1 -- 172.21.15.114:0/605391883 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa3d0076300 msgr2=0x7fa3d0078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:43.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.956+0000 7fa3ef200640 1 --2- 172.21.15.114:0/605391883 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa3d0076300 0x7fa3d0078810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0157f50 tx=0x7fa3dc007c40 comp rx=0 tx=0).stop 2024-05-27T09:35:43.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.956+0000 7fa3ef200640 1 -- 172.21.15.114:0/605391883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3f00724c0 msgr2=0x7fa3f00c3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:43.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.956+0000 7fa3ef200640 1 --2- 172.21.15.114:0/605391883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3f00724c0 0x7fa3f00c3950 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0000c00 tx=0x7fa3e00046c0 comp rx=0 tx=0).stop 2024-05-27T09:35:43.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.957+0000 7fa3ef200640 1 -- 172.21.15.114:0/605391883 shutdown_connections 2024-05-27T09:35:43.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.957+0000 7fa3ef200640 1 --2- 172.21.15.114:0/605391883 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa3d0076300 0x7fa3d0078810 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:43.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.957+0000 7fa3ef200640 1 --2- 172.21.15.114:0/605391883 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3f015afd0 0x7fa3f00c43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:43.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.957+0000 7fa3ef200640 1 --2- 172.21.15.114:0/605391883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f00ca5e0 0x7fa3f01e4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:43.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.957+0000 7fa3ef200640 1 --2- 172.21.15.114:0/605391883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3f00724c0 0x7fa3f00c3950 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:43.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.957+0000 7fa3ef200640 1 -- 172.21.15.114:0/605391883 >> 172.21.15.114:0/605391883 conn(0x7fa3f006d7a0 msgr2=0x7fa3f0075da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:43.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.957+0000 7fa3ef200640 1 -- 172.21.15.114:0/605391883 shutdown_connections 2024-05-27T09:35:43.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:43.957+0000 7fa3ef200640 1 -- 172.21.15.114:0/605391883 wait complete. 2024-05-27T09:35:43.984 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 37, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-05-27T09:27:18.079214+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 37, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4875": { "addr": "172.21.15.114:6835/2747052034", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6833", "nonce": 2747052034, "type": "v2" }, { "addr": "172.21.15.114:6835", "nonce": 2747052034, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4875, "incarnation": 25, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 25 }, "gid_5086": { "addr": "172.21.15.114:6834/1371544448", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6832", "nonce": 1371544448, "type": "v2" }, { "addr": "172.21.15.114:6834", "nonce": 1371544448, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5086, "incarnation": 31, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 27 }, "gid_5160": { "addr": "172.21.15.114:6837/1128209239", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6836", "nonce": 1128209239, "type": "v2" }, { "addr": "172.21.15.114:6837", "nonce": 1128209239, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5160, "incarnation": 37, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 49, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-05-27T09:35:40.437274+0000", "qdb_cluster": [ 4875, 5086 ], "qdb_leader": 5086, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5086, "mds_1": 4875, "mds_2": 5160 } } } ], "standbys": [ { "addr": "172.21.15.163:6835/3513522318", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 3513522318, "type": "v2" }, { "addr": "172.21.15.163:6835", "nonce": 3513522318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6838/1753596112", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6836", "nonce": 1753596112, "type": "v2" }, { "addr": "172.21.15.163:6838", "nonce": 1753596112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-05-27T09:35:45.986 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:35:46.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.219+0000 7fc587200640 1 Processor -- start 2024-05-27T09:35:46.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.219+0000 7fc587200640 1 -- start start 2024-05-27T09:35:46.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.222+0000 7fc587200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc5800c8780 0x7fc58015fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:46.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.222+0000 7fc587200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5800cd1e0 con 0x7fc5800b75b0 2024-05-27T09:35:46.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.222+0000 7fc587200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5800cd4c0 con 0x7fc5800b46e0 2024-05-27T09:35:46.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.222+0000 7fc587200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc5800c7f80 con 0x7fc5800c8780 2024-05-27T09:35:46.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.223+0000 7fc585e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc5800c8780 0x7fc58015fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:46.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.223+0000 7fc585e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc5800c8780 0x7fc58015fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46458/0 (socket says 172.21.15.114:46458) 2024-05-27T09:35:46.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.223+0000 7fc585e00640 1 -- 172.21.15.114:0/975774365 learned_addr learned my addr 172.21.15.114:0/975774365 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:46.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.223+0000 7fc585e00640 1 -- 172.21.15.114:0/975774365 >> v1:172.21.15.163:6789/0 conn(0x7fc5800b46e0 legacy=0x7fc580166fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:46.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.223+0000 7fc585e00640 1 -- 172.21.15.114:0/975774365 >> v1:172.21.15.114:6789/0 conn(0x7fc5800b75b0 legacy=0x7fc5800d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:46.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.223+0000 7fc585e00640 1 -- 172.21.15.114:0/975774365 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc580160160 con 0x7fc5800c8780 2024-05-27T09:35:46.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.223+0000 7fc584a00640 1 -- 172.21.15.114:0/975774365 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 653761743 0 0) 0x7fc5800cd4c0 con 0x7fc5800b46e0 2024-05-27T09:35:46.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.224+0000 7fc585e00640 1 --2- 172.21.15.114:0/975774365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5800c8780 0x7fc58015fc20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc574009870 tx=0x7fc57402a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75c14b3ff38d0e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:46.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.224+0000 7fc584a00640 1 -- 172.21.15.114:0/975774365 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574037070 con 0x7fc5800c8780 2024-05-27T09:35:46.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.225+0000 7fc584a00640 1 -- 172.21.15.114:0/975774365 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc574032640 con 0x7fc5800c8780 2024-05-27T09:35:46.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.225+0000 7fc584a00640 1 -- 172.21.15.114:0/975774365 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574032a10 con 0x7fc5800c8780 2024-05-27T09:35:46.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.225+0000 7fc587200640 1 -- 172.21.15.114:0/975774365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5800c8780 msgr2=0x7fc58015fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:46.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.225+0000 7fc587200640 1 --2- 172.21.15.114:0/975774365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5800c8780 0x7fc58015fc20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc574009870 tx=0x7fc57402a3c0 comp rx=0 tx=0).stop 2024-05-27T09:35:46.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.225+0000 7fc587200640 1 -- 172.21.15.114:0/975774365 shutdown_connections 2024-05-27T09:35:46.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.225+0000 7fc587200640 1 --2- 172.21.15.114:0/975774365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5800c8780 0x7fc58015fc20 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:46.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.225+0000 7fc587200640 1 -- 172.21.15.114:0/975774365 >> 172.21.15.114:0/975774365 conn(0x7fc58006e040 msgr2=0x7fc58006e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:46.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.226+0000 7fc587200640 1 -- 172.21.15.114:0/975774365 shutdown_connections 2024-05-27T09:35:46.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.226+0000 7fc587200640 1 -- 172.21.15.114:0/975774365 wait complete. 2024-05-27T09:35:46.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.228+0000 7fc587200640 1 Processor -- start 2024-05-27T09:35:46.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.228+0000 7fc587200640 1 -- start start 2024-05-27T09:35:46.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.229+0000 7fc587200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5800b46e0 0x7fc5800cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:46.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.229+0000 7fc587200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5800b75b0 0x7fc58014c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:46.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.229+0000 7fc587200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5800c8780 0x7fc58014c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:46.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.229+0000 7fc587200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc5800c6800 con 0x7fc5800b75b0 2024-05-27T09:35:46.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.229+0000 7fc587200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc5800cdf30 con 0x7fc5800c8780 2024-05-27T09:35:46.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.229+0000 7fc587200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fc5800cbaf0 con 0x7fc5800b46e0 2024-05-27T09:35:46.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.230+0000 7fc585e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5800b46e0 0x7fc5800cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:46.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.230+0000 7fc586800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5800c8780 0x7fc58014c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:46.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.230+0000 7fc585e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5800b46e0 0x7fc5800cb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:54434/0 (socket says 172.21.15.114:54434) 2024-05-27T09:35:46.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.230+0000 7fc585e00640 1 -- 172.21.15.114:0/105855923 learned_addr learned my addr 172.21.15.114:0/105855923 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:46.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.230+0000 7fc585400640 1 --2- 172.21.15.114:0/105855923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5800b75b0 0x7fc58014c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:46.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.230+0000 7fc586800640 1 -- 172.21.15.114:0/105855923 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5800b46e0 msgr2=0x7fc5800cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:46.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.230+0000 7fc586800640 1 --2- 172.21.15.114:0/105855923 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5800b46e0 0x7fc5800cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:46.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.230+0000 7fc586800640 1 -- 172.21.15.114:0/105855923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5800b75b0 msgr2=0x7fc58014c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:46.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.230+0000 7fc586800640 1 --2- 172.21.15.114:0/105855923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5800b75b0 0x7fc58014c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:46.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.230+0000 7fc586800640 1 -- 172.21.15.114:0/105855923 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5800cb620 con 0x7fc5800c8780 2024-05-27T09:35:46.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.230+0000 7fc585e00640 1 --2- 172.21.15.114:0/105855923 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5800b46e0 0x7fc5800cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:35:46.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.231+0000 7fc586800640 1 --2- 172.21.15.114:0/105855923 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5800c8780 0x7fc58014c780 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc5800cf7b0 tx=0x7fc57c00cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:46.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.231+0000 7fc57a000640 1 -- 172.21.15.114:0/105855923 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c003b70 con 0x7fc5800c8780 2024-05-27T09:35:46.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.231+0000 7fc57a000640 1 -- 172.21.15.114:0/105855923 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc57c003d30 con 0x7fc5800c8780 2024-05-27T09:35:46.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.231+0000 7fc587200640 1 -- 172.21.15.114:0/105855923 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5801f6660 con 0x7fc5800c8780 2024-05-27T09:35:46.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.231+0000 7fc57a000640 1 -- 172.21.15.114:0/105855923 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c0052b0 con 0x7fc5800c8780 2024-05-27T09:35:46.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.231+0000 7fc587200640 1 -- 172.21.15.114:0/105855923 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5800c8e70 con 0x7fc5800c8780 2024-05-27T09:35:46.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.233+0000 7fc587200640 1 -- 172.21.15.114:0/105855923 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc580050860 con 0x7fc5800c8780 2024-05-27T09:35:46.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.235+0000 7fc57a000640 1 -- 172.21.15.114:0/105855923 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc57c005450 con 0x7fc5800c8780 2024-05-27T09:35:46.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.236+0000 7fc57a000640 1 --2- 172.21.15.114:0/105855923 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc564076300 0x7fc564078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:46.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.236+0000 7fc57a000640 1 -- 172.21.15.114:0/105855923 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6168+0+0 (secure 0 0 0) 0x7fc57c097310 con 0x7fc5800c8780 2024-05-27T09:35:46.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.236+0000 7fc585e00640 1 --2- 172.21.15.114:0/105855923 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc564076300 0x7fc564078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:46.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.237+0000 7fc585e00640 1 --2- 172.21.15.114:0/105855923 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc564076300 0x7fc564078810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc574035040 tx=0x7fc57402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:46.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.241+0000 7fc57a000640 1 -- 172.21.15.114:0/105855923 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc57c0604e0 con 0x7fc5800c8780 2024-05-27T09:35:46.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.477+0000 7fc587200640 1 -- 172.21.15.114:0/105855923 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5800cf980 con 0x7fc5800c8780 2024-05-27T09:35:46.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.478+0000 7fc57a000640 1 -- 172.21.15.114:0/105855923 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+6058 (secure 0 0 0) 0x7fc57c0647e0 con 0x7fc5800c8780 2024-05-27T09:35:46.479 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:35:46.479 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:35:45.451870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:resolve","state_seq":20,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086,4875]},"id":1}]} 2024-05-27T09:35:46.479 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 38 2024-05-27T09:35:46.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.481+0000 7fc587200640 1 -- 172.21.15.114:0/105855923 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc564076300 msgr2=0x7fc564078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:46.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.481+0000 7fc587200640 1 --2- 172.21.15.114:0/105855923 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc564076300 0x7fc564078810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc574035040 tx=0x7fc57402e000 comp rx=0 tx=0).stop 2024-05-27T09:35:46.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.481+0000 7fc587200640 1 -- 172.21.15.114:0/105855923 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5800c8780 msgr2=0x7fc58014c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:46.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.481+0000 7fc587200640 1 --2- 172.21.15.114:0/105855923 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5800c8780 0x7fc58014c780 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc5800cf7b0 tx=0x7fc57c00cb90 comp rx=0 tx=0).stop 2024-05-27T09:35:46.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.481+0000 7fc587200640 1 -- 172.21.15.114:0/105855923 shutdown_connections 2024-05-27T09:35:46.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.481+0000 7fc587200640 1 --2- 172.21.15.114:0/105855923 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc564076300 0x7fc564078810 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:46.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.481+0000 7fc587200640 1 --2- 172.21.15.114:0/105855923 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5800c8780 0x7fc58014c780 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:46.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.481+0000 7fc587200640 1 --2- 172.21.15.114:0/105855923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5800b75b0 0x7fc58014c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:46.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.481+0000 7fc587200640 1 --2- 172.21.15.114:0/105855923 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5800b46e0 0x7fc5800cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:46.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.481+0000 7fc587200640 1 -- 172.21.15.114:0/105855923 >> 172.21.15.114:0/105855923 conn(0x7fc58006e040 msgr2=0x7fc5800d00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:46.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.481+0000 7fc587200640 1 -- 172.21.15.114:0/105855923 shutdown_connections 2024-05-27T09:35:46.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:46.481+0000 7fc587200640 1 -- 172.21.15.114:0/105855923 wait complete. 2024-05-27T09:35:46.523 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:35:48.524 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:35:48.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.758+0000 7fd49b200640 1 Processor -- start 2024-05-27T09:35:48.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.758+0000 7fd49b200640 1 -- start start 2024-05-27T09:35:48.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.761+0000 7fd49b200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd49c1534c0 0x7fd49c0c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:48.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.761+0000 7fd49b200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd49c0c8630 0x7fd49c074140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:48.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.761+0000 7fd49b200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd49c0c96d0 con 0x7fd49c0c8220 2024-05-27T09:35:48.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.761+0000 7fd49b200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fd49c0c2f40 con 0x7fd49c1534c0 2024-05-27T09:35:48.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.761+0000 7fd49b200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fd49c159190 con 0x7fd49c0c8630 2024-05-27T09:35:48.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.762+0000 7fd499e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd49c1534c0 0x7fd49c0c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:48.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.762+0000 7fd493e00640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fd49c0c8220 0x7fd49c159900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:46344/0 (socket says 172.21.15.114:46344) 2024-05-27T09:35:48.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.762+0000 7fd493e00640 1 -- 172.21.15.114:0/757524442 learned_addr learned my addr 172.21.15.114:0/757524442 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:35:48.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.762+0000 7fd49a800640 1 --2- 172.21.15.114:0/757524442 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd49c0c8630 0x7fd49c074140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:48.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.762+0000 7fd499400640 1 -- 172.21.15.114:0/757524442 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3078745359 0 0) 0x7fd49c0c96d0 con 0x7fd49c0c8220 2024-05-27T09:35:48.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.762+0000 7fd499400640 1 -- 172.21.15.114:0/757524442 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd480003520 con 0x7fd49c0c8220 2024-05-27T09:35:48.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.763+0000 7fd499e00640 1 -- 172.21.15.114:0/757524442 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd49c0c8630 msgr2=0x7fd49c074140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:35:48.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.763+0000 7fd499e00640 1 --2- 172.21.15.114:0/757524442 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd49c0c8630 0x7fd49c074140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:48.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.763+0000 7fd499e00640 1 -- 172.21.15.114:0/757524442 >> v1:172.21.15.163:6789/0 conn(0x7fd49c0c8220 legacy=0x7fd49c159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:48.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.763+0000 7fd499e00640 1 -- 172.21.15.114:0/757524442 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49c0c9970 con 0x7fd49c1534c0 2024-05-27T09:35:48.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.763+0000 7fd49a800640 1 --2- 172.21.15.114:0/757524442 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd49c0c8630 0x7fd49c074140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:35:48.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.764+0000 7fd499e00640 1 --2- 172.21.15.114:0/757524442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd49c1534c0 0x7fd49c0c2a00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0097a0 tx=0x7fd48c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d737ead15356824f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:48.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.764+0000 7fd499400640 1 -- 172.21.15.114:0/757524442 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c037070 con 0x7fd49c1534c0 2024-05-27T09:35:48.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.764+0000 7fd499400640 1 -- 172.21.15.114:0/757524442 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd48c032440 con 0x7fd49c1534c0 2024-05-27T09:35:48.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.764+0000 7fd499400640 1 -- 172.21.15.114:0/757524442 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c032810 con 0x7fd49c1534c0 2024-05-27T09:35:48.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.765+0000 7fd49b200640 1 -- 172.21.15.114:0/757524442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd49c1534c0 msgr2=0x7fd49c0c2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:48.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.765+0000 7fd49b200640 1 --2- 172.21.15.114:0/757524442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd49c1534c0 0x7fd49c0c2a00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0097a0 tx=0x7fd48c02a180 comp rx=0 tx=0).stop 2024-05-27T09:35:48.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.766+0000 7fd49b200640 1 -- 172.21.15.114:0/757524442 shutdown_connections 2024-05-27T09:35:48.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.766+0000 7fd49b200640 1 --2- 172.21.15.114:0/757524442 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd49c0c8630 0x7fd49c074140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:48.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.766+0000 7fd49b200640 1 --2- 172.21.15.114:0/757524442 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd49c1534c0 0x7fd49c0c2a00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:48.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.766+0000 7fd49b200640 1 -- 172.21.15.114:0/757524442 >> 172.21.15.114:0/757524442 conn(0x7fd49c0ca320 msgr2=0x7fd49c0ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:48.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.766+0000 7fd49b200640 1 -- 172.21.15.114:0/757524442 shutdown_connections 2024-05-27T09:35:48.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.766+0000 7fd49b200640 1 -- 172.21.15.114:0/757524442 wait complete. 2024-05-27T09:35:48.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.770+0000 7fd49b200640 1 Processor -- start 2024-05-27T09:35:48.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.770+0000 7fd49b200640 1 -- start start 2024-05-27T09:35:48.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.772+0000 7fd49b200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd49c0c8220 0x7fd49c0cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:48.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.772+0000 7fd49b200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd49c0c8630 0x7fd49c1f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:48.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.772+0000 7fd49b200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd49c1534c0 0x7fd49c150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:48.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.772+0000 7fd49b200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c0c9f30 con 0x7fd49c0c8630 2024-05-27T09:35:48.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.772+0000 7fd49b200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c1538d0 con 0x7fd49c1534c0 2024-05-27T09:35:48.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.772+0000 7fd49b200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fd49c1593e0 con 0x7fd49c0c8220 2024-05-27T09:35:48.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.773+0000 7fd493e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd49c0c8630 0x7fd49c1f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:48.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.773+0000 7fd493e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd49c0c8630 0x7fd49c1f1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46480/0 (socket says 172.21.15.114:46480) 2024-05-27T09:35:48.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.773+0000 7fd493e00640 1 -- 172.21.15.114:0/3246929874 learned_addr learned my addr 172.21.15.114:0/3246929874 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:48.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.773+0000 7fd499e00640 1 --2- 172.21.15.114:0/3246929874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd49c0c8220 0x7fd49c0cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:48.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.773+0000 7fd49a800640 1 --2- 172.21.15.114:0/3246929874 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd49c1534c0 0x7fd49c150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:48.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.773+0000 7fd493e00640 1 -- 172.21.15.114:0/3246929874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd49c0c8220 msgr2=0x7fd49c0cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:48.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.773+0000 7fd493e00640 1 --2- 172.21.15.114:0/3246929874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd49c0c8220 0x7fd49c0cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:48.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.773+0000 7fd493e00640 1 -- 172.21.15.114:0/3246929874 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd49c1534c0 msgr2=0x7fd49c150500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:48.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.773+0000 7fd493e00640 1 --2- 172.21.15.114:0/3246929874 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd49c1534c0 0x7fd49c150500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:48.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.773+0000 7fd493e00640 1 -- 172.21.15.114:0/3246929874 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49c1f24f0 con 0x7fd49c0c8630 2024-05-27T09:35:48.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.773+0000 7fd499e00640 1 --2- 172.21.15.114:0/3246929874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd49c0c8220 0x7fd49c0cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:35:48.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.773+0000 7fd49a800640 1 --2- 172.21.15.114:0/3246929874 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd49c1534c0 0x7fd49c150500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:35:48.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.774+0000 7fd493e00640 1 --2- 172.21.15.114:0/3246929874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd49c0c8630 0x7fd49c1f1fb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd48400c410 tx=0x7fd48400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:48.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.774+0000 7fd492000640 1 -- 172.21.15.114:0/3246929874 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484017070 con 0x7fd49c0c8630 2024-05-27T09:35:48.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.774+0000 7fd492000640 1 -- 172.21.15.114:0/3246929874 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd484005ce0 con 0x7fd49c0c8630 2024-05-27T09:35:48.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.774+0000 7fd49b200640 1 -- 172.21.15.114:0/3246929874 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd49c0c5c60 con 0x7fd49c0c8630 2024-05-27T09:35:48.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.774+0000 7fd492000640 1 -- 172.21.15.114:0/3246929874 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4840072d0 con 0x7fd49c0c8630 2024-05-27T09:35:48.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.774+0000 7fd49b200640 1 -- 172.21.15.114:0/3246929874 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd49c0c6220 con 0x7fd49c0c8630 2024-05-27T09:35:48.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.775+0000 7fd49b200640 1 -- 172.21.15.114:0/3246929874 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd49c0c4990 con 0x7fd49c0c8630 2024-05-27T09:35:48.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.777+0000 7fd492000640 1 -- 172.21.15.114:0/3246929874 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd48401a070 con 0x7fd49c0c8630 2024-05-27T09:35:48.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.778+0000 7fd492000640 1 --2- 172.21.15.114:0/3246929874 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd478076300 0x7fd478078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:48.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.779+0000 7fd492000640 1 -- 172.21.15.114:0/3246929874 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6168+0+0 (secure 0 0 0) 0x7fd484096a60 con 0x7fd49c0c8630 2024-05-27T09:35:48.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.779+0000 7fd499e00640 1 --2- 172.21.15.114:0/3246929874 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd478076300 0x7fd478078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:48.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.780+0000 7fd499e00640 1 --2- 172.21.15.114:0/3246929874 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd478076300 0x7fd478078810 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd48c034000 tx=0x7fd48c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:48.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:48.783+0000 7fd492000640 1 -- 172.21.15.114:0/3246929874 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd49c0c4990 con 0x7fd49c0c8630 2024-05-27T09:35:49.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:49.021+0000 7fd49b200640 1 -- 172.21.15.114:0/3246929874 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd49c050860 con 0x7fd49c0c8630 2024-05-27T09:35:49.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:49.022+0000 7fd492000640 1 -- 172.21.15.114:0/3246929874 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+6058 (secure 0 0 0) 0x7fd48405fba0 con 0x7fd49c0c8630 2024-05-27T09:35:49.024 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:35:49.024 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:35:45.451870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:resolve","state_seq":20,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086,4875]},"id":1}]} 2024-05-27T09:35:49.024 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 38 2024-05-27T09:35:49.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:49.025+0000 7fd49b200640 1 -- 172.21.15.114:0/3246929874 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd478076300 msgr2=0x7fd478078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:49.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:49.025+0000 7fd49b200640 1 --2- 172.21.15.114:0/3246929874 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd478076300 0x7fd478078810 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd48c034000 tx=0x7fd48c02e000 comp rx=0 tx=0).stop 2024-05-27T09:35:49.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:49.025+0000 7fd49b200640 1 -- 172.21.15.114:0/3246929874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd49c0c8630 msgr2=0x7fd49c1f1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:49.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:49.025+0000 7fd49b200640 1 --2- 172.21.15.114:0/3246929874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd49c0c8630 0x7fd49c1f1fb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd48400c410 tx=0x7fd48400c8d0 comp rx=0 tx=0).stop 2024-05-27T09:35:49.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:49.026+0000 7fd49b200640 1 -- 172.21.15.114:0/3246929874 shutdown_connections 2024-05-27T09:35:49.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:49.026+0000 7fd49b200640 1 --2- 172.21.15.114:0/3246929874 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd478076300 0x7fd478078810 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:49.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:49.026+0000 7fd49b200640 1 --2- 172.21.15.114:0/3246929874 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd49c1534c0 0x7fd49c150500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:49.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:49.026+0000 7fd49b200640 1 --2- 172.21.15.114:0/3246929874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd49c0c8630 0x7fd49c1f1fb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:49.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:49.026+0000 7fd49b200640 1 --2- 172.21.15.114:0/3246929874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd49c0c8220 0x7fd49c0cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:49.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:49.026+0000 7fd49b200640 1 -- 172.21.15.114:0/3246929874 >> 172.21.15.114:0/3246929874 conn(0x7fd49c0ca320 msgr2=0x7fd49c075da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:49.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:49.026+0000 7fd49b200640 1 -- 172.21.15.114:0/3246929874 shutdown_connections 2024-05-27T09:35:49.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:49.026+0000 7fd49b200640 1 -- 172.21.15.114:0/3246929874 wait complete. 2024-05-27T09:35:49.067 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:35:49.686 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:35:49.692 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:35:51.068 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:35:51.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.302+0000 7fa758c00640 1 Processor -- start 2024-05-27T09:35:51.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.302+0000 7fa758c00640 1 -- start start 2024-05-27T09:35:51.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.304+0000 7fa758c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7540741f0 0x7fa7540c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:51.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.304+0000 7fa758c00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa754074840 0x7fa75415ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:51.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.304+0000 7fa758c00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa75406fe50 con 0x7fa7541547d0 2024-05-27T09:35:51.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.304+0000 7fa758c00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa7540703d0 con 0x7fa754074840 2024-05-27T09:35:51.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.304+0000 7fa758c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa75415b260 con 0x7fa7540741f0 2024-05-27T09:35:51.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.304+0000 7fa752a00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fa7541547d0 0x7fa75406edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:45344/0 (socket says 172.21.15.114:45344) 2024-05-27T09:35:51.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.304+0000 7fa752a00640 1 -- 172.21.15.114:0/794868275 learned_addr learned my addr 172.21.15.114:0/794868275 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:35:51.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.304+0000 7fa753400640 1 --2- 172.21.15.114:0/794868275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7540741f0 0x7fa7540c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:51.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.305+0000 7fa753e00640 1 --2- 172.21.15.114:0/794868275 >> v2:172.21.15.114:3300/0 conn(0x7fa754074840 0x7fa75415ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:51.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.305+0000 7fa752000640 1 -- 172.21.15.114:0/794868275 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2829475467 0 0) 0x7fa75406fe50 con 0x7fa7541547d0 2024-05-27T09:35:51.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.305+0000 7fa752000640 1 -- 172.21.15.114:0/794868275 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa738003520 con 0x7fa7541547d0 2024-05-27T09:35:51.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.305+0000 7fa752000640 1 -- 172.21.15.114:0/794868275 <== mon.0 v1:172.21.15.114:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1572707929 0 0) 0x7fa738003520 con 0x7fa7541547d0 2024-05-27T09:35:51.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.305+0000 7fa752000640 1 -- 172.21.15.114:0/794868275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7540741f0 msgr2=0x7fa7540c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:35:51.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.306+0000 7fa752000640 1 --2- 172.21.15.114:0/794868275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7540741f0 0x7fa7540c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:51.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.306+0000 7fa752000640 1 -- 172.21.15.114:0/794868275 >> v2:172.21.15.114:3300/0 conn(0x7fa754074840 msgr2=0x7fa75415ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:35:51.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.306+0000 7fa752000640 1 --2- 172.21.15.114:0/794868275 >> v2:172.21.15.114:3300/0 conn(0x7fa754074840 0x7fa75415ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:51.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.306+0000 7fa752000640 1 -- 172.21.15.114:0/794868275 --> v1:172.21.15.114:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7540700f0 con 0x7fa7541547d0 2024-05-27T09:35:51.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.306+0000 7fa752000640 1 -- 172.21.15.114:0/794868275 <== mon.0 v1:172.21.15.114:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3867083960 0 0) 0x7fa740001710 con 0x7fa7541547d0 2024-05-27T09:35:51.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.306+0000 7fa752000640 1 -- 172.21.15.114:0/794868275 >> v1:172.21.15.114:6789/0 conn(0x7fa7541547d0 legacy=0x7fa75406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:51.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.306+0000 7fa752000640 1 --2- 172.21.15.114:0/794868275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa738003a50 0x7fa738003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:51.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.306+0000 7fa752000640 1 --2- 172.21.15.114:0/794868275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa738004800 0x7fa738004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:51.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.306+0000 7fa752000640 1 --2- 172.21.15.114:0/794868275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7540741f0 0x7fa738005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:51.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.306+0000 7fa752000640 1 -- 172.21.15.114:0/794868275 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa75415b260 con 0x7fa738003a50 2024-05-27T09:35:51.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.306+0000 7fa752000640 1 -- 172.21.15.114:0/794868275 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa73800bb00 con 0x7fa738004800 2024-05-27T09:35:51.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.306+0000 7fa752000640 1 -- 172.21.15.114:0/794868275 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa73800bcb0 con 0x7fa7540741f0 2024-05-27T09:35:51.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.306+0000 7fa753e00640 1 --2- 172.21.15.114:0/794868275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7540741f0 0x7fa738005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:51.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.306+0000 7fa752a00640 1 --2- 172.21.15.114:0/794868275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa738004800 0x7fa738004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:51.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.306+0000 7fa753400640 1 --2- 172.21.15.114:0/794868275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa738003a50 0x7fa738003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:51.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.307+0000 7fa753e00640 1 -- 172.21.15.114:0/794868275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa738004800 msgr2=0x7fa738004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:35:51.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.307+0000 7fa753e00640 1 --2- 172.21.15.114:0/794868275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa738004800 0x7fa738004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:51.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.307+0000 7fa753e00640 1 -- 172.21.15.114:0/794868275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa738003a50 msgr2=0x7fa738003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:35:51.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.307+0000 7fa753e00640 1 --2- 172.21.15.114:0/794868275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa738003a50 0x7fa738003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:51.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.307+0000 7fa753e00640 1 -- 172.21.15.114:0/794868275 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa738003520 con 0x7fa7540741f0 2024-05-27T09:35:51.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.307+0000 7fa753400640 1 --2- 172.21.15.114:0/794868275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa738003a50 0x7fa738003ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:35:51.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.308+0000 7fa753e00640 1 --2- 172.21.15.114:0/794868275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7540741f0 0x7fa738005150 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009f90 tx=0x7fa738005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=559dfd8a186a464f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:51.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.308+0000 7fa752000640 1 -- 172.21.15.114:0/794868275 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c009c10 con 0x7fa7540741f0 2024-05-27T09:35:51.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.308+0000 7fa752000640 1 -- 172.21.15.114:0/794868275 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa74c036070 con 0x7fa7540741f0 2024-05-27T09:35:51.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.308+0000 7fa752000640 1 -- 172.21.15.114:0/794868275 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c031880 con 0x7fa7540741f0 2024-05-27T09:35:51.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.309+0000 7fa758c00640 1 -- 172.21.15.114:0/794868275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7540741f0 msgr2=0x7fa738005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:51.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.309+0000 7fa758c00640 1 --2- 172.21.15.114:0/794868275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7540741f0 0x7fa738005150 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009f90 tx=0x7fa738005830 comp rx=0 tx=0).stop 2024-05-27T09:35:51.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.309+0000 7fa753e00640 1 -- 172.21.15.114:0/794868275 reap_dead start 2024-05-27T09:35:51.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.309+0000 7fa758c00640 1 -- 172.21.15.114:0/794868275 shutdown_connections 2024-05-27T09:35:51.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.309+0000 7fa758c00640 1 -- 172.21.15.114:0/794868275 >> 172.21.15.114:0/794868275 conn(0x7fa7540ca320 msgr2=0x7fa7540ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:51.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.309+0000 7fa758c00640 1 -- 172.21.15.114:0/794868275 shutdown_connections 2024-05-27T09:35:51.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.309+0000 7fa758c00640 1 -- 172.21.15.114:0/794868275 wait complete. 2024-05-27T09:35:51.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.312+0000 7fa758c00640 1 Processor -- start 2024-05-27T09:35:51.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.312+0000 7fa758c00640 1 -- start start 2024-05-27T09:35:51.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.314+0000 7fa758c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7540741f0 0x7fa754156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:51.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.314+0000 7fa758c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa754151980 0x7fa7540c4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:51.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.314+0000 7fa758c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa754153090 0x7fa75415dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:51.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.314+0000 7fa758c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa754073790 con 0x7fa754153090 2024-05-27T09:35:51.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.314+0000 7fa758c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa7540734b0 con 0x7fa754151980 2024-05-27T09:35:51.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.314+0000 7fa758c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa75406f500 con 0x7fa7540741f0 2024-05-27T09:35:51.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.314+0000 7fa753e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa754153090 0x7fa75415dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:51.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.314+0000 7fa753400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7540741f0 0x7fa754156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:51.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.314+0000 7fa753e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa754153090 0x7fa75415dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46504/0 (socket says 172.21.15.114:46504) 2024-05-27T09:35:51.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.314+0000 7fa753e00640 1 -- 172.21.15.114:0/2697150707 learned_addr learned my addr 172.21.15.114:0/2697150707 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:51.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.314+0000 7fa752a00640 1 --2- 172.21.15.114:0/2697150707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa754151980 0x7fa7540c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:51.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.315+0000 7fa753e00640 1 -- 172.21.15.114:0/2697150707 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7540741f0 msgr2=0x7fa754156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:51.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.315+0000 7fa753e00640 1 --2- 172.21.15.114:0/2697150707 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7540741f0 0x7fa754156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:51.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.315+0000 7fa753e00640 1 -- 172.21.15.114:0/2697150707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa754151980 msgr2=0x7fa7540c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:51.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.315+0000 7fa753e00640 1 --2- 172.21.15.114:0/2697150707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa754151980 0x7fa7540c4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:51.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.315+0000 7fa753e00640 1 -- 172.21.15.114:0/2697150707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa754157040 con 0x7fa754153090 2024-05-27T09:35:51.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.315+0000 7fa753e00640 1 --2- 172.21.15.114:0/2697150707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa754153090 0x7fa75415dc50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa74c033040 tx=0x7fa74c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:51.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.315+0000 7fa74b400640 1 -- 172.21.15.114:0/2697150707 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c009c10 con 0x7fa754153090 2024-05-27T09:35:51.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.315+0000 7fa74b400640 1 -- 172.21.15.114:0/2697150707 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa74c031db0 con 0x7fa754153090 2024-05-27T09:35:51.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.315+0000 7fa74b400640 1 -- 172.21.15.114:0/2697150707 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c003bd0 con 0x7fa754153090 2024-05-27T09:35:51.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.316+0000 7fa758c00640 1 -- 172.21.15.114:0/2697150707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa75415c4f0 con 0x7fa754153090 2024-05-27T09:35:51.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.316+0000 7fa758c00640 1 -- 172.21.15.114:0/2697150707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa75415cab0 con 0x7fa754153090 2024-05-27T09:35:51.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.317+0000 7fa758c00640 1 -- 172.21.15.114:0/2697150707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa75414b360 con 0x7fa754153090 2024-05-27T09:35:51.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.319+0000 7fa74b400640 1 -- 172.21.15.114:0/2697150707 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa74c005d10 con 0x7fa754153090 2024-05-27T09:35:51.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.320+0000 7fa74b400640 1 --2- 172.21.15.114:0/2697150707 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa734076300 0x7fa734078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:51.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.320+0000 7fa74b400640 1 -- 172.21.15.114:0/2697150707 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6168+0+0 (secure 0 0 0) 0x7fa74c0b8b70 con 0x7fa754153090 2024-05-27T09:35:51.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.320+0000 7fa753400640 1 --2- 172.21.15.114:0/2697150707 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa734076300 0x7fa734078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:51.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.321+0000 7fa753400640 1 --2- 172.21.15.114:0/2697150707 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa734076300 0x7fa734078810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa744006f80 tx=0x7fa744008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:51.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.329+0000 7fa74b400640 1 -- 172.21.15.114:0/2697150707 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa74c081c10 con 0x7fa754153090 2024-05-27T09:35:51.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.567+0000 7fa758c00640 1 -- 172.21.15.114:0/2697150707 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa754074620 con 0x7fa754153090 2024-05-27T09:35:51.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.568+0000 7fa74b400640 1 -- 172.21.15.114:0/2697150707 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+6060 (secure 0 0 0) 0x7fa74c08a250 con 0x7fa754153090 2024-05-27T09:35:51.570 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:35:51.571 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:35:50.516058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:reconnect","state_seq":22,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[4875,5086]},"id":1}]} 2024-05-27T09:35:51.571 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 39 2024-05-27T09:35:51.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.571+0000 7fa758c00640 1 -- 172.21.15.114:0/2697150707 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa734076300 msgr2=0x7fa734078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:51.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.571+0000 7fa758c00640 1 --2- 172.21.15.114:0/2697150707 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa734076300 0x7fa734078810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa744006f80 tx=0x7fa744008040 comp rx=0 tx=0).stop 2024-05-27T09:35:51.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.571+0000 7fa758c00640 1 -- 172.21.15.114:0/2697150707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa754153090 msgr2=0x7fa75415dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:51.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.571+0000 7fa758c00640 1 --2- 172.21.15.114:0/2697150707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa754153090 0x7fa75415dc50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa74c033040 tx=0x7fa74c009750 comp rx=0 tx=0).stop 2024-05-27T09:35:51.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.572+0000 7fa758c00640 1 -- 172.21.15.114:0/2697150707 shutdown_connections 2024-05-27T09:35:51.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.572+0000 7fa758c00640 1 --2- 172.21.15.114:0/2697150707 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa734076300 0x7fa734078810 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:51.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.572+0000 7fa758c00640 1 --2- 172.21.15.114:0/2697150707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa754153090 0x7fa75415dc50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:51.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.572+0000 7fa758c00640 1 --2- 172.21.15.114:0/2697150707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa754151980 0x7fa7540c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:51.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.572+0000 7fa758c00640 1 --2- 172.21.15.114:0/2697150707 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7540741f0 0x7fa754156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:51.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.572+0000 7fa758c00640 1 -- 172.21.15.114:0/2697150707 >> 172.21.15.114:0/2697150707 conn(0x7fa7540ca320 msgr2=0x7fa7540711b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:51.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.572+0000 7fa758c00640 1 -- 172.21.15.114:0/2697150707 shutdown_connections 2024-05-27T09:35:51.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:51.572+0000 7fa758c00640 1 -- 172.21.15.114:0/2697150707 wait complete. 2024-05-27T09:35:51.613 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:35:53.614 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:35:53.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.878+0000 7fdd66200640 1 Processor -- start 2024-05-27T09:35:53.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.878+0000 7fdd66200640 1 -- start start 2024-05-27T09:35:53.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.880+0000 7fdd66200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd600cc350 0x7fdd600cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:53.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.880+0000 7fdd66200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd6006e1c0 con 0x7fdd60156980 2024-05-27T09:35:53.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.880+0000 7fdd66200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd6006e4a0 con 0x7fdd600cb740 2024-05-27T09:35:53.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.880+0000 7fdd66200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fdd6006ed40 con 0x7fdd600cc350 2024-05-27T09:35:53.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.880+0000 7fdd64e00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fdd60156980 0x7fdd60160330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:45354/0 (socket says 172.21.15.114:45354) 2024-05-27T09:35:53.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.880+0000 7fdd64e00640 1 -- 172.21.15.114:0/3141717913 learned_addr learned my addr 172.21.15.114:0/3141717913 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:35:53.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.880+0000 7fdd5fe00640 1 --2- 172.21.15.114:0/3141717913 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd600cc350 0x7fdd600cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:53.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.881+0000 7fdd5fe00640 1 -- 172.21.15.114:0/3141717913 >> v1:172.21.15.163:6789/0 conn(0x7fdd600cb740 legacy=0x7fdd600cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:53.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.881+0000 7fdd5fe00640 1 -- 172.21.15.114:0/3141717913 >> v1:172.21.15.114:6789/0 conn(0x7fdd60156980 legacy=0x7fdd60160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:53.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.881+0000 7fdd5fe00640 1 -- 172.21.15.114:0/3141717913 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6006e780 con 0x7fdd600cc350 2024-05-27T09:35:53.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.881+0000 7fdd5f400640 1 -- 172.21.15.114:0/3141717913 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1377877553 0 0) 0x7fdd6006e1c0 con 0x7fdd60156980 2024-05-27T09:35:53.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.882+0000 7fdd5fe00640 1 --2- 172.21.15.114:0/3141717913 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd600cc350 0x7fdd600cc760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0097d0 tx=0x7fdd4c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c2f5ca7621be413 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:53.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.882+0000 7fdd5f400640 1 -- 172.21.15.114:0/3141717913 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c037070 con 0x7fdd600cc350 2024-05-27T09:35:53.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.882+0000 7fdd5f400640 1 -- 172.21.15.114:0/3141717913 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd4c032440 con 0x7fdd600cc350 2024-05-27T09:35:53.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.882+0000 7fdd5f400640 1 -- 172.21.15.114:0/3141717913 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c032810 con 0x7fdd600cc350 2024-05-27T09:35:53.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.883+0000 7fdd66200640 1 -- 172.21.15.114:0/3141717913 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd600cc350 msgr2=0x7fdd600cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:53.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.883+0000 7fdd66200640 1 --2- 172.21.15.114:0/3141717913 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd600cc350 0x7fdd600cc760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0097d0 tx=0x7fdd4c02a180 comp rx=0 tx=0).stop 2024-05-27T09:35:53.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.883+0000 7fdd66200640 1 -- 172.21.15.114:0/3141717913 shutdown_connections 2024-05-27T09:35:53.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.883+0000 7fdd66200640 1 --2- 172.21.15.114:0/3141717913 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd600cc350 0x7fdd600cc760 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:53.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.883+0000 7fdd66200640 1 -- 172.21.15.114:0/3141717913 >> 172.21.15.114:0/3141717913 conn(0x7fdd60071460 msgr2=0x7fdd60068600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:53.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.884+0000 7fdd66200640 1 -- 172.21.15.114:0/3141717913 shutdown_connections 2024-05-27T09:35:53.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.884+0000 7fdd66200640 1 -- 172.21.15.114:0/3141717913 wait complete. 2024-05-27T09:35:53.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.886+0000 7fdd66200640 1 Processor -- start 2024-05-27T09:35:53.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.886+0000 7fdd66200640 1 -- start start 2024-05-27T09:35:53.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.887+0000 7fdd66200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd600cb740 0x7fdd60159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:53.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.887+0000 7fdd66200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd600cc350 0x7fdd601581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:53.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.888+0000 7fdd66200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd60156980 0x7fdd6014a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:53.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.888+0000 7fdd66200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdd60156d90 con 0x7fdd600cc350 2024-05-27T09:35:53.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.888+0000 7fdd66200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6015fe10 con 0x7fdd60156980 2024-05-27T09:35:53.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.888+0000 7fdd66200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fdd60156f10 con 0x7fdd600cb740 2024-05-27T09:35:53.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.888+0000 7fdd5fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd600cc350 0x7fdd601581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:53.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.888+0000 7fdd5fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd600cc350 0x7fdd601581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46506/0 (socket says 172.21.15.114:46506) 2024-05-27T09:35:53.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.888+0000 7fdd5fe00640 1 -- 172.21.15.114:0/3932060832 learned_addr learned my addr 172.21.15.114:0/3932060832 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:53.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.888+0000 7fdd64e00640 1 --2- 172.21.15.114:0/3932060832 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd600cb740 0x7fdd60159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:53.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.888+0000 7fdd65800640 1 --2- 172.21.15.114:0/3932060832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd60156980 0x7fdd6014a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:53.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.888+0000 7fdd5fe00640 1 -- 172.21.15.114:0/3932060832 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd600cb740 msgr2=0x7fdd60159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:53.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.888+0000 7fdd5fe00640 1 --2- 172.21.15.114:0/3932060832 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd600cb740 0x7fdd60159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:53.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.888+0000 7fdd5fe00640 1 -- 172.21.15.114:0/3932060832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd60156980 msgr2=0x7fdd6014a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:53.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.888+0000 7fdd5fe00640 1 --2- 172.21.15.114:0/3932060832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd60156980 0x7fdd6014a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:53.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.888+0000 7fdd5fe00640 1 -- 172.21.15.114:0/3932060832 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd601f6e80 con 0x7fdd600cc350 2024-05-27T09:35:53.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.889+0000 7fdd5fe00640 1 --2- 172.21.15.114:0/3932060832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd600cc350 0x7fdd601581f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c035040 tx=0x7fdd4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:53.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.889+0000 7fdd5cc00640 1 -- 172.21.15.114:0/3932060832 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c037040 con 0x7fdd600cc350 2024-05-27T09:35:53.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.889+0000 7fdd5cc00640 1 -- 172.21.15.114:0/3932060832 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd4c004250 con 0x7fdd600cc350 2024-05-27T09:35:53.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.889+0000 7fdd66200640 1 -- 172.21.15.114:0/3932060832 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd60157260 con 0x7fdd600cc350 2024-05-27T09:35:53.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.889+0000 7fdd5cc00640 1 -- 172.21.15.114:0/3932060832 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c032e40 con 0x7fdd600cc350 2024-05-27T09:35:53.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.889+0000 7fdd66200640 1 -- 172.21.15.114:0/3932060832 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd60159610 con 0x7fdd600cc350 2024-05-27T09:35:53.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.891+0000 7fdd66200640 1 -- 172.21.15.114:0/3932060832 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd6006d5d0 con 0x7fdd600cc350 2024-05-27T09:35:53.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.894+0000 7fdd5cc00640 1 -- 172.21.15.114:0/3932060832 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdd4c0306f0 con 0x7fdd600cc350 2024-05-27T09:35:53.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.895+0000 7fdd5cc00640 1 --2- 172.21.15.114:0/3932060832 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdd400762d0 0x7fdd400787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:53.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.895+0000 7fdd5cc00640 1 -- 172.21.15.114:0/3932060832 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6168+0+0 (secure 0 0 0) 0x7fdd4c0b8e60 con 0x7fdd600cc350 2024-05-27T09:35:53.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.895+0000 7fdd64e00640 1 --2- 172.21.15.114:0/3932060832 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdd400762d0 0x7fdd400787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:53.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.896+0000 7fdd64e00640 1 --2- 172.21.15.114:0/3932060832 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdd400762d0 0x7fdd400787e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdd54000f60 tx=0x7fdd54007e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:53.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:53.902+0000 7fdd5cc00640 1 -- 172.21.15.114:0/3932060832 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdd4c04c070 con 0x7fdd600cc350 2024-05-27T09:35:54.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:54.330+0000 7fdd66200640 1 -- 172.21.15.114:0/3932060832 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd6006a380 con 0x7fdd600cc350 2024-05-27T09:35:54.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:54.331+0000 7fdd5cc00640 1 -- 172.21.15.114:0/3932060832 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+6060 (secure 0 0 0) 0x7fdd6006a380 con 0x7fdd600cc350 2024-05-27T09:35:54.332 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:35:54.332 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:35:50.516058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:reconnect","state_seq":22,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[4875,5086]},"id":1}]} 2024-05-27T09:35:54.333 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 39 2024-05-27T09:35:54.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:54.334+0000 7fdd66200640 1 -- 172.21.15.114:0/3932060832 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdd400762d0 msgr2=0x7fdd400787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:54.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:54.334+0000 7fdd66200640 1 --2- 172.21.15.114:0/3932060832 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdd400762d0 0x7fdd400787e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdd54000f60 tx=0x7fdd54007e40 comp rx=0 tx=0).stop 2024-05-27T09:35:54.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:54.334+0000 7fdd66200640 1 -- 172.21.15.114:0/3932060832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd600cc350 msgr2=0x7fdd601581f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:54.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:54.334+0000 7fdd66200640 1 --2- 172.21.15.114:0/3932060832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd600cc350 0x7fdd601581f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c035040 tx=0x7fdd4c0046c0 comp rx=0 tx=0).stop 2024-05-27T09:35:54.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:54.334+0000 7fdd66200640 1 -- 172.21.15.114:0/3932060832 shutdown_connections 2024-05-27T09:35:54.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:54.334+0000 7fdd66200640 1 --2- 172.21.15.114:0/3932060832 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdd400762d0 0x7fdd400787e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:54.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:54.334+0000 7fdd66200640 1 --2- 172.21.15.114:0/3932060832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd60156980 0x7fdd6014a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:54.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:54.334+0000 7fdd66200640 1 --2- 172.21.15.114:0/3932060832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd600cc350 0x7fdd601581f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:54.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:54.334+0000 7fdd66200640 1 --2- 172.21.15.114:0/3932060832 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd600cb740 0x7fdd60159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:54.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:54.334+0000 7fdd66200640 1 -- 172.21.15.114:0/3932060832 >> 172.21.15.114:0/3932060832 conn(0x7fdd60071460 msgr2=0x7fdd60075da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:54.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:54.334+0000 7fdd66200640 1 -- 172.21.15.114:0/3932060832 shutdown_connections 2024-05-27T09:35:54.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:54.334+0000 7fdd66200640 1 -- 172.21.15.114:0/3932060832 wait complete. 2024-05-27T09:35:54.363 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:35:56.365 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:35:56.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.623+0000 7f2f5b200640 1 Processor -- start 2024-05-27T09:35:56.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.623+0000 7f2f5b200640 1 -- start start 2024-05-27T09:35:56.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.625+0000 7f2f5b200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2f541593b0 0x7f2f54142720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:56.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.625+0000 7f2f5b200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f54073260 0x7f2f5415a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:56.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.625+0000 7f2f5b200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2f5415ade0 0x7f2f540c2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:56.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.625+0000 7f2f5b200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f2f54074d20 con 0x7f2f541593b0 2024-05-27T09:35:56.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.625+0000 7f2f5b200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f2f540739a0 con 0x7f2f5415ade0 2024-05-27T09:35:56.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.625+0000 7f2f5b200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f2f54073c80 con 0x7f2f54073260 2024-05-27T09:35:56.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.625+0000 7f2f53e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2f541593b0 0x7f2f54142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:56.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.625+0000 7f2f53e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f2f541593b0 0x7f2f54142720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48616/0 (socket says 172.21.15.114:48616) 2024-05-27T09:35:56.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.625+0000 7f2f53e00640 1 -- 172.21.15.114:0/2149288554 learned_addr learned my addr 172.21.15.114:0/2149288554 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:56.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.625+0000 7f2f4be00640 1 --2- 172.21.15.114:0/2149288554 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f54073260 0x7f2f5415a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:56.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.625+0000 7f2f59200640 1 --2- 172.21.15.114:0/2149288554 >> v2:172.21.15.163:3300/0 conn(0x7f2f5415ade0 0x7f2f540c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:56.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.626+0000 7f2f59200640 1 -- 172.21.15.114:0/2149288554 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f54073260 msgr2=0x7f2f5415a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:35:56.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.626+0000 7f2f59200640 1 --2- 172.21.15.114:0/2149288554 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f54073260 0x7f2f5415a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:56.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.626+0000 7f2f59200640 1 -- 172.21.15.114:0/2149288554 >> v2:172.21.15.114:3300/0 conn(0x7f2f541593b0 msgr2=0x7f2f54142720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:35:56.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.626+0000 7f2f59200640 1 --2- 172.21.15.114:0/2149288554 >> v2:172.21.15.114:3300/0 conn(0x7f2f541593b0 0x7f2f54142720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:56.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.626+0000 7f2f59200640 1 -- 172.21.15.114:0/2149288554 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f54073f60 con 0x7f2f5415ade0 2024-05-27T09:35:56.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.627+0000 7f2f59200640 1 --2- 172.21.15.114:0/2149288554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f5415ade0 0x7f2f540c2990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2f540c48f0 tx=0x7f2f4c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6721f1892e6a9ea2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:56.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.627+0000 7f2f53400640 1 -- 172.21.15.114:0/2149288554 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c007740 con 0x7f2f5415ade0 2024-05-27T09:35:56.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.627+0000 7f2f53400640 1 -- 172.21.15.114:0/2149288554 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4c0078e0 con 0x7f2f5415ade0 2024-05-27T09:35:56.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.627+0000 7f2f53400640 1 -- 172.21.15.114:0/2149288554 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c007cb0 con 0x7f2f5415ade0 2024-05-27T09:35:56.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.628+0000 7f2f5b200640 1 -- 172.21.15.114:0/2149288554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f5415ade0 msgr2=0x7f2f540c2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:56.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.628+0000 7f2f5b200640 1 --2- 172.21.15.114:0/2149288554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f5415ade0 0x7f2f540c2990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2f540c48f0 tx=0x7f2f4c02a660 comp rx=0 tx=0).stop 2024-05-27T09:35:56.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.628+0000 7f2f5b200640 1 -- 172.21.15.114:0/2149288554 shutdown_connections 2024-05-27T09:35:56.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.628+0000 7f2f5b200640 1 --2- 172.21.15.114:0/2149288554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f5415ade0 0x7f2f540c2990 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:56.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.628+0000 7f2f5b200640 1 --2- 172.21.15.114:0/2149288554 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f54073260 0x7f2f5415a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:56.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.628+0000 7f2f5b200640 1 --2- 172.21.15.114:0/2149288554 >> v2:172.21.15.114:3300/0 conn(0x7f2f541593b0 0x7f2f54142720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:56.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.628+0000 7f2f5b200640 1 -- 172.21.15.114:0/2149288554 >> 172.21.15.114:0/2149288554 conn(0x7f2f5406f6e0 msgr2=0x7f2f54070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:56.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.628+0000 7f2f5b200640 1 -- 172.21.15.114:0/2149288554 shutdown_connections 2024-05-27T09:35:56.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.629+0000 7f2f5b200640 1 -- 172.21.15.114:0/2149288554 wait complete. 2024-05-27T09:35:56.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.631+0000 7f2f5b200640 1 Processor -- start 2024-05-27T09:35:56.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.631+0000 7f2f5b200640 1 -- start start 2024-05-27T09:35:56.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.633+0000 7f2f5b200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f54073260 0x7f2f541f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:56.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.633+0000 7f2f5b200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f541593b0 0x7f2f54155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:56.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.633+0000 7f2f5b200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f5415ade0 0x7f2f541410d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:56.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.633+0000 7f2f5b200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2f54074800 con 0x7f2f5415ade0 2024-05-27T09:35:56.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.633+0000 7f2f5b200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2f54074520 con 0x7f2f541593b0 2024-05-27T09:35:56.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.633+0000 7f2f5b200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f2f54073670 con 0x7f2f54073260 2024-05-27T09:35:56.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.633+0000 7f2f59200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f5415ade0 0x7f2f541410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:56.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.633+0000 7f2f59200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f5415ade0 0x7f2f541410d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48626/0 (socket says 172.21.15.114:48626) 2024-05-27T09:35:56.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.633+0000 7f2f59200640 1 -- 172.21.15.114:0/1721498352 learned_addr learned my addr 172.21.15.114:0/1721498352 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:56.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.633+0000 7f2f53e00640 1 --2- 172.21.15.114:0/1721498352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f54073260 0x7f2f541f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:56.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.633+0000 7f2f4be00640 1 --2- 172.21.15.114:0/1721498352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f541593b0 0x7f2f54155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:56.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.634+0000 7f2f59200640 1 -- 172.21.15.114:0/1721498352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f54073260 msgr2=0x7f2f541f2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:56.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.634+0000 7f2f59200640 1 --2- 172.21.15.114:0/1721498352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f54073260 0x7f2f541f2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:56.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.634+0000 7f2f59200640 1 -- 172.21.15.114:0/1721498352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f541593b0 msgr2=0x7f2f54155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:56.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.634+0000 7f2f59200640 1 --2- 172.21.15.114:0/1721498352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f541593b0 0x7f2f54155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:56.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.634+0000 7f2f59200640 1 -- 172.21.15.114:0/1721498352 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f54155b60 con 0x7f2f5415ade0 2024-05-27T09:35:56.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.634+0000 7f2f53e00640 1 --2- 172.21.15.114:0/1721498352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f54073260 0x7f2f541f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:35:56.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.634+0000 7f2f4be00640 1 --2- 172.21.15.114:0/1721498352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f541593b0 0x7f2f54155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:35:56.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.634+0000 7f2f59200640 1 --2- 172.21.15.114:0/1721498352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f5415ade0 0x7f2f541410d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009e50 tx=0x7f2f4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:56.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.634+0000 7f2f50c00640 1 -- 172.21.15.114:0/1721498352 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c03f070 con 0x7f2f5415ade0 2024-05-27T09:35:56.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.635+0000 7f2f50c00640 1 -- 172.21.15.114:0/1721498352 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4c031bf0 con 0x7f2f5415ade0 2024-05-27T09:35:56.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.635+0000 7f2f50c00640 1 -- 172.21.15.114:0/1721498352 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c009450 con 0x7f2f5415ade0 2024-05-27T09:35:56.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.635+0000 7f2f5b200640 1 -- 172.21.15.114:0/1721498352 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f541f3c20 con 0x7f2f5415ade0 2024-05-27T09:35:56.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.635+0000 7f2f5b200640 1 -- 172.21.15.114:0/1721498352 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f541f3940 con 0x7f2f5415ade0 2024-05-27T09:35:56.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.636+0000 7f2f5b200640 1 -- 172.21.15.114:0/1721498352 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f5406a380 con 0x7f2f5415ade0 2024-05-27T09:35:56.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.637+0000 7f2f50c00640 1 -- 172.21.15.114:0/1721498352 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2f4c03b410 con 0x7f2f5415ade0 2024-05-27T09:35:56.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.638+0000 7f2f50c00640 1 --2- 172.21.15.114:0/1721498352 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2f300762d0 0x7f2f300787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:56.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.638+0000 7f2f50c00640 1 -- 172.21.15.114:0/1721498352 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6168+0+0 (secure 0 0 0) 0x7f2f4c0b8e00 con 0x7f2f5415ade0 2024-05-27T09:35:56.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.639+0000 7f2f53e00640 1 --2- 172.21.15.114:0/1721498352 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2f300762d0 0x7f2f300787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:56.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.639+0000 7f2f53e00640 1 --2- 172.21.15.114:0/1721498352 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2f300762d0 0x7f2f300787e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009ea0 tx=0x7f2f44009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:56.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.647+0000 7f2f50c00640 1 -- 172.21.15.114:0/1721498352 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2f4c081f30 con 0x7f2f5415ade0 2024-05-27T09:35:56.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.912+0000 7f2f5b200640 1 -- 172.21.15.114:0/1721498352 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f540c41f0 con 0x7f2f5415ade0 2024-05-27T09:35:56.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.913+0000 7f2f50c00640 1 -- 172.21.15.114:0/1721498352 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+6057 (secure 0 0 0) 0x7f2f4c086230 con 0x7f2f5415ade0 2024-05-27T09:35:56.914 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:35:56.914 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:35:55.973421+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:rejoin","state_seq":24,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086,4875]},"id":1}]} 2024-05-27T09:35:56.914 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 40 2024-05-27T09:35:56.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.915+0000 7f2f5b200640 1 -- 172.21.15.114:0/1721498352 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2f300762d0 msgr2=0x7f2f300787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:56.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.915+0000 7f2f5b200640 1 --2- 172.21.15.114:0/1721498352 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2f300762d0 0x7f2f300787e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009ea0 tx=0x7f2f44009240 comp rx=0 tx=0).stop 2024-05-27T09:35:56.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.915+0000 7f2f5b200640 1 -- 172.21.15.114:0/1721498352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f5415ade0 msgr2=0x7f2f541410d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:56.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.915+0000 7f2f5b200640 1 --2- 172.21.15.114:0/1721498352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f5415ade0 0x7f2f541410d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009e50 tx=0x7f2f4c004a40 comp rx=0 tx=0).stop 2024-05-27T09:35:56.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.915+0000 7f2f5b200640 1 -- 172.21.15.114:0/1721498352 shutdown_connections 2024-05-27T09:35:56.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.915+0000 7f2f5b200640 1 --2- 172.21.15.114:0/1721498352 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2f300762d0 0x7f2f300787e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:56.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.915+0000 7f2f5b200640 1 --2- 172.21.15.114:0/1721498352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f5415ade0 0x7f2f541410d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:56.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.915+0000 7f2f5b200640 1 --2- 172.21.15.114:0/1721498352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f541593b0 0x7f2f54155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:56.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.915+0000 7f2f5b200640 1 --2- 172.21.15.114:0/1721498352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f54073260 0x7f2f541f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:56.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.915+0000 7f2f5b200640 1 -- 172.21.15.114:0/1721498352 >> 172.21.15.114:0/1721498352 conn(0x7f2f5406f6e0 msgr2=0x7f2f540b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:56.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.916+0000 7f2f5b200640 1 -- 172.21.15.114:0/1721498352 shutdown_connections 2024-05-27T09:35:56.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:56.916+0000 7f2f5b200640 1 -- 172.21.15.114:0/1721498352 wait complete. 2024-05-27T09:35:56.941 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 40, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-05-27T09:27:18.079214+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 40, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4875": { "addr": "172.21.15.114:6835/2747052034", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6833", "nonce": 2747052034, "type": "v2" }, { "addr": "172.21.15.114:6835", "nonce": 2747052034, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4875, "incarnation": 25, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 25 }, "gid_5086": { "addr": "172.21.15.114:6834/1371544448", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6832", "nonce": 1371544448, "type": "v2" }, { "addr": "172.21.15.114:6834", "nonce": 1371544448, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5086, "incarnation": 31, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 27 }, "gid_5160": { "addr": "172.21.15.114:6837/1128209239", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6836", "nonce": 1128209239, "type": "v2" }, { "addr": "172.21.15.114:6837", "nonce": 1128209239, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5160, "incarnation": 37, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:rejoin", "state_seq": 24 } }, "last_failure": 0, "last_failure_osd_epoch": 49, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-05-27T09:35:55.973421+0000", "qdb_cluster": [ 5086, 4875 ], "qdb_leader": 5086, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5086, "mds_1": 4875, "mds_2": 5160 } } } ], "standbys": [ { "addr": "172.21.15.163:6835/3513522318", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 3513522318, "type": "v2" }, { "addr": "172.21.15.163:6835", "nonce": 3513522318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6838/1753596112", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6836", "nonce": 1753596112, "type": "v2" }, { "addr": "172.21.15.163:6838", "nonce": 1753596112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-05-27T09:35:58.943 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:35:59.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.223+0000 7f37e6400640 1 Processor -- start 2024-05-27T09:35:59.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.223+0000 7f37e6400640 1 -- start start 2024-05-27T09:35:59.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.225+0000 7f37e6400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37e0074a60 0x7f37e0154820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:59.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.225+0000 7f37e6400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f37e00ca9e0 0x7f37e0073670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:59.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.225+0000 7f37e6400640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37e015a810 con 0x7f37e0073260 2024-05-27T09:35:59.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.225+0000 7f37e6400640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f37e015db60 con 0x7f37e00ca9e0 2024-05-27T09:35:59.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.226+0000 7f37e6400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f37e015aaf0 con 0x7f37e0074a60 2024-05-27T09:35:59.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.226+0000 7f37d7e00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f37e0073260 0x7f37e00ca2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57582/0 (socket says 172.21.15.114:57582) 2024-05-27T09:35:59.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.226+0000 7f37dfe00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f37e00ca9e0 0x7f37e0073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:59.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.226+0000 7f37d7e00640 1 -- 172.21.15.114:0/74454696 learned_addr learned my addr 172.21.15.114:0/74454696 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:35:59.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.226+0000 7f37df400640 1 --2- 172.21.15.114:0/74454696 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37e0074a60 0x7f37e0154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:59.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.226+0000 7f37dfe00640 1 -- 172.21.15.114:0/74454696 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37e0074a60 msgr2=0x7f37e0154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:35:59.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.226+0000 7f37dfe00640 1 --2- 172.21.15.114:0/74454696 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37e0074a60 0x7f37e0154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:59.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.226+0000 7f37dfe00640 1 -- 172.21.15.114:0/74454696 >> v1:172.21.15.114:6789/0 conn(0x7f37e0073260 legacy=0x7f37e00ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:59.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.227+0000 7f37dfe00640 1 -- 172.21.15.114:0/74454696 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e015add0 con 0x7f37e00ca9e0 2024-05-27T09:35:59.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.226+0000 7f37dea00640 1 -- 172.21.15.114:0/74454696 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3557860423 0 0) 0x7f37e015a810 con 0x7f37e0073260 2024-05-27T09:35:59.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.228+0000 7f37dfe00640 1 --2- 172.21.15.114:0/74454696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37e00ca9e0 0x7f37e0073670 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009d90 tx=0x7f37d802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805aaf0a01cb0618 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:59.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.228+0000 7f37dea00640 1 -- 172.21.15.114:0/74454696 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8007740 con 0x7f37e00ca9e0 2024-05-27T09:35:59.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.228+0000 7f37dea00640 1 -- 172.21.15.114:0/74454696 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d80078e0 con 0x7f37e00ca9e0 2024-05-27T09:35:59.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.228+0000 7f37dea00640 1 -- 172.21.15.114:0/74454696 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8007cb0 con 0x7f37e00ca9e0 2024-05-27T09:35:59.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.228+0000 7f37e6400640 1 -- 172.21.15.114:0/74454696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37e00ca9e0 msgr2=0x7f37e0073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:59.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.228+0000 7f37e6400640 1 --2- 172.21.15.114:0/74454696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37e00ca9e0 0x7f37e0073670 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009d90 tx=0x7f37d802a6f0 comp rx=0 tx=0).stop 2024-05-27T09:35:59.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.229+0000 7f37e6400640 1 -- 172.21.15.114:0/74454696 shutdown_connections 2024-05-27T09:35:59.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.229+0000 7f37e6400640 1 --2- 172.21.15.114:0/74454696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37e00ca9e0 0x7f37e0073670 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:59.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.229+0000 7f37e6400640 1 --2- 172.21.15.114:0/74454696 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37e0074a60 0x7f37e0154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:59.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.229+0000 7f37e6400640 1 -- 172.21.15.114:0/74454696 >> 172.21.15.114:0/74454696 conn(0x7f37e006f6e0 msgr2=0x7f37e0070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:59.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.229+0000 7f37e6400640 1 -- 172.21.15.114:0/74454696 shutdown_connections 2024-05-27T09:35:59.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.229+0000 7f37e6400640 1 -- 172.21.15.114:0/74454696 wait complete. 2024-05-27T09:35:59.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.231+0000 7f37e6400640 1 Processor -- start 2024-05-27T09:35:59.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.231+0000 7f37e6400640 1 -- start start 2024-05-27T09:35:59.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.233+0000 7f37e6400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37e0073260 0x7f37e01f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:59.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.233+0000 7f37e6400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37e0074a60 0x7f37e00c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:59.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.233+0000 7f37e6400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37e00ca9e0 0x7f37e01520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:59.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.233+0000 7f37e6400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f37e015a4e0 con 0x7f37e0074a60 2024-05-27T09:35:59.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.233+0000 7f37e6400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f37e015b390 con 0x7f37e00ca9e0 2024-05-27T09:35:59.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.233+0000 7f37e6400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f37e015dff0 con 0x7f37e0073260 2024-05-27T09:35:59.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.233+0000 7f37d7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37e0074a60 0x7f37e00c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:59.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.233+0000 7f37d7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37e0074a60 0x7f37e00c3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48646/0 (socket says 172.21.15.114:48646) 2024-05-27T09:35:59.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.233+0000 7f37d7e00640 1 -- 172.21.15.114:0/2252286454 learned_addr learned my addr 172.21.15.114:0/2252286454 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:35:59.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.233+0000 7f37df400640 1 --2- 172.21.15.114:0/2252286454 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37e0073260 0x7f37e01f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:59.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.233+0000 7f37dfe00640 1 --2- 172.21.15.114:0/2252286454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37e00ca9e0 0x7f37e01520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:59.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.233+0000 7f37d7e00640 1 -- 172.21.15.114:0/2252286454 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37e0073260 msgr2=0x7f37e01f65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:59.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.233+0000 7f37d7e00640 1 --2- 172.21.15.114:0/2252286454 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37e0073260 0x7f37e01f65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:59.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.233+0000 7f37d7e00640 1 -- 172.21.15.114:0/2252286454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37e00ca9e0 msgr2=0x7f37e01520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:59.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.233+0000 7f37d7e00640 1 --2- 172.21.15.114:0/2252286454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37e00ca9e0 0x7f37e01520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:59.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.233+0000 7f37d7e00640 1 -- 172.21.15.114:0/2252286454 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e01f6b10 con 0x7f37e0074a60 2024-05-27T09:35:59.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.234+0000 7f37d7e00640 1 --2- 172.21.15.114:0/2252286454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37e0074a60 0x7f37e00c3bc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f37c800c430 tx=0x7f37c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:59.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.234+0000 7f37d7400640 1 -- 172.21.15.114:0/2252286454 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8017070 con 0x7f37e0074a60 2024-05-27T09:35:59.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.234+0000 7f37d7400640 1 -- 172.21.15.114:0/2252286454 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37c8005ce0 con 0x7f37e0074a60 2024-05-27T09:35:59.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.234+0000 7f37d7400640 1 -- 172.21.15.114:0/2252286454 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8007310 con 0x7f37e0074a60 2024-05-27T09:35:59.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.234+0000 7f37e6400640 1 -- 172.21.15.114:0/2252286454 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e0153810 con 0x7f37e0074a60 2024-05-27T09:35:59.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.234+0000 7f37e6400640 1 -- 172.21.15.114:0/2252286454 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e0153dd0 con 0x7f37e0074a60 2024-05-27T09:35:59.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.236+0000 7f37e6400640 1 -- 172.21.15.114:0/2252286454 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e006a380 con 0x7f37e0074a60 2024-05-27T09:35:59.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.237+0000 7f37d7400640 1 -- 172.21.15.114:0/2252286454 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f37c801a070 con 0x7f37e0074a60 2024-05-27T09:35:59.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.238+0000 7f37d7400640 1 --2- 172.21.15.114:0/2252286454 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f37bc076300 0x7f37bc078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:59.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.238+0000 7f37d7400640 1 -- 172.21.15.114:0/2252286454 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6168+0+0 (secure 0 0 0) 0x7f37c80969d0 con 0x7f37e0074a60 2024-05-27T09:35:59.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.238+0000 7f37df400640 1 --2- 172.21.15.114:0/2252286454 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f37bc076300 0x7f37bc078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:59.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.239+0000 7f37df400640 1 --2- 172.21.15.114:0/2252286454 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f37bc076300 0x7f37bc078810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009aa0 tx=0x7f37d0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:59.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.244+0000 7f37d7400640 1 -- 172.21.15.114:0/2252286454 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f37c805fba0 con 0x7f37e0074a60 2024-05-27T09:35:59.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.491+0000 7f37e6400640 1 -- 172.21.15.114:0/2252286454 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37e00c7c50 con 0x7f37e0074a60 2024-05-27T09:35:59.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.492+0000 7f37d7400640 1 -- 172.21.15.114:0/2252286454 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+6057 (secure 0 0 0) 0x7f37c8063ea0 con 0x7f37e0074a60 2024-05-27T09:35:59.494 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:35:59.494 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:35:55.973421+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:rejoin","state_seq":24,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086,4875]},"id":1}]} 2024-05-27T09:35:59.494 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 40 2024-05-27T09:35:59.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.495+0000 7f37e6400640 1 -- 172.21.15.114:0/2252286454 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f37bc076300 msgr2=0x7f37bc078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:59.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.495+0000 7f37e6400640 1 --2- 172.21.15.114:0/2252286454 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f37bc076300 0x7f37bc078810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009aa0 tx=0x7f37d0009190 comp rx=0 tx=0).stop 2024-05-27T09:35:59.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.495+0000 7f37e6400640 1 -- 172.21.15.114:0/2252286454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37e0074a60 msgr2=0x7f37e00c3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:59.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.495+0000 7f37e6400640 1 --2- 172.21.15.114:0/2252286454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37e0074a60 0x7f37e00c3bc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f37c800c430 tx=0x7f37c800c8f0 comp rx=0 tx=0).stop 2024-05-27T09:35:59.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.495+0000 7f37e6400640 1 -- 172.21.15.114:0/2252286454 shutdown_connections 2024-05-27T09:35:59.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.495+0000 7f37e6400640 1 --2- 172.21.15.114:0/2252286454 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f37bc076300 0x7f37bc078810 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:59.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.495+0000 7f37e6400640 1 --2- 172.21.15.114:0/2252286454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37e00ca9e0 0x7f37e01520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:59.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.496+0000 7f37e6400640 1 --2- 172.21.15.114:0/2252286454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37e0074a60 0x7f37e00c3bc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:59.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.496+0000 7f37e6400640 1 --2- 172.21.15.114:0/2252286454 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37e0073260 0x7f37e01f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:59.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.496+0000 7f37e6400640 1 -- 172.21.15.114:0/2252286454 >> 172.21.15.114:0/2252286454 conn(0x7f37e006f6e0 msgr2=0x7f37e006c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:59.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.496+0000 7f37e6400640 1 -- 172.21.15.114:0/2252286454 shutdown_connections 2024-05-27T09:35:59.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:35:59.496+0000 7f37e6400640 1 -- 172.21.15.114:0/2252286454 wait complete. 2024-05-27T09:35:59.524 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:36:01.034 INFO:tasks.ceph.mds.c.smithi114.stderr:2024-05-27T09:36:01.034+0000 7fd2b8600640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-27T09:36:01.526 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:36:01.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.831+0000 7fe2e8e00640 1 Processor -- start 2024-05-27T09:36:01.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.831+0000 7fe2e8e00640 1 -- start start 2024-05-27T09:36:01.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.834+0000 7fe2e8e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e4071d20 0x7fe2e415deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:01.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.834+0000 7fe2e8e00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe2e415e3f0 0x7fe2e4164e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:01.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.834+0000 7fe2e8e00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2e40c9d30 con 0x7fe2e4154a30 2024-05-27T09:36:01.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.834+0000 7fe2e8e00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fe2e40c9ae0 con 0x7fe2e415e3f0 2024-05-27T09:36:01.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.834+0000 7fe2e8e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fe2e40c86a0 con 0x7fe2e4071d20 2024-05-27T09:36:01.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.834+0000 7fe2e1400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e4071d20 0x7fe2e415deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:01.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.834+0000 7fe2e1400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e4071d20 0x7fe2e415deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:49530/0 (socket says 172.21.15.114:49530) 2024-05-27T09:36:01.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.834+0000 7fe2e1e00640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe2e4154a30 0x7fe2e40c73b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:56570/0 (socket says 172.21.15.114:56570) 2024-05-27T09:36:01.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.834+0000 7fe2e1400640 1 -- 172.21.15.114:0/1856810626 learned_addr learned my addr 172.21.15.114:0/1856810626 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:36:01.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.834+0000 7fe2e2800640 1 --2- 172.21.15.114:0/1856810626 >> v2:172.21.15.163:3300/0 conn(0x7fe2e415e3f0 0x7fe2e4164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:01.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.835+0000 7fe2e1400640 1 -- 172.21.15.114:0/1856810626 >> v2:172.21.15.163:3300/0 conn(0x7fe2e415e3f0 msgr2=0x7fe2e4164e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:36:01.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.835+0000 7fe2e1400640 1 --2- 172.21.15.114:0/1856810626 >> v2:172.21.15.163:3300/0 conn(0x7fe2e415e3f0 0x7fe2e4164e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:01.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.835+0000 7fe2e1400640 1 -- 172.21.15.114:0/1856810626 >> v1:172.21.15.163:6789/0 conn(0x7fe2e4154a30 legacy=0x7fe2e40c73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:01.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.835+0000 7fe2e0a00640 1 -- 172.21.15.114:0/1856810626 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2089549406 0 0) 0x7fe2e40c9d30 con 0x7fe2e4154a30 2024-05-27T09:36:01.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.835+0000 7fe2e1400640 1 -- 172.21.15.114:0/1856810626 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e40c8980 con 0x7fe2e4071d20 2024-05-27T09:36:01.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.836+0000 7fe2e1400640 1 --2- 172.21.15.114:0/1856810626 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e4071d20 0x7fe2e415deb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4009870 tx=0x7fe2d402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c945ee49f72a1e56 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:01.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.836+0000 7fe2e0a00640 1 -- 172.21.15.114:0/1856810626 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4037070 con 0x7fe2e4071d20 2024-05-27T09:36:01.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.836+0000 7fe2e0a00640 1 -- 172.21.15.114:0/1856810626 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2d4032440 con 0x7fe2e4071d20 2024-05-27T09:36:01.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.836+0000 7fe2e0a00640 1 -- 172.21.15.114:0/1856810626 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4032810 con 0x7fe2e4071d20 2024-05-27T09:36:01.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.837+0000 7fe2e8e00640 1 -- 172.21.15.114:0/1856810626 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e4071d20 msgr2=0x7fe2e415deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:01.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.837+0000 7fe2e8e00640 1 --2- 172.21.15.114:0/1856810626 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e4071d20 0x7fe2e415deb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4009870 tx=0x7fe2d402a270 comp rx=0 tx=0).stop 2024-05-27T09:36:01.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.837+0000 7fe2e8e00640 1 -- 172.21.15.114:0/1856810626 shutdown_connections 2024-05-27T09:36:01.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.837+0000 7fe2e8e00640 1 --2- 172.21.15.114:0/1856810626 >> v2:172.21.15.163:3300/0 conn(0x7fe2e415e3f0 0x7fe2e4164e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:01.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.837+0000 7fe2e8e00640 1 --2- 172.21.15.114:0/1856810626 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e4071d20 0x7fe2e415deb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:01.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.837+0000 7fe2e8e00640 1 -- 172.21.15.114:0/1856810626 >> 172.21.15.114:0/1856810626 conn(0x7fe2e40cdca0 msgr2=0x7fe2e40ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:01.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.837+0000 7fe2e8e00640 1 -- 172.21.15.114:0/1856810626 shutdown_connections 2024-05-27T09:36:01.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.837+0000 7fe2e8e00640 1 -- 172.21.15.114:0/1856810626 wait complete. 2024-05-27T09:36:01.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.840+0000 7fe2e8e00640 1 Processor -- start 2024-05-27T09:36:01.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.840+0000 7fe2e8e00640 1 -- start start 2024-05-27T09:36:01.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.841+0000 7fe2e8e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2e4071d20 0x7fe2e41f34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:01.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.841+0000 7fe2e8e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e4154a30 0x7fe2e40737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:01.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.841+0000 7fe2e8e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2e415e3f0 0x7fe2e4158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:01.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.841+0000 7fe2e8e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e40709e0 con 0x7fe2e4071d20 2024-05-27T09:36:01.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.841+0000 7fe2e8e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e415e800 con 0x7fe2e415e3f0 2024-05-27T09:36:01.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.841+0000 7fe2e8e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fe2e4070b60 con 0x7fe2e4154a30 2024-05-27T09:36:01.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.842+0000 7fe2e1e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2e4071d20 0x7fe2e41f34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:01.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.842+0000 7fe2e1e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2e4071d20 0x7fe2e41f34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48648/0 (socket says 172.21.15.114:48648) 2024-05-27T09:36:01.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.842+0000 7fe2e1e00640 1 -- 172.21.15.114:0/1627444977 learned_addr learned my addr 172.21.15.114:0/1627444977 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:36:01.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.842+0000 7fe2e1400640 1 --2- 172.21.15.114:0/1627444977 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e4154a30 0x7fe2e40737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:01.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.842+0000 7fe2e2800640 1 --2- 172.21.15.114:0/1627444977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2e415e3f0 0x7fe2e4158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:01.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.842+0000 7fe2e1e00640 1 -- 172.21.15.114:0/1627444977 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e4154a30 msgr2=0x7fe2e40737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:01.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.842+0000 7fe2e1e00640 1 --2- 172.21.15.114:0/1627444977 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e4154a30 0x7fe2e40737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:01.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.842+0000 7fe2e1e00640 1 -- 172.21.15.114:0/1627444977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2e415e3f0 msgr2=0x7fe2e4158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:01.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.842+0000 7fe2e1e00640 1 --2- 172.21.15.114:0/1627444977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2e415e3f0 0x7fe2e4158af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:01.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.842+0000 7fe2e1e00640 1 -- 172.21.15.114:0/1627444977 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e41f39f0 con 0x7fe2e4071d20 2024-05-27T09:36:01.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.842+0000 7fe2e1400640 1 --2- 172.21.15.114:0/1627444977 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e4154a30 0x7fe2e40737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:36:01.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.843+0000 7fe2e1e00640 1 --2- 172.21.15.114:0/1627444977 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2e4071d20 0x7fe2e41f34b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe2d000c3e0 tx=0x7fe2d000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:01.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.843+0000 7fe2da000640 1 -- 172.21.15.114:0/1627444977 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d00074e0 con 0x7fe2e4071d20 2024-05-27T09:36:01.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.843+0000 7fe2da000640 1 -- 172.21.15.114:0/1627444977 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2d00047e0 con 0x7fe2e4071d20 2024-05-27T09:36:01.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.843+0000 7fe2da000640 1 -- 172.21.15.114:0/1627444977 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0004bd0 con 0x7fe2e4071d20 2024-05-27T09:36:01.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.843+0000 7fe2e8e00640 1 -- 172.21.15.114:0/1627444977 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2e4154f90 con 0x7fe2e4071d20 2024-05-27T09:36:01.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.843+0000 7fe2e8e00640 1 -- 172.21.15.114:0/1627444977 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2e4155510 con 0x7fe2e4071d20 2024-05-27T09:36:01.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.844+0000 7fe2e8e00640 1 -- 172.21.15.114:0/1627444977 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2e406d6a0 con 0x7fe2e4071d20 2024-05-27T09:36:01.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.846+0000 7fe2da000640 1 -- 172.21.15.114:0/1627444977 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe2d0011070 con 0x7fe2e4071d20 2024-05-27T09:36:01.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.847+0000 7fe2da000640 1 --2- 172.21.15.114:0/1627444977 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe2c4076300 0x7fe2c4078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:01.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.847+0000 7fe2da000640 1 -- 172.21.15.114:0/1627444977 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6168+0+0 (secure 0 0 0) 0x7fe2d0097260 con 0x7fe2e4071d20 2024-05-27T09:36:01.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.847+0000 7fe2e1400640 1 --2- 172.21.15.114:0/1627444977 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe2c4076300 0x7fe2c4078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:01.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.848+0000 7fe2e1400640 1 --2- 172.21.15.114:0/1627444977 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe2c4076300 0x7fe2c4078810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4034000 tx=0x7fe2d402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:01.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:01.852+0000 7fe2da000640 1 -- 172.21.15.114:0/1627444977 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe2e406d6a0 con 0x7fe2e4071d20 2024-05-27T09:36:02.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.118+0000 7fe2e8e00640 1 -- 172.21.15.114:0/1627444977 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2e406a380 con 0x7fe2e4071d20 2024-05-27T09:36:02.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.119+0000 7fe2da000640 1 -- 172.21.15.114:0/1627444977 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+6062 (secure 0 0 0) 0x7fe2d0060300 con 0x7fe2e4071d20 2024-05-27T09:36:02.120 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:36:02.120 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:36:01.014181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:active","state_seq":26,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[4875,5086,5160]},"id":1}]} 2024-05-27T09:36:02.120 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 41 2024-05-27T09:36:02.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.123+0000 7fe2e8e00640 1 -- 172.21.15.114:0/1627444977 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe2c4076300 msgr2=0x7fe2c4078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:02.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.123+0000 7fe2e8e00640 1 --2- 172.21.15.114:0/1627444977 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe2c4076300 0x7fe2c4078810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4034000 tx=0x7fe2d402e000 comp rx=0 tx=0).stop 2024-05-27T09:36:02.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.123+0000 7fe2e8e00640 1 -- 172.21.15.114:0/1627444977 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2e4071d20 msgr2=0x7fe2e41f34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:02.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.123+0000 7fe2e8e00640 1 --2- 172.21.15.114:0/1627444977 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2e4071d20 0x7fe2e41f34b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe2d000c3e0 tx=0x7fe2d000c8a0 comp rx=0 tx=0).stop 2024-05-27T09:36:02.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.123+0000 7fe2e8e00640 1 -- 172.21.15.114:0/1627444977 shutdown_connections 2024-05-27T09:36:02.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.123+0000 7fe2e8e00640 1 --2- 172.21.15.114:0/1627444977 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe2c4076300 0x7fe2c4078810 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:02.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.123+0000 7fe2e8e00640 1 --2- 172.21.15.114:0/1627444977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2e415e3f0 0x7fe2e4158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:02.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.123+0000 7fe2e8e00640 1 --2- 172.21.15.114:0/1627444977 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e4154a30 0x7fe2e40737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:02.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.123+0000 7fe2e8e00640 1 --2- 172.21.15.114:0/1627444977 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2e4071d20 0x7fe2e41f34b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:02.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.123+0000 7fe2e8e00640 1 -- 172.21.15.114:0/1627444977 >> 172.21.15.114:0/1627444977 conn(0x7fe2e40cdca0 msgr2=0x7fe2e41514e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:02.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.123+0000 7fe2e8e00640 1 -- 172.21.15.114:0/1627444977 shutdown_connections 2024-05-27T09:36:02.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.123+0000 7fe2e8e00640 1 -- 172.21.15.114:0/1627444977 wait complete. 2024-05-27T09:36:02.152 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=2, replacing gid=4828 2024-05-27T09:36:02.153 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 0 secs before reviving mds.f 2024-05-27T09:36:02.153 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2024-05-27T09:36:02.153 INFO:tasks.ceph.mds.f:Restarting daemon 2024-05-27T09:36:02.153 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-05-27T09:36:02.156 INFO:tasks.ceph.mds.f:Started 2024-05-27T09:36:02.156 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:36:02.249 INFO:tasks.ceph.mds.f.smithi163.stdout:starting mds.f at 2024-05-27T09:36:02.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.438+0000 7fb299c00640 1 Processor -- start 2024-05-27T09:36:02.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.438+0000 7fb299c00640 1 -- start start 2024-05-27T09:36:02.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.440+0000 7fb299c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2940ca510 0x7fb2941569e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:02.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.440+0000 7fb299c00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb2940c91a0 0x7fb294073ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:02.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.440+0000 7fb299c00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2940c5770 con 0x7fb2940c72d0 2024-05-27T09:36:02.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.440+0000 7fb299c00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fb29415cbd0 con 0x7fb2940c91a0 2024-05-27T09:36:02.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.440+0000 7fb299c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fb2940c5a50 con 0x7fb2940ca510 2024-05-27T09:36:02.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.440+0000 7fb292a00640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb2940c72d0 0x7fb29415d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:56584/0 (socket says 172.21.15.114:56584) 2024-05-27T09:36:02.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.441+0000 7fb292a00640 1 -- 172.21.15.114:0/563374076 learned_addr learned my addr 172.21.15.114:0/563374076 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:36:02.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.441+0000 7fb292000640 1 --2- 172.21.15.114:0/563374076 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2940ca510 0x7fb2941569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:02.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.441+0000 7fb293400640 1 --2- 172.21.15.114:0/563374076 >> v2:172.21.15.163:3300/0 conn(0x7fb2940c91a0 0x7fb294073ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:02.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.441+0000 7fb291600640 1 -- 172.21.15.114:0/563374076 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3485697354 0 0) 0x7fb2940c5770 con 0x7fb2940c72d0 2024-05-27T09:36:02.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.441+0000 7fb291600640 1 -- 172.21.15.114:0/563374076 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb278003520 con 0x7fb2940c72d0 2024-05-27T09:36:02.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.441+0000 7fb293400640 1 -- 172.21.15.114:0/563374076 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2940ca510 msgr2=0x7fb2941569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:36:02.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.441+0000 7fb293400640 1 --2- 172.21.15.114:0/563374076 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2940ca510 0x7fb2941569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:02.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.441+0000 7fb293400640 1 -- 172.21.15.114:0/563374076 >> v1:172.21.15.163:6789/0 conn(0x7fb2940c72d0 legacy=0x7fb29415d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:02.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.441+0000 7fb293400640 1 -- 172.21.15.114:0/563374076 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29415c860 con 0x7fb2940c91a0 2024-05-27T09:36:02.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.442+0000 7fb293400640 1 --2- 172.21.15.114:0/563374076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2940c91a0 0x7fb294073ac0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009d90 tx=0x7fb28c02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c85a6a9faeb5ce9b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:02.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.442+0000 7fb291600640 1 -- 172.21.15.114:0/563374076 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c007740 con 0x7fb2940c91a0 2024-05-27T09:36:02.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.442+0000 7fb291600640 1 -- 172.21.15.114:0/563374076 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb28c0078e0 con 0x7fb2940c91a0 2024-05-27T09:36:02.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.442+0000 7fb291600640 1 -- 172.21.15.114:0/563374076 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c007cb0 con 0x7fb2940c91a0 2024-05-27T09:36:02.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.443+0000 7fb299c00640 1 -- 172.21.15.114:0/563374076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2940c91a0 msgr2=0x7fb294073ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:02.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.443+0000 7fb299c00640 1 --2- 172.21.15.114:0/563374076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2940c91a0 0x7fb294073ac0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009d90 tx=0x7fb28c02a6f0 comp rx=0 tx=0).stop 2024-05-27T09:36:02.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.443+0000 7fb299c00640 1 -- 172.21.15.114:0/563374076 shutdown_connections 2024-05-27T09:36:02.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.443+0000 7fb299c00640 1 --2- 172.21.15.114:0/563374076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2940c91a0 0x7fb294073ac0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:02.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.443+0000 7fb299c00640 1 --2- 172.21.15.114:0/563374076 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2940ca510 0x7fb2941569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:02.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.443+0000 7fb299c00640 1 -- 172.21.15.114:0/563374076 >> 172.21.15.114:0/563374076 conn(0x7fb2940ce0d0 msgr2=0x7fb2940ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:02.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.443+0000 7fb299c00640 1 -- 172.21.15.114:0/563374076 shutdown_connections 2024-05-27T09:36:02.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.443+0000 7fb299c00640 1 -- 172.21.15.114:0/563374076 wait complete. 2024-05-27T09:36:02.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.448+0000 7fb299c00640 1 Processor -- start 2024-05-27T09:36:02.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.448+0000 7fb299c00640 1 -- start start 2024-05-27T09:36:02.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.450+0000 7fb299c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2940c72d0 0x7fb2941519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:02.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.450+0000 7fb299c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2940c91a0 0x7fb2941504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:02.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.450+0000 7fb299c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2940ca510 0x7fb294150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:02.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.450+0000 7fb299c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb294074cd0 con 0x7fb2940c72d0 2024-05-27T09:36:02.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.450+0000 7fb299c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb294074b50 con 0x7fb2940ca510 2024-05-27T09:36:02.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.450+0000 7fb299c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fb294074000 con 0x7fb2940c91a0 2024-05-27T09:36:02.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.450+0000 7fb292a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2940c72d0 0x7fb2941519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:02.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.451+0000 7fb292000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2940c91a0 0x7fb2941504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:02.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.451+0000 7fb292a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2940c72d0 0x7fb2941519e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48658/0 (socket says 172.21.15.114:48658) 2024-05-27T09:36:02.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.451+0000 7fb292a00640 1 -- 172.21.15.114:0/2983799576 learned_addr learned my addr 172.21.15.114:0/2983799576 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:36:02.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.451+0000 7fb293400640 1 --2- 172.21.15.114:0/2983799576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2940ca510 0x7fb294150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:02.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.451+0000 7fb292000640 1 -- 172.21.15.114:0/2983799576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2940ca510 msgr2=0x7fb294150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:02.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.451+0000 7fb292000640 1 --2- 172.21.15.114:0/2983799576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2940ca510 0x7fb294150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:02.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.451+0000 7fb292000640 1 -- 172.21.15.114:0/2983799576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2940c72d0 msgr2=0x7fb2941519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:02.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.451+0000 7fb292000640 1 --2- 172.21.15.114:0/2983799576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2940c72d0 0x7fb2941519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:02.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.451+0000 7fb292000640 1 -- 172.21.15.114:0/2983799576 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb294151f20 con 0x7fb2940c91a0 2024-05-27T09:36:02.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.451+0000 7fb293400640 1 --2- 172.21.15.114:0/2983799576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2940ca510 0x7fb294150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:36:02.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.452+0000 7fb292000640 1 --2- 172.21.15.114:0/2983799576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2940c91a0 0x7fb2941504c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb28400e8a0 tx=0x7fb28400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:02.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.452+0000 7fb28aa00640 1 -- 172.21.15.114:0/2983799576 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb284017070 con 0x7fb2940c91a0 2024-05-27T09:36:02.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.452+0000 7fb28aa00640 1 -- 172.21.15.114:0/2983799576 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb28400c940 con 0x7fb2940c91a0 2024-05-27T09:36:02.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.452+0000 7fb299c00640 1 -- 172.21.15.114:0/2983799576 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb294151220 con 0x7fb2940c91a0 2024-05-27T09:36:02.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.452+0000 7fb28aa00640 1 -- 172.21.15.114:0/2983799576 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb284009e60 con 0x7fb2940c91a0 2024-05-27T09:36:02.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.452+0000 7fb299c00640 1 -- 172.21.15.114:0/2983799576 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2941f2e80 con 0x7fb2940c91a0 2024-05-27T09:36:02.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.454+0000 7fb299c00640 1 -- 172.21.15.114:0/2983799576 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb294071080 con 0x7fb2940c91a0 2024-05-27T09:36:02.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.454+0000 7fb28aa00640 1 -- 172.21.15.114:0/2983799576 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb28401a070 con 0x7fb2940c91a0 2024-05-27T09:36:02.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.456+0000 7fb28aa00640 1 --2- 172.21.15.114:0/2983799576 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb2740762d0 0x7fb2740787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:02.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.456+0000 7fb28aa00640 1 -- 172.21.15.114:0/2983799576 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(49..49 src has 1..49) ==== 6168+0+0 (secure 0 0 0) 0x7fb2840974c0 con 0x7fb2940c91a0 2024-05-27T09:36:02.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.456+0000 7fb292a00640 1 --2- 172.21.15.114:0/2983799576 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb2740762d0 0x7fb2740787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:02.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.457+0000 7fb292a00640 1 --2- 172.21.15.114:0/2983799576 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb2740762d0 0x7fb2740787e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb280005880 tx=0x7fb280009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:02.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.460+0000 7fb28aa00640 1 -- 172.21.15.114:0/2983799576 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb294071080 con 0x7fb2940c91a0 2024-05-27T09:36:02.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.735+0000 7fb299c00640 1 -- 172.21.15.114:0/2983799576 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb29406a380 con 0x7fb2940c91a0 2024-05-27T09:36:02.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.736+0000 7fb28aa00640 1 -- 172.21.15.114:0/2983799576 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+6062 (secure 0 0 0) 0x7fb2840605f0 con 0x7fb2940c91a0 2024-05-27T09:36:02.737 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:36:02.737 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:36:01.014181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:active","state_seq":26,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[4875,5086,5160]},"id":1}]} 2024-05-27T09:36:02.737 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 41 2024-05-27T09:36:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.740+0000 7fb299c00640 1 -- 172.21.15.114:0/2983799576 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb2740762d0 msgr2=0x7fb2740787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.740+0000 7fb299c00640 1 --2- 172.21.15.114:0/2983799576 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb2740762d0 0x7fb2740787e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb280005880 tx=0x7fb280009040 comp rx=0 tx=0).stop 2024-05-27T09:36:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.740+0000 7fb299c00640 1 -- 172.21.15.114:0/2983799576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2940c91a0 msgr2=0x7fb2941504c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.740+0000 7fb299c00640 1 --2- 172.21.15.114:0/2983799576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2940c91a0 0x7fb2941504c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb28400e8a0 tx=0x7fb28400ed60 comp rx=0 tx=0).stop 2024-05-27T09:36:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.740+0000 7fb299c00640 1 -- 172.21.15.114:0/2983799576 shutdown_connections 2024-05-27T09:36:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.740+0000 7fb299c00640 1 --2- 172.21.15.114:0/2983799576 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb2740762d0 0x7fb2740787e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.740+0000 7fb299c00640 1 --2- 172.21.15.114:0/2983799576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2940ca510 0x7fb294150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.740+0000 7fb299c00640 1 --2- 172.21.15.114:0/2983799576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2940c91a0 0x7fb2941504c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.740+0000 7fb299c00640 1 --2- 172.21.15.114:0/2983799576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2940c72d0 0x7fb2941519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.740+0000 7fb299c00640 1 -- 172.21.15.114:0/2983799576 >> 172.21.15.114:0/2983799576 conn(0x7fb2940ce0d0 msgr2=0x7fb294075da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.740+0000 7fb299c00640 1 -- 172.21.15.114:0/2983799576 shutdown_connections 2024-05-27T09:36:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:02.740+0000 7fb299c00640 1 -- 172.21.15.114:0/2983799576 wait complete. 2024-05-27T09:36:02.767 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is in active, standby or standby-replay 2024-05-27T09:36:04.770 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:36:05.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.060+0000 7fc5c8a00640 1 Processor -- start 2024-05-27T09:36:05.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.060+0000 7fc5c8a00640 1 -- start start 2024-05-27T09:36:05.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.062+0000 7fc5c8a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc5c40b75b0 0x7fc5c40d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:05.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.062+0000 7fc5c8a00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5c40cd1e0 con 0x7fc5c40c8780 2024-05-27T09:36:05.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.062+0000 7fc5c8a00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5c40cd4c0 con 0x7fc5c40b46e0 2024-05-27T09:36:05.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.062+0000 7fc5c8a00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc5c40c7f80 con 0x7fc5c40b75b0 2024-05-27T09:36:05.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.062+0000 7fc5c3200640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc5c40b46e0 0x7fc5c4166fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:40184/0 (socket says 172.21.15.114:40184) 2024-05-27T09:36:05.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.062+0000 7fc5c3200640 1 -- 172.21.15.114:0/3244919120 learned_addr learned my addr 172.21.15.114:0/3244919120 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:36:05.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.062+0000 7fc5c1e00640 1 --2- 172.21.15.114:0/3244919120 >> v2:172.21.15.114:3300/0 conn(0x7fc5c40b75b0 0x7fc5c40d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:05.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.063+0000 7fc5c1400640 1 -- 172.21.15.114:0/3244919120 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1697570073 0 0) 0x7fc5c40cd4c0 con 0x7fc5c40b46e0 2024-05-27T09:36:05.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.063+0000 7fc5c1400640 1 -- 172.21.15.114:0/3244919120 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5a8003520 con 0x7fc5c40b46e0 2024-05-27T09:36:05.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.063+0000 7fc5c1e00640 1 -- 172.21.15.114:0/3244919120 >> v1:172.21.15.163:6789/0 conn(0x7fc5c40b46e0 legacy=0x7fc5c4166fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:05.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.063+0000 7fc5c1400640 1 -- 172.21.15.114:0/3244919120 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3324501961 0 0) 0x7fc5c40cd1e0 con 0x7fc5c40c8780 2024-05-27T09:36:05.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.063+0000 7fc5c1e00640 1 -- 172.21.15.114:0/3244919120 >> v1:172.21.15.114:6789/0 conn(0x7fc5c40c8780 legacy=0x7fc5c415fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:05.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.063+0000 7fc5c1e00640 1 -- 172.21.15.114:0/3244919120 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c40c6bb0 con 0x7fc5c40b75b0 2024-05-27T09:36:05.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.065+0000 7fc5c1e00640 1 --2- 172.21.15.114:0/3244919120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5c40b75b0 0x7fc5c40d1e50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40c6e40 tx=0x7fc5b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8344ef60393c13 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:05.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.065+0000 7fc5c1400640 1 -- 172.21.15.114:0/3244919120 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0037070 con 0x7fc5c40b75b0 2024-05-27T09:36:05.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.065+0000 7fc5c1400640 1 -- 172.21.15.114:0/3244919120 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b002acf0 con 0x7fc5c40b75b0 2024-05-27T09:36:05.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.065+0000 7fc5c1400640 1 -- 172.21.15.114:0/3244919120 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc001a70 con 0x7fc5c40b75b0 2024-05-27T09:36:05.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.066+0000 7fc5c8a00640 1 -- 172.21.15.114:0/3244919120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5c40b75b0 msgr2=0x7fc5c40d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:05.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.066+0000 7fc5c8a00640 1 --2- 172.21.15.114:0/3244919120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5c40b75b0 0x7fc5c40d1e50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40c6e40 tx=0x7fc5b002a1e0 comp rx=0 tx=0).stop 2024-05-27T09:36:05.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.066+0000 7fc5c8a00640 1 -- 172.21.15.114:0/3244919120 shutdown_connections 2024-05-27T09:36:05.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.066+0000 7fc5c8a00640 1 --2- 172.21.15.114:0/3244919120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5c40b75b0 0x7fc5c40d1e50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:05.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.066+0000 7fc5c8a00640 1 -- 172.21.15.114:0/3244919120 >> 172.21.15.114:0/3244919120 conn(0x7fc5c406e040 msgr2=0x7fc5c406e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:05.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.066+0000 7fc5c8a00640 1 -- 172.21.15.114:0/3244919120 shutdown_connections 2024-05-27T09:36:05.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.066+0000 7fc5c8a00640 1 -- 172.21.15.114:0/3244919120 wait complete. 2024-05-27T09:36:05.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.069+0000 7fc5c8a00640 1 Processor -- start 2024-05-27T09:36:05.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.069+0000 7fc5c8a00640 1 -- start start 2024-05-27T09:36:05.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.071+0000 7fc5c8a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5c40b46e0 0x7fc5c41f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:05.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.071+0000 7fc5c8a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5c40b75b0 0x7fc5c41e80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:05.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.071+0000 7fc5c8a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5c40c8780 0x7fc5c40d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:05.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.071+0000 7fc5c8a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c40ce0b0 con 0x7fc5c40b46e0 2024-05-27T09:36:05.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.071+0000 7fc5c8a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c40cdf30 con 0x7fc5c40c8780 2024-05-27T09:36:05.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.071+0000 7fc5c8a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c4156c40 con 0x7fc5c40b75b0 2024-05-27T09:36:05.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.071+0000 7fc5c2800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5c40b46e0 0x7fc5c41f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:05.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.071+0000 7fc5c2800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5c40b46e0 0x7fc5c41f5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35702/0 (socket says 172.21.15.114:35702) 2024-05-27T09:36:05.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.071+0000 7fc5c2800640 1 -- 172.21.15.114:0/2404282137 learned_addr learned my addr 172.21.15.114:0/2404282137 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:36:05.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.071+0000 7fc5c3200640 1 --2- 172.21.15.114:0/2404282137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5c40c8780 0x7fc5c40d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:05.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.071+0000 7fc5c1e00640 1 --2- 172.21.15.114:0/2404282137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5c40b75b0 0x7fc5c41e80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:05.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.072+0000 7fc5c2800640 1 -- 172.21.15.114:0/2404282137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5c40b75b0 msgr2=0x7fc5c41e80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:05.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.072+0000 7fc5c2800640 1 --2- 172.21.15.114:0/2404282137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5c40b75b0 0x7fc5c41e80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:05.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.072+0000 7fc5c2800640 1 -- 172.21.15.114:0/2404282137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5c40c8780 msgr2=0x7fc5c40d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:05.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.072+0000 7fc5c2800640 1 --2- 172.21.15.114:0/2404282137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5c40c8780 0x7fc5c40d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:05.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.072+0000 7fc5c2800640 1 -- 172.21.15.114:0/2404282137 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c41f62f0 con 0x7fc5c40b46e0 2024-05-27T09:36:05.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.072+0000 7fc5c2800640 1 --2- 172.21.15.114:0/2404282137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5c40b46e0 0x7fc5c41f5db0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400c3c0 tx=0x7fc5b400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:05.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.072+0000 7fc5baa00640 1 -- 172.21.15.114:0/2404282137 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4017070 con 0x7fc5c40b46e0 2024-05-27T09:36:05.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.073+0000 7fc5baa00640 1 -- 172.21.15.114:0/2404282137 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b40035f0 con 0x7fc5c40b46e0 2024-05-27T09:36:05.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.073+0000 7fc5baa00640 1 -- 172.21.15.114:0/2404282137 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b40039c0 con 0x7fc5c40b46e0 2024-05-27T09:36:05.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.073+0000 7fc5c8a00640 1 -- 172.21.15.114:0/2404282137 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c41f28e0 con 0x7fc5c40b46e0 2024-05-27T09:36:05.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.073+0000 7fc5c8a00640 1 -- 172.21.15.114:0/2404282137 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c41f2ea0 con 0x7fc5c40b46e0 2024-05-27T09:36:05.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.074+0000 7fc5c8a00640 1 -- 172.21.15.114:0/2404282137 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c4050860 con 0x7fc5c40b46e0 2024-05-27T09:36:05.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.076+0000 7fc5baa00640 1 -- 172.21.15.114:0/2404282137 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc5b401a070 con 0x7fc5c40b46e0 2024-05-27T09:36:05.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.077+0000 7fc5baa00640 1 --2- 172.21.15.114:0/2404282137 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc5a4076300 0x7fc5a4078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:05.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.077+0000 7fc5baa00640 1 -- 172.21.15.114:0/2404282137 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6168+0+0 (secure 0 0 0) 0x7fc5b4096a00 con 0x7fc5c40b46e0 2024-05-27T09:36:05.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.078+0000 7fc5c1e00640 1 --2- 172.21.15.114:0/2404282137 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc5a4076300 0x7fc5a4078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:05.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.079+0000 7fc5c1e00640 1 --2- 172.21.15.114:0/2404282137 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc5a4076300 0x7fc5a4078810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0034000 tx=0x7fc5b002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:05.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.083+0000 7fc5baa00640 1 -- 172.21.15.114:0/2404282137 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc5b405fb40 con 0x7fc5c40b46e0 2024-05-27T09:36:05.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.492+0000 7fc5c8a00640 1 -- 172.21.15.114:0/2404282137 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5c41542d0 con 0x7fc5c40b46e0 2024-05-27T09:36:05.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.494+0000 7fc5baa00640 1 -- 172.21.15.114:0/2404282137 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+6890 (secure 0 0 0) 0x7fc5b4068180 con 0x7fc5c40b46e0 2024-05-27T09:36:05.494 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:36:05.494 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5072,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/1057092553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":1057092553},{"type":"v1","addr":"172.21.15.163:6839","nonce":1057092553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:36:01.014181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:active","state_seq":26,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5160,5086,4875]},"id":1}]} 2024-05-27T09:36:05.495 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 42 2024-05-27T09:36:05.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.498+0000 7fc5c8a00640 1 -- 172.21.15.114:0/2404282137 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc5a4076300 msgr2=0x7fc5a4078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:05.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.498+0000 7fc5c8a00640 1 --2- 172.21.15.114:0/2404282137 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc5a4076300 0x7fc5a4078810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0034000 tx=0x7fc5b002e000 comp rx=0 tx=0).stop 2024-05-27T09:36:05.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.498+0000 7fc5c8a00640 1 -- 172.21.15.114:0/2404282137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5c40b46e0 msgr2=0x7fc5c41f5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:05.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.498+0000 7fc5c8a00640 1 --2- 172.21.15.114:0/2404282137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5c40b46e0 0x7fc5c41f5db0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400c3c0 tx=0x7fc5b400c880 comp rx=0 tx=0).stop 2024-05-27T09:36:05.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.498+0000 7fc5c8a00640 1 -- 172.21.15.114:0/2404282137 shutdown_connections 2024-05-27T09:36:05.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.498+0000 7fc5c8a00640 1 --2- 172.21.15.114:0/2404282137 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc5a4076300 0x7fc5a4078810 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:05.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.498+0000 7fc5c8a00640 1 --2- 172.21.15.114:0/2404282137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5c40c8780 0x7fc5c40d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:05.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.498+0000 7fc5c8a00640 1 --2- 172.21.15.114:0/2404282137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5c40b75b0 0x7fc5c41e80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:05.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.498+0000 7fc5c8a00640 1 --2- 172.21.15.114:0/2404282137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5c40b46e0 0x7fc5c41f5db0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:05.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.498+0000 7fc5c8a00640 1 -- 172.21.15.114:0/2404282137 >> 172.21.15.114:0/2404282137 conn(0x7fc5c406e040 msgr2=0x7fc5c40ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:05.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.498+0000 7fc5c8a00640 1 -- 172.21.15.114:0/2404282137 shutdown_connections 2024-05-27T09:36:05.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:05.498+0000 7fc5c8a00640 1 -- 172.21.15.114:0/2404282137 wait complete. 2024-05-27T09:36:05.532 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2024-05-27T09:36:05.532 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 53 secs before thrashing 2024-05-27T09:36:20.033 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:36:20.043 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:36:50.180 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:36:50.189 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:36:58.533 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:36:58.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.816+0000 7f93e7e00640 1 Processor -- start 2024-05-27T09:36:58.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.817+0000 7f93e7e00640 1 -- start start 2024-05-27T09:36:58.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.819+0000 7f93e7e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93e8153140 0x7f93e80b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:58.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.819+0000 7f93e7e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f93e80cdb50 0x7f93e80ce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:58.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.819+0000 7f93e7e00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f93e80cee50 0x7f93e80ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:58.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.819+0000 7f93e7e00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f93e815c930 con 0x7f93e80cdb50 2024-05-27T09:36:58.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.819+0000 7f93e7e00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f93e815cc10 con 0x7f93e80cee50 2024-05-27T09:36:58.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.819+0000 7f93e7e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f93e815cef0 con 0x7f93e8153140 2024-05-27T09:36:58.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.819+0000 7f93dfe00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f93e80cdb50 0x7f93e80ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:58.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.819+0000 7f93e6a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93e8153140 0x7f93e80b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:58.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.819+0000 7f93e7400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f93e80cee50 0x7f93e80ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:58.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.819+0000 7f93e6a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93e8153140 0x7f93e80b45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:35444/0 (socket says 172.21.15.114:35444) 2024-05-27T09:36:58.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.819+0000 7f93dfe00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f93e80cdb50 0x7f93e80ce910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38778/0 (socket says 172.21.15.114:38778) 2024-05-27T09:36:58.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.819+0000 7f93e6a00640 1 -- 172.21.15.114:0/2966343581 learned_addr learned my addr 172.21.15.114:0/2966343581 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:36:58.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.820+0000 7f93e6a00640 1 -- 172.21.15.114:0/2966343581 >> v2:172.21.15.163:3300/0 conn(0x7f93e80cee50 msgr2=0x7f93e80ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:36:58.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.820+0000 7f93e6a00640 1 --2- 172.21.15.114:0/2966343581 >> v2:172.21.15.163:3300/0 conn(0x7f93e80cee50 0x7f93e80ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:58.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.820+0000 7f93e6a00640 1 -- 172.21.15.114:0/2966343581 >> v2:172.21.15.114:3300/0 conn(0x7f93e80cdb50 msgr2=0x7f93e80ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:36:58.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.820+0000 7f93e6a00640 1 --2- 172.21.15.114:0/2966343581 >> v2:172.21.15.114:3300/0 conn(0x7f93e80cdb50 0x7f93e80ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:58.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.820+0000 7f93e6a00640 1 -- 172.21.15.114:0/2966343581 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e815d1d0 con 0x7f93e8153140 2024-05-27T09:36:58.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.821+0000 7f93e6a00640 1 --2- 172.21.15.114:0/2966343581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93e8153140 0x7f93e80b45c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f93e806f4b0 tx=0x7f93d802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=122838fad4c93dd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:58.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.821+0000 7f93e6000640 1 -- 172.21.15.114:0/2966343581 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d8037070 con 0x7f93e8153140 2024-05-27T09:36:58.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.821+0000 7f93e6000640 1 -- 172.21.15.114:0/2966343581 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d8032440 con 0x7f93e8153140 2024-05-27T09:36:58.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.821+0000 7f93e6000640 1 -- 172.21.15.114:0/2966343581 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d8032810 con 0x7f93e8153140 2024-05-27T09:36:58.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.822+0000 7f93e7e00640 1 -- 172.21.15.114:0/2966343581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93e8153140 msgr2=0x7f93e80b45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:58.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.822+0000 7f93e7e00640 1 --2- 172.21.15.114:0/2966343581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93e8153140 0x7f93e80b45c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f93e806f4b0 tx=0x7f93d802a270 comp rx=0 tx=0).stop 2024-05-27T09:36:58.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.823+0000 7f93e7e00640 1 -- 172.21.15.114:0/2966343581 shutdown_connections 2024-05-27T09:36:58.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.823+0000 7f93e7e00640 1 --2- 172.21.15.114:0/2966343581 >> v2:172.21.15.163:3300/0 conn(0x7f93e80cee50 0x7f93e80ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:58.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.823+0000 7f93e7e00640 1 --2- 172.21.15.114:0/2966343581 >> v2:172.21.15.114:3300/0 conn(0x7f93e80cdb50 0x7f93e80ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:58.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.823+0000 7f93e7e00640 1 --2- 172.21.15.114:0/2966343581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93e8153140 0x7f93e80b45c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:58.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.823+0000 7f93e7e00640 1 -- 172.21.15.114:0/2966343581 >> 172.21.15.114:0/2966343581 conn(0x7f93e808d960 msgr2=0x7f93e808dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:58.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.823+0000 7f93e7e00640 1 -- 172.21.15.114:0/2966343581 shutdown_connections 2024-05-27T09:36:58.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.823+0000 7f93e7e00640 1 -- 172.21.15.114:0/2966343581 wait complete. 2024-05-27T09:36:58.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.826+0000 7f93e7e00640 1 Processor -- start 2024-05-27T09:36:58.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.827+0000 7f93e7e00640 1 -- start start 2024-05-27T09:36:58.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.829+0000 7f93e7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93e80cdb50 0x7f93e80c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:58.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.829+0000 7f93e7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93e80cee50 0x7f93e81506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:58.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.829+0000 7f93e7e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93e8153140 0x7f93e814dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:58.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.829+0000 7f93e7e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f93e80cc3d0 con 0x7f93e80cdb50 2024-05-27T09:36:58.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.829+0000 7f93e7e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f93e80cc250 con 0x7f93e80cee50 2024-05-27T09:36:58.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.829+0000 7f93e7e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f93e815c4f0 con 0x7f93e8153140 2024-05-27T09:36:58.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.830+0000 7f93e6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93e80cdb50 0x7f93e80c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:58.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.830+0000 7f93e6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93e80cdb50 0x7f93e80c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38780/0 (socket says 172.21.15.114:38780) 2024-05-27T09:36:58.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.830+0000 7f93e6a00640 1 -- 172.21.15.114:0/2549740822 learned_addr learned my addr 172.21.15.114:0/2549740822 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:36:58.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.830+0000 7f93dfe00640 1 --2- 172.21.15.114:0/2549740822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93e80cee50 0x7f93e81506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:58.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.830+0000 7f93e7400640 1 --2- 172.21.15.114:0/2549740822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93e8153140 0x7f93e814dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:58.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.830+0000 7f93e6a00640 1 -- 172.21.15.114:0/2549740822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93e8153140 msgr2=0x7f93e814dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:58.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.830+0000 7f93e6a00640 1 --2- 172.21.15.114:0/2549740822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93e8153140 0x7f93e814dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:58.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.830+0000 7f93e6a00640 1 -- 172.21.15.114:0/2549740822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93e80cee50 msgr2=0x7f93e81506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:58.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.830+0000 7f93e6a00640 1 --2- 172.21.15.114:0/2549740822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93e80cee50 0x7f93e81506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:58.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.830+0000 7f93e6a00640 1 -- 172.21.15.114:0/2549740822 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e81f6cc0 con 0x7f93e80cdb50 2024-05-27T09:36:58.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.831+0000 7f93e6a00640 1 --2- 172.21.15.114:0/2549740822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93e80cdb50 0x7f93e80c8fb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f93d8000c00 tx=0x7f93d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:58.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.831+0000 7f93dea00640 1 -- 172.21.15.114:0/2549740822 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d8041020 con 0x7f93e80cdb50 2024-05-27T09:36:58.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.831+0000 7f93dea00640 1 -- 172.21.15.114:0/2549740822 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d803d620 con 0x7f93e80cdb50 2024-05-27T09:36:58.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.831+0000 7f93e7e00640 1 -- 172.21.15.114:0/2549740822 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e80c94f0 con 0x7f93e80cdb50 2024-05-27T09:36:58.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.831+0000 7f93dea00640 1 -- 172.21.15.114:0/2549740822 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d8037040 con 0x7f93e80cdb50 2024-05-27T09:36:58.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.831+0000 7f93e7e00640 1 -- 172.21.15.114:0/2549740822 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e81f2d00 con 0x7f93e80cdb50 2024-05-27T09:36:58.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.833+0000 7f93e7e00640 1 -- 172.21.15.114:0/2549740822 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e80c8c00 con 0x7f93e80cdb50 2024-05-27T09:36:58.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.834+0000 7f93dea00640 1 -- 172.21.15.114:0/2549740822 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f93d8044020 con 0x7f93e80cdb50 2024-05-27T09:36:58.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.835+0000 7f93dea00640 1 --2- 172.21.15.114:0/2549740822 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f93c4076300 0x7f93c4078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:58.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.835+0000 7f93dea00640 1 -- 172.21.15.114:0/2549740822 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6168+0+0 (secure 0 0 0) 0x7f93d80b7730 con 0x7f93e80cdb50 2024-05-27T09:36:58.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.835+0000 7f93dfe00640 1 --2- 172.21.15.114:0/2549740822 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f93c4076300 0x7f93c4078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:58.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.836+0000 7f93dfe00640 1 --2- 172.21.15.114:0/2549740822 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f93c4076300 0x7f93c4078810 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f93e81ed170 tx=0x7f93d0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:58.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:58.840+0000 7f93dea00640 1 -- 172.21.15.114:0/2549740822 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f93e80c8c00 con 0x7f93e80cdb50 2024-05-27T09:36:59.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.248+0000 7f93e7e00640 1 -- 172.21.15.114:0/2549740822 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93e8050860 con 0x7f93e80cdb50 2024-05-27T09:36:59.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.249+0000 7f93dea00640 1 -- 172.21.15.114:0/2549740822 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+6890 (secure 0 0 0) 0x7f93d804c020 con 0x7f93e80cdb50 2024-05-27T09:36:59.249 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:36:59.249 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5072,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/1057092553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":1057092553},{"type":"v1","addr":"172.21.15.163:6839","nonce":1057092553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:36:01.014181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:active","state_seq":26,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5160,5086,4875]},"id":1}]} 2024-05-27T09:36:59.250 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 42 2024-05-27T09:36:59.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.251+0000 7f93e7e00640 1 -- 172.21.15.114:0/2549740822 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f93c4076300 msgr2=0x7f93c4078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:59.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.251+0000 7f93e7e00640 1 --2- 172.21.15.114:0/2549740822 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f93c4076300 0x7f93c4078810 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f93e81ed170 tx=0x7f93d0009190 comp rx=0 tx=0).stop 2024-05-27T09:36:59.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.251+0000 7f93e7e00640 1 -- 172.21.15.114:0/2549740822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93e80cdb50 msgr2=0x7f93e80c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:59.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.251+0000 7f93e7e00640 1 --2- 172.21.15.114:0/2549740822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93e80cdb50 0x7f93e80c8fb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f93d8000c00 tx=0x7f93d80046c0 comp rx=0 tx=0).stop 2024-05-27T09:36:59.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.251+0000 7f93e7e00640 1 -- 172.21.15.114:0/2549740822 shutdown_connections 2024-05-27T09:36:59.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.251+0000 7f93e7e00640 1 --2- 172.21.15.114:0/2549740822 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f93c4076300 0x7f93c4078810 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:59.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.251+0000 7f93e7e00640 1 --2- 172.21.15.114:0/2549740822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93e8153140 0x7f93e814dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:59.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.251+0000 7f93e7e00640 1 --2- 172.21.15.114:0/2549740822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93e80cee50 0x7f93e81506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:59.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.251+0000 7f93e7e00640 1 --2- 172.21.15.114:0/2549740822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93e80cdb50 0x7f93e80c8fb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:59.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.251+0000 7f93e7e00640 1 -- 172.21.15.114:0/2549740822 >> 172.21.15.114:0/2549740822 conn(0x7f93e808d960 msgr2=0x7f93e8151460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:59.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.252+0000 7f93e7e00640 1 -- 172.21.15.114:0/2549740822 shutdown_connections 2024-05-27T09:36:59.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.252+0000 7f93e7e00640 1 -- 172.21.15.114:0/2549740822 wait complete. 2024-05-27T09:36:59.277 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=1) 2024-05-27T09:36:59.277 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-27T09:36:59.277 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:36:59.318 INFO:tasks.ceph.mds.a:Stopped 2024-05-27T09:36:59.319 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:36:59.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.575+0000 7f96e7200640 1 Processor -- start 2024-05-27T09:36:59.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.575+0000 7f96e7200640 1 -- start start 2024-05-27T09:36:59.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.577+0000 7f96e7200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f96e8153140 0x7f96e80b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:59.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.577+0000 7f96e7200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96e80ccdd0 0x7f96e80cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:59.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.577+0000 7f96e7200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96e815c930 con 0x7f96e80cdb50 2024-05-27T09:36:59.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.577+0000 7f96e7200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f96e815cc10 con 0x7f96e8153140 2024-05-27T09:36:59.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.577+0000 7f96e7200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f96e815cef0 con 0x7f96e80ccdd0 2024-05-27T09:36:59.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.578+0000 7f96e5e00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f96e8153140 0x7f96e80b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:59.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.578+0000 7f96e5400640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f96e80cdb50 0x7f96e80ce910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:38884/0 (socket says 172.21.15.114:38884) 2024-05-27T09:36:59.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.578+0000 7f96e6800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96e80ccdd0 0x7f96e80cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:59.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.578+0000 7f96e5e00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f96e8153140 0x7f96e80b45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:51364/0 (socket says 172.21.15.114:51364) 2024-05-27T09:36:59.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.578+0000 7f96e5400640 1 -- 172.21.15.114:0/1694830524 learned_addr learned my addr 172.21.15.114:0/1694830524 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:36:59.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.578+0000 7f96e5e00640 1 -- 172.21.15.114:0/1694830524 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96e80ccdd0 msgr2=0x7f96e80cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:36:59.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.578+0000 7f96e5e00640 1 --2- 172.21.15.114:0/1694830524 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96e80ccdd0 0x7f96e80cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:59.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.578+0000 7f96e5e00640 1 -- 172.21.15.114:0/1694830524 >> v1:172.21.15.163:6789/0 conn(0x7f96e80cdb50 legacy=0x7f96e80ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:59.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.578+0000 7f96e4a00640 1 -- 172.21.15.114:0/1694830524 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2231851262 0 0) 0x7f96e815c930 con 0x7f96e80cdb50 2024-05-27T09:36:59.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.578+0000 7f96e5e00640 1 -- 172.21.15.114:0/1694830524 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e815d1d0 con 0x7f96e8153140 2024-05-27T09:36:59.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.578+0000 7f96e6800640 1 --2- 172.21.15.114:0/1694830524 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96e80ccdd0 0x7f96e80cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:36:59.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.580+0000 7f96e5e00640 1 --2- 172.21.15.114:0/1694830524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96e8153140 0x7f96e80b45c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f96d80097a0 tx=0x7f96d802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=743b3ad6683c84f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:59.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.580+0000 7f96e4a00640 1 -- 172.21.15.114:0/1694830524 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8037070 con 0x7f96e8153140 2024-05-27T09:36:59.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.580+0000 7f96e4a00640 1 -- 172.21.15.114:0/1694830524 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96d8032440 con 0x7f96e8153140 2024-05-27T09:36:59.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.580+0000 7f96e4a00640 1 -- 172.21.15.114:0/1694830524 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8032810 con 0x7f96e8153140 2024-05-27T09:36:59.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.581+0000 7f96e7200640 1 -- 172.21.15.114:0/1694830524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96e8153140 msgr2=0x7f96e80b45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:59.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.581+0000 7f96e7200640 1 --2- 172.21.15.114:0/1694830524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96e8153140 0x7f96e80b45c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f96d80097a0 tx=0x7f96d802a210 comp rx=0 tx=0).stop 2024-05-27T09:36:59.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.581+0000 7f96e7200640 1 -- 172.21.15.114:0/1694830524 shutdown_connections 2024-05-27T09:36:59.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.581+0000 7f96e7200640 1 --2- 172.21.15.114:0/1694830524 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96e80ccdd0 0x7f96e80cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:59.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.581+0000 7f96e7200640 1 --2- 172.21.15.114:0/1694830524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96e8153140 0x7f96e80b45c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:59.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.581+0000 7f96e7200640 1 -- 172.21.15.114:0/1694830524 >> 172.21.15.114:0/1694830524 conn(0x7f96e808d960 msgr2=0x7f96e808dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:59.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.581+0000 7f96e7200640 1 -- 172.21.15.114:0/1694830524 shutdown_connections 2024-05-27T09:36:59.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.581+0000 7f96e7200640 1 -- 172.21.15.114:0/1694830524 wait complete. 2024-05-27T09:36:59.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.585+0000 7f96e7200640 1 Processor -- start 2024-05-27T09:36:59.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.585+0000 7f96e7200640 1 -- start start 2024-05-27T09:36:59.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.588+0000 7f96e7200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96e80ccdd0 0x7f96e81e8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:59.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.588+0000 7f96e7200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96e80cdb50 0x7f96e814dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:59.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.588+0000 7f96e7200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96e8153140 0x7f96e815e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:59.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.588+0000 7f96e7200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f96e80cf1a0 con 0x7f96e80cdb50 2024-05-27T09:36:59.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.588+0000 7f96e7200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f96e80cf020 con 0x7f96e80ccdd0 2024-05-27T09:36:59.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.588+0000 7f96e7200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f96e80cc250 con 0x7f96e8153140 2024-05-27T09:36:59.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.588+0000 7f96e5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96e80ccdd0 0x7f96e81e8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:59.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.588+0000 7f96e5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96e80ccdd0 0x7f96e81e8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:51380/0 (socket says 172.21.15.114:51380) 2024-05-27T09:36:59.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.588+0000 7f96e5e00640 1 -- 172.21.15.114:0/3954244223 learned_addr learned my addr 172.21.15.114:0/3954244223 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:36:59.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.588+0000 7f96e6800640 1 --2- 172.21.15.114:0/3954244223 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96e8153140 0x7f96e815e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:59.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.588+0000 7f96e5400640 1 --2- 172.21.15.114:0/3954244223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96e80cdb50 0x7f96e814dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:59.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.589+0000 7f96e5e00640 1 -- 172.21.15.114:0/3954244223 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96e8153140 msgr2=0x7f96e815e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:59.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.589+0000 7f96e5e00640 1 --2- 172.21.15.114:0/3954244223 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96e8153140 0x7f96e815e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:59.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.589+0000 7f96e5e00640 1 -- 172.21.15.114:0/3954244223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96e80cdb50 msgr2=0x7f96e814dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:59.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.589+0000 7f96e5e00640 1 --2- 172.21.15.114:0/3954244223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96e80cdb50 0x7f96e814dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:59.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.589+0000 7f96e5e00640 1 -- 172.21.15.114:0/3954244223 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e81f2470 con 0x7f96e80ccdd0 2024-05-27T09:36:59.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.589+0000 7f96e5e00640 1 --2- 172.21.15.114:0/3954244223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96e80ccdd0 0x7f96e81e8870 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f96d8000c00 tx=0x7f96d80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:59.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.589+0000 7f96de000640 1 -- 172.21.15.114:0/3954244223 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8037050 con 0x7f96e80ccdd0 2024-05-27T09:36:59.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.589+0000 7f96de000640 1 -- 172.21.15.114:0/3954244223 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96d8038040 con 0x7f96e80ccdd0 2024-05-27T09:36:59.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.589+0000 7f96de000640 1 -- 172.21.15.114:0/3954244223 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8004470 con 0x7f96e80ccdd0 2024-05-27T09:36:59.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.589+0000 7f96e7200640 1 -- 172.21.15.114:0/3954244223 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e81f26f0 con 0x7f96e80ccdd0 2024-05-27T09:36:59.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.590+0000 7f96e7200640 1 -- 172.21.15.114:0/3954244223 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e81f2cb0 con 0x7f96e80ccdd0 2024-05-27T09:36:59.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.591+0000 7f96e7200640 1 -- 172.21.15.114:0/3954244223 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e8050860 con 0x7f96e80ccdd0 2024-05-27T09:36:59.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.592+0000 7f96de000640 1 -- 172.21.15.114:0/3954244223 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f96d8003610 con 0x7f96e80ccdd0 2024-05-27T09:36:59.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.594+0000 7f96de000640 1 --2- 172.21.15.114:0/3954244223 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f96c8076300 0x7f96c8078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:59.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.594+0000 7f96de000640 1 -- 172.21.15.114:0/3954244223 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6168+0+0 (secure 0 0 0) 0x7f96d80b7fe0 con 0x7f96e80ccdd0 2024-05-27T09:36:59.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.594+0000 7f96e5400640 1 --2- 172.21.15.114:0/3954244223 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f96c8076300 0x7f96c8078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:59.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.595+0000 7f96e5400640 1 --2- 172.21.15.114:0/3954244223 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f96c8076300 0x7f96c8078810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f96e81f00a0 tx=0x7f96d4007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:59.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.598+0000 7f96de000640 1 -- 172.21.15.114:0/3954244223 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f96d804c020 con 0x7f96e80ccdd0 2024-05-27T09:36:59.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.861+0000 7f96e7200640 1 -- 172.21.15.114:0/3954244223 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96e81608f0 con 0x7f96e80ccdd0 2024-05-27T09:36:59.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.862+0000 7f96de000640 1 -- 172.21.15.114:0/3954244223 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+6890 (secure 0 0 0) 0x7f96e81f2470 con 0x7f96e80ccdd0 2024-05-27T09:36:59.863 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:36:59.863 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5072,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/1057092553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":1057092553},{"type":"v1","addr":"172.21.15.163:6839","nonce":1057092553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:36:01.014181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:active","state_seq":26,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5160,5086,4875]},"id":1}]} 2024-05-27T09:36:59.863 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 42 2024-05-27T09:36:59.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.865+0000 7f96e7200640 1 -- 172.21.15.114:0/3954244223 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f96c8076300 msgr2=0x7f96c8078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:59.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.865+0000 7f96e7200640 1 --2- 172.21.15.114:0/3954244223 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f96c8076300 0x7f96c8078810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f96e81f00a0 tx=0x7f96d4007c00 comp rx=0 tx=0).stop 2024-05-27T09:36:59.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.865+0000 7f96e7200640 1 -- 172.21.15.114:0/3954244223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96e80ccdd0 msgr2=0x7f96e81e8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:59.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.865+0000 7f96e7200640 1 --2- 172.21.15.114:0/3954244223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96e80ccdd0 0x7f96e81e8870 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f96d8000c00 tx=0x7f96d80046c0 comp rx=0 tx=0).stop 2024-05-27T09:36:59.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.865+0000 7f96e7200640 1 -- 172.21.15.114:0/3954244223 shutdown_connections 2024-05-27T09:36:59.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.865+0000 7f96e7200640 1 --2- 172.21.15.114:0/3954244223 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f96c8076300 0x7f96c8078810 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:59.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.865+0000 7f96e7200640 1 --2- 172.21.15.114:0/3954244223 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96e8153140 0x7f96e815e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:59.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.865+0000 7f96e7200640 1 --2- 172.21.15.114:0/3954244223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96e80cdb50 0x7f96e814dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:59.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.865+0000 7f96e7200640 1 --2- 172.21.15.114:0/3954244223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96e80ccdd0 0x7f96e81e8870 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:59.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.865+0000 7f96e7200640 1 -- 172.21.15.114:0/3954244223 >> 172.21.15.114:0/3954244223 conn(0x7f96e808d960 msgr2=0x7f96e808dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:59.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.865+0000 7f96e7200640 1 -- 172.21.15.114:0/3954244223 shutdown_connections 2024-05-27T09:36:59.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:36:59.865+0000 7f96e7200640 1 -- 172.21.15.114:0/3954244223 wait complete. 2024-05-27T09:36:59.895 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-05-27T09:37:01.474 INFO:tasks.workunit.client.0.smithi114.stderr:Updating files: 3% (396/12563) Updating files: 4% (503/12563) Updating files: 5% (629/12563) Updating files: 5% (743/12563) Updating files: 6% (754/12563) Updating files: 7% (880/12563) Updating files: 7% (977/12563) Updating files: 8% (1006/12563) Updating files: 9% (1131/12563) Updating files: 9% (1209/12563) Updating files: 10% (1257/12563) Updating files: 11% (1382/12563) Updating files: 12% (1508/12563) Updating files: 12% (1601/12563) Updating files: 13% (1634/12563) Updating files: 14% (1759/12563) Updating files: 14% (1790/12563) Updating files: 15% (1885/12563) Updating files: 16% (2011/12563) Updating files: 16% (2084/12563) Updating files: 17% (2136/12563) Updating files: 18% (2262/12563) Updating files: 18% (2334/12563) Updating files: 19% (2387/12563) Updating files: 20% (2513/12563) Updating files: 20% (2558/12563) Updating files: 21% (2639/12563) Updating files: 22% (2764/12563) Updating files: 23% (2890/12563) Updating files: 23% (2903/12563) Updating files: 24% (3016/12563) Updating files: 25% (3141/12563) Updating files: 25% (3210/12563) Updating files: 26% (3267/12563) Updating files: 27% (3393/12563) Updating files: 28% (3518/12563) Updating files: 28% (3641/12563) Updating files: 29% (3644/12563) Updating files: 30% (3769/12563) Updating files: 30% (3825/12563) Updating files: 31% (3895/12563) Updating files: 31% (4004/12563) Updating files: 32% (4021/12563) Updating files: 33% (4146/12563) Updating files: 34% (4272/12563) Updating files: 34% (4323/12563) Updating files: 35% (4398/12563) Updating files: 35% (4450/12563) Updating files: 36% (4523/12563) Updating files: 37% (4649/12563) Updating files: 37% (4752/12563) Updating files: 38% (4774/12563) Updating files: 39% (4900/12563) Updating files: 39% (5022/12563) Updating files: 40% (5026/12563) Updating files: 41% (5151/12563) Updating files: 41% (5225/12563) Updating files: 41% (5263/12563) Updating files: 42% (5277/12563) Updating files: 43% (5403/12563) Updating files: 43% (5508/12563) Updating files: 44% (5528/12563) Updating files: 45% (5654/12563) Updating files: 45% (5747/12563) Updating files: 46% (5779/12563) Updating files: 47% (5905/12563) Updating files: 47% (5971/12563) Updating files: 48% (6031/12563) Updating files: 49% (6156/12563) Updating files: 49% (6274/12563) Updating files: 50% (6282/12563) Updating files: 51% (6408/12563) Updating files: 52% (6533/12563) Updating files: 52% (6584/12563) Updating files: 53% (6659/12563) Updating files: 54% (6785/12563) Updating files: 54% (6830/12563) Updating files: 55% (6910/12563) Updating files: 56% (7036/12563) Updating files: 56% (7143/12563) Updating files: 57% (7161/12563) Updating files: 58% (7287/12563) Updating files: 58% (7401/12563) Updating files: 59% (7413/12563) Updating files: 60% (7538/12563) Updating files: 60% (7656/12563) Updating files: 61% (7664/12563) Updating files: 61% (7754/12563) Updating files: 62% (7790/12563) Updating files: 63% (7915/12563) Updating files: 63% (8040/12563) Updating files: 64% (8041/12563) Updating files: 65% (8166/12563) Updating files: 66% (8292/12563) Updating files: 66% (8341/12563) Updating files: 67% (8418/12563) Updating files: 67% (8530/12563) Updating files: 68% (8543/12563) Updating files: 69% (8669/12563) Updating files: 69% (8718/12563) Updating files: 70% (8795/12563) Updating files: 70% (8902/12563) Updating files: 71% (8920/12563) Updating files: 72% (9046/12563) Updating files: 72% (9101/12563) Updating files: 72% (9153/12563) Updating files: 73% (9171/12563) Updating files: 74% (9297/12563) Updating files: 74% (9383/12563) Updating files: 75% (9423/12563) Updating files: 76% (9548/12563) Updating files: 76% (9644/12563) Updating files: 77% (9674/12563) Updating files: 78% (9800/12563) Updating files: 78% (9823/12563) Updating files: 79% (9925/12563) Updating files: 79% (9990/12563) Updating files: 80% (10051/12563) Updating files: 80% (10094/12563) Updating files: 81% (10177/12563) Updating files: 81% (10239/12563) Updating files: 82% (10302/12563) Updating files: 83% (10428/12563) Updating files: 83% (10494/12563) Updating files: 84% (10553/12563) Updating files: 84% (10625/12563) Updating files: 85% (10679/12563) Updating files: 85% (10776/12563) Updating files: 86% (10805/12563) Updating files: 87% (10930/12563) Updating files: 88% (11056/12563) Updating files: 88% (11086/12563) Updating files: 89% (11182/12563) Updating files: 90% (11307/12563) Updating files: 90% (11325/12563) Updating files: 91% (11433/12563) Updating files: 92% (11558/12563) Updating files: 92% (11571/12563) Updating files: 93% (11684/12563) Updating files: 94% (11810/12563) Updating files: 94% (11906/12563) Updating files: 95% (11935/12563) Updating files: 96% (12061/12563) Updating files: 96% (12064/12563) Updating files: 97% (12187/12563) Updating files: 98% (12312/12563) Updating files: 98% (12378/12563) Updating files: 99% (12438/12563) Updating files: 100% (12563/12563) Updating files: 100% (12563/12563), done. 2024-05-27T09:37:01.580 DEBUG:teuthology.orchestra.run:got remote process result: 128 2024-05-27T09:37:01.580 INFO:tasks.workunit:Stopping ['fs/snaps'] on client.0... 2024-05-27T09:37:01.580 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-27T09:37:01.896 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:37:02.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.164+0000 7f2b0da00640 1 Processor -- start 2024-05-27T09:37:02.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.164+0000 7f2b0da00640 1 -- start start 2024-05-27T09:37:02.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.166+0000 7f2b0da00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2b08156790 0x7f2b080c9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:02.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.166+0000 7f2b0da00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b080c9960 0x7f2b08166dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:02.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.166+0000 7f2b0da00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b080c54f0 con 0x7f2b08068990 2024-05-27T09:37:02.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.166+0000 7f2b0da00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f2b080c52a0 con 0x7f2b08156790 2024-05-27T09:37:02.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.166+0000 7f2b0da00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f2b08156ba0 con 0x7f2b080c9960 2024-05-27T09:37:02.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.166+0000 7f2b07e00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f2b08068990 0x7f2b08074360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:42522/0 (socket says 172.21.15.114:42522) 2024-05-27T09:37:02.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.166+0000 7f2b07e00640 1 -- 172.21.15.114:0/3757312313 learned_addr learned my addr 172.21.15.114:0/3757312313 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:02.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.167+0000 7f2aff400640 1 --2- 172.21.15.114:0/3757312313 >> v2:172.21.15.163:3300/0 conn(0x7f2b08156790 0x7f2b080c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:02.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.167+0000 7f2b0d000640 1 --2- 172.21.15.114:0/3757312313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b080c9960 0x7f2b08166dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:02.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.167+0000 7f2b07400640 1 -- 172.21.15.114:0/3757312313 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2275516161 0 0) 0x7f2b080c54f0 con 0x7f2b08068990 2024-05-27T09:37:02.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.167+0000 7f2b07400640 1 -- 172.21.15.114:0/3757312313 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2aec003520 con 0x7f2b08068990 2024-05-27T09:37:02.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.167+0000 7f2b0d000640 1 -- 172.21.15.114:0/3757312313 >> v2:172.21.15.163:3300/0 conn(0x7f2b08156790 msgr2=0x7f2b080c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:02.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.167+0000 7f2b0d000640 1 --2- 172.21.15.114:0/3757312313 >> v2:172.21.15.163:3300/0 conn(0x7f2b08156790 0x7f2b080c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:02.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.167+0000 7f2b0d000640 1 -- 172.21.15.114:0/3757312313 >> v1:172.21.15.114:6789/0 conn(0x7f2b08068990 legacy=0x7f2b08074360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:02.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.167+0000 7f2b0d000640 1 -- 172.21.15.114:0/3757312313 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b080ca0d0 con 0x7f2b080c9960 2024-05-27T09:37:02.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.167+0000 7f2aff400640 1 --2- 172.21.15.114:0/3757312313 >> v2:172.21.15.163:3300/0 conn(0x7f2b08156790 0x7f2b080c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:02.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.168+0000 7f2b0d000640 1 --2- 172.21.15.114:0/3757312313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b080c9960 0x7f2b08166dc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009d90 tx=0x7f2b0002a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0bca6df5ff001b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:02.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.168+0000 7f2b07400640 1 -- 172.21.15.114:0/3757312313 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b00007740 con 0x7f2b080c9960 2024-05-27T09:37:02.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.169+0000 7f2b07400640 1 -- 172.21.15.114:0/3757312313 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b000078e0 con 0x7f2b080c9960 2024-05-27T09:37:02.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.169+0000 7f2b07400640 1 -- 172.21.15.114:0/3757312313 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b00007cb0 con 0x7f2b080c9960 2024-05-27T09:37:02.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.169+0000 7f2b0da00640 1 -- 172.21.15.114:0/3757312313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b080c9960 msgr2=0x7f2b08166dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:02.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.169+0000 7f2b0da00640 1 --2- 172.21.15.114:0/3757312313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b080c9960 0x7f2b08166dc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009d90 tx=0x7f2b0002a530 comp rx=0 tx=0).stop 2024-05-27T09:37:02.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.169+0000 7f2b0da00640 1 -- 172.21.15.114:0/3757312313 shutdown_connections 2024-05-27T09:37:02.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.169+0000 7f2b0da00640 1 --2- 172.21.15.114:0/3757312313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b080c9960 0x7f2b08166dc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:02.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.169+0000 7f2b0da00640 1 --2- 172.21.15.114:0/3757312313 >> v2:172.21.15.163:3300/0 conn(0x7f2b08156790 0x7f2b080c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:02.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.169+0000 7f2b0da00640 1 -- 172.21.15.114:0/3757312313 >> 172.21.15.114:0/3757312313 conn(0x7f2b08073840 msgr2=0x7f2b08073c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:02.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.169+0000 7f2b0da00640 1 -- 172.21.15.114:0/3757312313 shutdown_connections 2024-05-27T09:37:02.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.169+0000 7f2b0da00640 1 -- 172.21.15.114:0/3757312313 wait complete. 2024-05-27T09:37:02.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.172+0000 7f2b0da00640 1 Processor -- start 2024-05-27T09:37:02.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.172+0000 7f2b0da00640 1 -- start start 2024-05-27T09:37:02.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.173+0000 7f2b0da00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b08068990 0x7f2b08146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:02.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.173+0000 7f2b0da00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b080c9960 0x7f2b081465b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:02.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.173+0000 7f2b0da00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b08156790 0x7f2b0815fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:02.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.173+0000 7f2b0da00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2b080c6900 con 0x7f2b08068990 2024-05-27T09:37:02.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.173+0000 7f2b0da00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0806df10 con 0x7f2b080c9960 2024-05-27T09:37:02.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.173+0000 7f2b0da00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f2b080c8140 con 0x7f2b08156790 2024-05-27T09:37:02.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.173+0000 7f2b07e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b08068990 0x7f2b08146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:02.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.174+0000 7f2aff400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b080c9960 0x7f2b081465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:02.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.174+0000 7f2b0d000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b08156790 0x7f2b0815fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:02.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.174+0000 7f2aff400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b080c9960 0x7f2b081465b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:51394/0 (socket says 172.21.15.114:51394) 2024-05-27T09:37:02.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.174+0000 7f2aff400640 1 -- 172.21.15.114:0/1111646422 learned_addr learned my addr 172.21.15.114:0/1111646422 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:02.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.174+0000 7f2aff400640 1 -- 172.21.15.114:0/1111646422 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b08156790 msgr2=0x7f2b0815fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:02.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.174+0000 7f2aff400640 1 --2- 172.21.15.114:0/1111646422 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b08156790 0x7f2b0815fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:02.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.174+0000 7f2aff400640 1 -- 172.21.15.114:0/1111646422 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b08068990 msgr2=0x7f2b08146f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:02.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.174+0000 7f2aff400640 1 --2- 172.21.15.114:0/1111646422 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b08068990 0x7f2b08146f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:02.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.174+0000 7f2aff400640 1 -- 172.21.15.114:0/1111646422 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b08155bd0 con 0x7f2b080c9960 2024-05-27T09:37:02.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.174+0000 7f2b07e00640 1 --2- 172.21.15.114:0/1111646422 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b08068990 0x7f2b08146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:02.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.174+0000 7f2aff400640 1 --2- 172.21.15.114:0/1111646422 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b080c9960 0x7f2b081465b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2af000e900 tx=0x7f2af000edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:02.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.175+0000 7f2b04c00640 1 -- 172.21.15.114:0/1111646422 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0017070 con 0x7f2b080c9960 2024-05-27T09:37:02.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.175+0000 7f2b04c00640 1 -- 172.21.15.114:0/1111646422 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af000c9a0 con 0x7f2b080c9960 2024-05-27T09:37:02.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.175+0000 7f2b0da00640 1 -- 172.21.15.114:0/1111646422 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b081f2310 con 0x7f2b080c9960 2024-05-27T09:37:02.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.175+0000 7f2b04c00640 1 -- 172.21.15.114:0/1111646422 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af000cd90 con 0x7f2b080c9960 2024-05-27T09:37:02.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.175+0000 7f2b0da00640 1 -- 172.21.15.114:0/1111646422 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b08154760 con 0x7f2b080c9960 2024-05-27T09:37:02.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.176+0000 7f2b0da00640 1 -- 172.21.15.114:0/1111646422 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b08050860 con 0x7f2b080c9960 2024-05-27T09:37:02.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.177+0000 7f2b04c00640 1 -- 172.21.15.114:0/1111646422 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2af001a070 con 0x7f2b080c9960 2024-05-27T09:37:02.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.178+0000 7f2b04c00640 1 --2- 172.21.15.114:0/1111646422 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2ae4076300 0x7f2ae4078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:02.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.179+0000 7f2b04c00640 1 -- 172.21.15.114:0/1111646422 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6168+0+0 (secure 0 0 0) 0x7f2af0097900 con 0x7f2b080c9960 2024-05-27T09:37:02.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.179+0000 7f2b07e00640 1 --2- 172.21.15.114:0/1111646422 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2ae4076300 0x7f2ae4078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:02.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.179+0000 7f2b07e00640 1 --2- 172.21.15.114:0/1111646422 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2ae4076300 0x7f2ae4078810 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2af8007a60 tx=0x7f2af80079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:02.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.182+0000 7f2b04c00640 1 -- 172.21.15.114:0/1111646422 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2b081f2310 con 0x7f2b080c9960 2024-05-27T09:37:02.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.428+0000 7f2b0da00640 1 -- 172.21.15.114:0/1111646422 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b080c7d10 con 0x7f2b080c9960 2024-05-27T09:37:02.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.429+0000 7f2b04c00640 1 -- 172.21.15.114:0/1111646422 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+6890 (secure 0 0 0) 0x7f2af0060ab0 con 0x7f2b080c9960 2024-05-27T09:37:02.431 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:37:02.431 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5072,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/1057092553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":1057092553},{"type":"v1","addr":"172.21.15.163:6839","nonce":1057092553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:36:01.014181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:active","state_seq":26,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5160,5086,4875]},"id":1}]} 2024-05-27T09:37:02.431 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 42 2024-05-27T09:37:02.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.432+0000 7f2b0da00640 1 -- 172.21.15.114:0/1111646422 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2ae4076300 msgr2=0x7f2ae4078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:02.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.432+0000 7f2b0da00640 1 --2- 172.21.15.114:0/1111646422 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2ae4076300 0x7f2ae4078810 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2af8007a60 tx=0x7f2af80079b0 comp rx=0 tx=0).stop 2024-05-27T09:37:02.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.432+0000 7f2b0da00640 1 -- 172.21.15.114:0/1111646422 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b080c9960 msgr2=0x7f2b081465b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:02.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.432+0000 7f2b0da00640 1 --2- 172.21.15.114:0/1111646422 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b080c9960 0x7f2b081465b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2af000e900 tx=0x7f2af000edc0 comp rx=0 tx=0).stop 2024-05-27T09:37:02.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.432+0000 7f2b0da00640 1 -- 172.21.15.114:0/1111646422 shutdown_connections 2024-05-27T09:37:02.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.432+0000 7f2b0da00640 1 --2- 172.21.15.114:0/1111646422 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2ae4076300 0x7f2ae4078810 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:02.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.432+0000 7f2b0da00640 1 --2- 172.21.15.114:0/1111646422 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b08156790 0x7f2b0815fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:02.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.432+0000 7f2b0da00640 1 --2- 172.21.15.114:0/1111646422 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b080c9960 0x7f2b081465b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:02.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.432+0000 7f2b0da00640 1 --2- 172.21.15.114:0/1111646422 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2b08068990 0x7f2b08146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:02.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.432+0000 7f2b0da00640 1 -- 172.21.15.114:0/1111646422 >> 172.21.15.114:0/1111646422 conn(0x7f2b08073840 msgr2=0x7f2b0814bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:02.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.432+0000 7f2b0da00640 1 -- 172.21.15.114:0/1111646422 shutdown_connections 2024-05-27T09:37:02.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:02.433+0000 7f2b0da00640 1 -- 172.21.15.114:0/1111646422 wait complete. 2024-05-27T09:37:02.460 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-05-27T09:37:02.660 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi114 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6507388dd5057528934822c0163b0c347ef1d5d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh' 2024-05-27T09:37:02.907 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=9e7a4c8b869448ea9bdb01b3774aa0ae Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi114 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6507388dd5057528934822c0163b0c347ef1d5d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh' 2024-05-27T09:37:02.910 DEBUG:teuthology.run_tasks:Unwinding manager mds_thrash 2024-05-27T09:37:02.920 INFO:tasks.mds_thrash:joining mds_thrasher 2024-05-27T09:37:04.461 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:37:04.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.739+0000 7f2544400640 1 Processor -- start 2024-05-27T09:37:04.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.739+0000 7f2544400640 1 -- start start 2024-05-27T09:37:04.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.741+0000 7f2544400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f253c0c3820 0x7f253c073310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:04.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.741+0000 7f2544400640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f253c1615d0 con 0x7f253c074860 2024-05-27T09:37:04.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.741+0000 7f2544400640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f253c161890 con 0x7f253c164db0 2024-05-27T09:37:04.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.741+0000 7f2544400640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f253c0cade0 con 0x7f253c0c3820 2024-05-27T09:37:04.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.742+0000 7f2542400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f253c0c3820 0x7f253c073310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:04.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.742+0000 7f2541a00640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f253c164db0 0x7f253c15aac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:38886/0 (socket says 172.21.15.114:38886) 2024-05-27T09:37:04.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.742+0000 7f2541a00640 1 -- 172.21.15.114:0/2698271902 learned_addr learned my addr 172.21.15.114:0/2698271902 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:04.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.742+0000 7f253be00640 1 -- 172.21.15.114:0/2698271902 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3346762697 0 0) 0x7f253c161890 con 0x7f253c164db0 2024-05-27T09:37:04.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.742+0000 7f2542400640 1 -- 172.21.15.114:0/2698271902 >> v1:172.21.15.163:6789/0 conn(0x7f253c164db0 legacy=0x7f253c15aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:04.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.742+0000 7f2542400640 1 -- 172.21.15.114:0/2698271902 >> v1:172.21.15.114:6789/0 conn(0x7f253c074860 legacy=0x7f253c160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:04.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.742+0000 7f2542400640 1 -- 172.21.15.114:0/2698271902 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253c1642d0 con 0x7f253c0c3820 2024-05-27T09:37:04.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.743+0000 7f2542400640 1 --2- 172.21.15.114:0/2698271902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f253c0c3820 0x7f253c073310 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2534009d90 tx=0x7f253402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b08bab1a8101f5f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:04.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.743+0000 7f253be00640 1 -- 172.21.15.114:0/2698271902 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2534007740 con 0x7f253c0c3820 2024-05-27T09:37:04.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.744+0000 7f253be00640 1 -- 172.21.15.114:0/2698271902 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25340078e0 con 0x7f253c0c3820 2024-05-27T09:37:04.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.744+0000 7f253be00640 1 -- 172.21.15.114:0/2698271902 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2534007cb0 con 0x7f253c0c3820 2024-05-27T09:37:04.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.744+0000 7f2544400640 1 -- 172.21.15.114:0/2698271902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f253c0c3820 msgr2=0x7f253c073310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:04.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.744+0000 7f2544400640 1 --2- 172.21.15.114:0/2698271902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f253c0c3820 0x7f253c073310 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2534009d90 tx=0x7f253402a5f0 comp rx=0 tx=0).stop 2024-05-27T09:37:04.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.744+0000 7f2544400640 1 -- 172.21.15.114:0/2698271902 shutdown_connections 2024-05-27T09:37:04.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.744+0000 7f2544400640 1 --2- 172.21.15.114:0/2698271902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f253c0c3820 0x7f253c073310 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:04.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.744+0000 7f2544400640 1 -- 172.21.15.114:0/2698271902 >> 172.21.15.114:0/2698271902 conn(0x7f253c06f6e0 msgr2=0x7f253c070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:04.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.745+0000 7f2544400640 1 -- 172.21.15.114:0/2698271902 shutdown_connections 2024-05-27T09:37:04.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.745+0000 7f2544400640 1 -- 172.21.15.114:0/2698271902 wait complete. 2024-05-27T09:37:04.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.747+0000 7f2544400640 1 Processor -- start 2024-05-27T09:37:04.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.747+0000 7f2544400640 1 -- start start 2024-05-27T09:37:04.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.748+0000 7f2544400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f253c074860 0x7f253c162fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:04.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.748+0000 7f2544400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f253c0c3820 0x7f253c1582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:04.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.748+0000 7f2544400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f253c164db0 0x7f253c1f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:04.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.748+0000 7f2544400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f253c161250 con 0x7f253c164db0 2024-05-27T09:37:04.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.748+0000 7f2544400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f253c164890 con 0x7f253c0c3820 2024-05-27T09:37:04.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.748+0000 7f2544400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f253c0c3c30 con 0x7f253c074860 2024-05-27T09:37:04.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.749+0000 7f2541000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f253c0c3820 0x7f253c1582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:04.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.749+0000 7f2541a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f253c074860 0x7f253c162fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:04.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.749+0000 7f2542400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f253c164db0 0x7f253c1f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:04.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.749+0000 7f2541000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f253c0c3820 0x7f253c1582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:51396/0 (socket says 172.21.15.114:51396) 2024-05-27T09:37:04.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.749+0000 7f2541000640 1 -- 172.21.15.114:0/1221610206 learned_addr learned my addr 172.21.15.114:0/1221610206 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:04.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.749+0000 7f2541000640 1 -- 172.21.15.114:0/1221610206 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f253c074860 msgr2=0x7f253c162fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:04.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.749+0000 7f2541000640 1 --2- 172.21.15.114:0/1221610206 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f253c074860 0x7f253c162fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:04.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.749+0000 7f2541000640 1 -- 172.21.15.114:0/1221610206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f253c164db0 msgr2=0x7f253c1f6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:04.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.749+0000 7f2541000640 1 --2- 172.21.15.114:0/1221610206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f253c164db0 0x7f253c1f6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:04.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.749+0000 7f2541000640 1 -- 172.21.15.114:0/1221610206 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253c1f7260 con 0x7f253c0c3820 2024-05-27T09:37:04.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.749+0000 7f2541a00640 1 --2- 172.21.15.114:0/1221610206 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f253c074860 0x7f253c162fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:04.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.749+0000 7f2542400640 1 --2- 172.21.15.114:0/1221610206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f253c164db0 0x7f253c1f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:04.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.750+0000 7f2541000640 1 --2- 172.21.15.114:0/1221610206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f253c0c3820 0x7f253c1582a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f252800c580 tx=0x7f252800ca40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:04.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.750+0000 7f2539600640 1 -- 172.21.15.114:0/1221610206 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528017070 con 0x7f253c0c3820 2024-05-27T09:37:04.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.750+0000 7f2539600640 1 -- 172.21.15.114:0/1221610206 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25280035f0 con 0x7f253c0c3820 2024-05-27T09:37:04.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.750+0000 7f2539600640 1 -- 172.21.15.114:0/1221610206 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25280039e0 con 0x7f253c0c3820 2024-05-27T09:37:04.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.750+0000 7f2544400640 1 -- 172.21.15.114:0/1221610206 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f253c163a30 con 0x7f253c0c3820 2024-05-27T09:37:04.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.750+0000 7f2544400640 1 -- 172.21.15.114:0/1221610206 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f253c157c50 con 0x7f253c0c3820 2024-05-27T09:37:04.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.751+0000 7f2544400640 1 -- 172.21.15.114:0/1221610206 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f253c06a380 con 0x7f253c0c3820 2024-05-27T09:37:04.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.752+0000 7f2539600640 1 -- 172.21.15.114:0/1221610206 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f252801a070 con 0x7f253c0c3820 2024-05-27T09:37:04.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.753+0000 7f2539600640 1 --2- 172.21.15.114:0/1221610206 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f251c076300 0x7f251c078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:04.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.754+0000 7f2539600640 1 -- 172.21.15.114:0/1221610206 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6168+0+0 (secure 0 0 0) 0x7f2528096bc0 con 0x7f253c0c3820 2024-05-27T09:37:04.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.754+0000 7f2541a00640 1 --2- 172.21.15.114:0/1221610206 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f251c076300 0x7f251c078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:04.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.754+0000 7f2541a00640 1 --2- 172.21.15.114:0/1221610206 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f251c076300 0x7f251c078810 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2530000f60 tx=0x7f2530007e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:04.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:04.758+0000 7f2539600640 1 -- 172.21.15.114:0/1221610206 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f252805fcf0 con 0x7f253c0c3820 2024-05-27T09:37:05.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:05.005+0000 7f2544400640 1 -- 172.21.15.114:0/1221610206 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f253c073e30 con 0x7f253c0c3820 2024-05-27T09:37:05.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:05.006+0000 7f2539600640 1 -- 172.21.15.114:0/1221610206 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+6890 (secure 0 0 0) 0x7f2528068330 con 0x7f253c0c3820 2024-05-27T09:37:05.008 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:37:05.008 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5072,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/1057092553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":1057092553},{"type":"v1","addr":"172.21.15.163:6839","nonce":1057092553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:36:01.014181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:active","state_seq":26,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5160,5086,4875]},"id":1}]} 2024-05-27T09:37:05.008 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 42 2024-05-27T09:37:05.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:05.009+0000 7f2544400640 1 -- 172.21.15.114:0/1221610206 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f251c076300 msgr2=0x7f251c078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:05.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:05.009+0000 7f2544400640 1 --2- 172.21.15.114:0/1221610206 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f251c076300 0x7f251c078810 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2530000f60 tx=0x7f2530007e40 comp rx=0 tx=0).stop 2024-05-27T09:37:05.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:05.009+0000 7f2544400640 1 -- 172.21.15.114:0/1221610206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f253c0c3820 msgr2=0x7f253c1582a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:05.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:05.009+0000 7f2544400640 1 --2- 172.21.15.114:0/1221610206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f253c0c3820 0x7f253c1582a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f252800c580 tx=0x7f252800ca40 comp rx=0 tx=0).stop 2024-05-27T09:37:05.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:05.009+0000 7f2544400640 1 -- 172.21.15.114:0/1221610206 shutdown_connections 2024-05-27T09:37:05.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:05.009+0000 7f2544400640 1 --2- 172.21.15.114:0/1221610206 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f251c076300 0x7f251c078810 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:05.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:05.009+0000 7f2544400640 1 --2- 172.21.15.114:0/1221610206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f253c164db0 0x7f253c1f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:05.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:05.009+0000 7f2544400640 1 --2- 172.21.15.114:0/1221610206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f253c0c3820 0x7f253c1582a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:05.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:05.009+0000 7f2544400640 1 --2- 172.21.15.114:0/1221610206 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f253c074860 0x7f253c162fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:05.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:05.009+0000 7f2544400640 1 -- 172.21.15.114:0/1221610206 >> 172.21.15.114:0/1221610206 conn(0x7f253c06f6e0 msgr2=0x7f253c0c8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:05.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:05.009+0000 7f2544400640 1 -- 172.21.15.114:0/1221610206 shutdown_connections 2024-05-27T09:37:05.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:05.009+0000 7f2544400640 1 -- 172.21.15.114:0/1221610206 wait complete. 2024-05-27T09:37:05.040 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-05-27T09:37:07.041 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:37:07.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.323+0000 7f9da3e00640 1 Processor -- start 2024-05-27T09:37:07.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.324+0000 7f9da3e00640 1 -- start start 2024-05-27T09:37:07.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.326+0000 7f9da3e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d9c154820 0x7f9d9c0ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:07.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.326+0000 7f9da3e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f9d9c0ca810 0x7f9d9c15dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:07.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.326+0000 7f9da3e00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d9c0cac20 con 0x7f9d9c074a60 2024-05-27T09:37:07.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.326+0000 7f9da3e00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f9d9c154c30 con 0x7f9d9c0ca810 2024-05-27T09:37:07.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.326+0000 7f9da3e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f9d9c15db60 con 0x7f9d9c154820 2024-05-27T09:37:07.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.326+0000 7f9da1400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f9d9c074a60 0x7f9d9c073260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:39332/0 (socket says 172.21.15.114:39332) 2024-05-27T09:37:07.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.326+0000 7f9da1400640 1 -- 172.21.15.114:0/1990256305 learned_addr learned my addr 172.21.15.114:0/1990256305 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:07.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.326+0000 7f9da1e00640 1 --2- 172.21.15.114:0/1990256305 >> v2:172.21.15.114:3300/0 conn(0x7f9d9c0ca810 0x7f9d9c15dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:07.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.326+0000 7f9da0a00640 1 --2- 172.21.15.114:0/1990256305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d9c154820 0x7f9d9c0ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:07.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.327+0000 7f9da1e00640 1 -- 172.21.15.114:0/1990256305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d9c154820 msgr2=0x7f9d9c0ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:07.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.327+0000 7f9da1e00640 1 --2- 172.21.15.114:0/1990256305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d9c154820 0x7f9d9c0ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:07.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.327+0000 7f9da1e00640 1 -- 172.21.15.114:0/1990256305 >> v1:172.21.15.114:6789/0 conn(0x7f9d9c074a60 legacy=0x7f9d9c073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:07.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.327+0000 7f9da1e00640 1 -- 172.21.15.114:0/1990256305 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9c15a810 con 0x7f9d9c0ca810 2024-05-27T09:37:07.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.327+0000 7f9d9be00640 1 -- 172.21.15.114:0/1990256305 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2680260620 0 0) 0x7f9d9c0cac20 con 0x7f9d9c074a60 2024-05-27T09:37:07.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.328+0000 7f9da1e00640 1 --2- 172.21.15.114:0/1990256305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d9c0ca810 0x7f9d9c15dff0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9d940097a0 tx=0x7f9d9402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2f348a198b2dc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:07.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.328+0000 7f9d9be00640 1 -- 172.21.15.114:0/1990256305 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94037070 con 0x7f9d9c0ca810 2024-05-27T09:37:07.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.328+0000 7f9d9be00640 1 -- 172.21.15.114:0/1990256305 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d94032440 con 0x7f9d9c0ca810 2024-05-27T09:37:07.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.328+0000 7f9d9be00640 1 -- 172.21.15.114:0/1990256305 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94032810 con 0x7f9d9c0ca810 2024-05-27T09:37:07.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.329+0000 7f9da3e00640 1 -- 172.21.15.114:0/1990256305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d9c0ca810 msgr2=0x7f9d9c15dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:07.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.329+0000 7f9da3e00640 1 --2- 172.21.15.114:0/1990256305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d9c0ca810 0x7f9d9c15dff0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9d940097a0 tx=0x7f9d9402a210 comp rx=0 tx=0).stop 2024-05-27T09:37:07.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.329+0000 7f9da3e00640 1 -- 172.21.15.114:0/1990256305 shutdown_connections 2024-05-27T09:37:07.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.329+0000 7f9da3e00640 1 --2- 172.21.15.114:0/1990256305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d9c0ca810 0x7f9d9c15dff0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:07.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.329+0000 7f9da3e00640 1 --2- 172.21.15.114:0/1990256305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d9c154820 0x7f9d9c0ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:07.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.329+0000 7f9da3e00640 1 -- 172.21.15.114:0/1990256305 >> 172.21.15.114:0/1990256305 conn(0x7f9d9c06f6e0 msgr2=0x7f9d9c070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:07.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.329+0000 7f9da3e00640 1 -- 172.21.15.114:0/1990256305 shutdown_connections 2024-05-27T09:37:07.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.329+0000 7f9da3e00640 1 -- 172.21.15.114:0/1990256305 wait complete. 2024-05-27T09:37:07.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.331+0000 7f9da3e00640 1 Processor -- start 2024-05-27T09:37:07.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.332+0000 7f9da3e00640 1 -- start start 2024-05-27T09:37:07.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.333+0000 7f9da3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d9c074a60 0x7f9d9c1e8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:07.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.333+0000 7f9da3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d9c0ca810 0x7f9d9c1e8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:07.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.333+0000 7f9da3e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d9c154820 0x7f9d9c149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:07.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.333+0000 7f9da3e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c15b0b0 con 0x7f9d9c0ca810 2024-05-27T09:37:07.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.333+0000 7f9da3e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c15add0 con 0x7f9d9c074a60 2024-05-27T09:37:07.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.333+0000 7f9da3e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f9d9c15b390 con 0x7f9d9c154820 2024-05-27T09:37:07.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.333+0000 7f9da0a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d9c0ca810 0x7f9d9c1e8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:07.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.333+0000 7f9da0a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d9c0ca810 0x7f9d9c1e8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48236/0 (socket says 172.21.15.114:48236) 2024-05-27T09:37:07.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.333+0000 7f9da0a00640 1 -- 172.21.15.114:0/646358371 learned_addr learned my addr 172.21.15.114:0/646358371 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:07.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.333+0000 7f9da1e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d9c154820 0x7f9d9c149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:07.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.333+0000 7f9da1400640 1 --2- 172.21.15.114:0/646358371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d9c074a60 0x7f9d9c1e8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:07.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.334+0000 7f9da1e00640 1 -- 172.21.15.114:0/646358371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d9c074a60 msgr2=0x7f9d9c1e8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:07.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.334+0000 7f9da1e00640 1 --2- 172.21.15.114:0/646358371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d9c074a60 0x7f9d9c1e8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:07.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.334+0000 7f9da1e00640 1 -- 172.21.15.114:0/646358371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d9c0ca810 msgr2=0x7f9d9c1e8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:07.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.334+0000 7f9da1e00640 1 --2- 172.21.15.114:0/646358371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d9c0ca810 0x7f9d9c1e8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:07.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.334+0000 7f9da1e00640 1 -- 172.21.15.114:0/646358371 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9c159e40 con 0x7f9d9c154820 2024-05-27T09:37:07.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.334+0000 7f9da0a00640 1 --2- 172.21.15.114:0/646358371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d9c0ca810 0x7f9d9c1e8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:07.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.334+0000 7f9da1e00640 1 --2- 172.21.15.114:0/646358371 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d9c154820 0x7f9d9c149f50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9d94031000 tx=0x7f9d940046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:07.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.334+0000 7f9d99600640 1 -- 172.21.15.114:0/646358371 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94041030 con 0x7f9d9c154820 2024-05-27T09:37:07.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.335+0000 7f9d99600640 1 -- 172.21.15.114:0/646358371 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d9403d4e0 con 0x7f9d9c154820 2024-05-27T09:37:07.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.335+0000 7f9d99600640 1 -- 172.21.15.114:0/646358371 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94037040 con 0x7f9d9c154820 2024-05-27T09:37:07.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.335+0000 7f9da3e00640 1 -- 172.21.15.114:0/646358371 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d9c0c7570 con 0x7f9d9c154820 2024-05-27T09:37:07.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.335+0000 7f9da3e00640 1 -- 172.21.15.114:0/646358371 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d9c0cd8c0 con 0x7f9d9c154820 2024-05-27T09:37:07.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.336+0000 7f9da3e00640 1 -- 172.21.15.114:0/646358371 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d9c06a380 con 0x7f9d9c154820 2024-05-27T09:37:07.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.337+0000 7f9d99600640 1 -- 172.21.15.114:0/646358371 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9d94044070 con 0x7f9d9c154820 2024-05-27T09:37:07.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.338+0000 7f9d99600640 1 --2- 172.21.15.114:0/646358371 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9d7c076300 0x7f9d7c078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:07.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.338+0000 7f9d99600640 1 -- 172.21.15.114:0/646358371 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(49..49 src has 1..49) ==== 6168+0+0 (secure 0 0 0) 0x7f9d940b7d30 con 0x7f9d9c154820 2024-05-27T09:37:07.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.338+0000 7f9da1400640 1 --2- 172.21.15.114:0/646358371 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9d7c076300 0x7f9d7c078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:07.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.339+0000 7f9da1400640 1 --2- 172.21.15.114:0/646358371 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9d7c076300 0x7f9d7c078810 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c007fd0 tx=0x7f9d8c007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:07.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.343+0000 7f9d99600640 1 -- 172.21.15.114:0/646358371 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9d94080e50 con 0x7f9d9c154820 2024-05-27T09:37:07.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.590+0000 7f9da3e00640 1 -- 172.21.15.114:0/646358371 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d9c0ca5f0 con 0x7f9d9c154820 2024-05-27T09:37:07.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.591+0000 7f9d99600640 1 -- 172.21.15.114:0/646358371 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+6890 (secure 0 0 0) 0x7f9d94085150 con 0x7f9d9c154820 2024-05-27T09:37:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:37:07.593 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5072,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/1057092553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":1057092553},{"type":"v1","addr":"172.21.15.163:6839","nonce":1057092553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:36:01.014181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:active","state_seq":26,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5160,5086,4875]},"id":1}]} 2024-05-27T09:37:07.593 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 42 2024-05-27T09:37:07.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.593+0000 7f9da3e00640 1 -- 172.21.15.114:0/646358371 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9d7c076300 msgr2=0x7f9d7c078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:07.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.594+0000 7f9da3e00640 1 --2- 172.21.15.114:0/646358371 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9d7c076300 0x7f9d7c078810 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c007fd0 tx=0x7f9d8c007de0 comp rx=0 tx=0).stop 2024-05-27T09:37:07.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.594+0000 7f9da3e00640 1 -- 172.21.15.114:0/646358371 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d9c154820 msgr2=0x7f9d9c149f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.594+0000 7f9da3e00640 1 --2- 172.21.15.114:0/646358371 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d9c154820 0x7f9d9c149f50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9d94031000 tx=0x7f9d940046c0 comp rx=0 tx=0).stop 2024-05-27T09:37:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.594+0000 7f9da3e00640 1 -- 172.21.15.114:0/646358371 shutdown_connections 2024-05-27T09:37:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.594+0000 7f9da3e00640 1 --2- 172.21.15.114:0/646358371 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f9d7c076300 0x7f9d7c078810 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.594+0000 7f9da3e00640 1 --2- 172.21.15.114:0/646358371 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d9c154820 0x7f9d9c149f50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.594+0000 7f9da3e00640 1 --2- 172.21.15.114:0/646358371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9d9c0ca810 0x7f9d9c1e8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.594+0000 7f9da3e00640 1 --2- 172.21.15.114:0/646358371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d9c074a60 0x7f9d9c1e8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.594+0000 7f9da3e00640 1 -- 172.21.15.114:0/646358371 >> 172.21.15.114:0/646358371 conn(0x7f9d9c06f6e0 msgr2=0x7f9d9c1516e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.594+0000 7f9da3e00640 1 -- 172.21.15.114:0/646358371 shutdown_connections 2024-05-27T09:37:07.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:07.594+0000 7f9da3e00640 1 -- 172.21.15.114:0/646358371 wait complete. 2024-05-27T09:37:07.621 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-05-27T09:37:09.623 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:37:09.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.902+0000 7fa9a1c00640 1 Processor -- start 2024-05-27T09:37:09.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.902+0000 7fa9a1c00640 1 -- start start 2024-05-27T09:37:09.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.904+0000 7fa9a1c00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa99c06f600 0x7fa99c06fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:09.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.904+0000 7fa9a1c00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa99c074660 con 0x7fa99c0b34a0 2024-05-27T09:37:09.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.904+0000 7fa9a1c00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa99c074940 con 0x7fa99c0c8580 2024-05-27T09:37:09.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.904+0000 7fa9a1c00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fa99c159950 con 0x7fa99c06f600 2024-05-27T09:37:09.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.905+0000 7fa99be00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fa99c0b34a0 0x7fa99c159240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:39336/0 (socket says 172.21.15.114:39336) 2024-05-27T09:37:09.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.905+0000 7fa99be00640 1 -- 172.21.15.114:0/774879441 learned_addr learned my addr 172.21.15.114:0/774879441 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:09.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.905+0000 7fa9a1200640 1 --2- 172.21.15.114:0/774879441 >> v2:172.21.15.163:3300/0 conn(0x7fa99c06f600 0x7fa99c06fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:09.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.905+0000 7fa99aa00640 1 -- 172.21.15.114:0/774879441 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4208734928 0 0) 0x7fa99c074660 con 0x7fa99c0b34a0 2024-05-27T09:37:09.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.905+0000 7fa99aa00640 1 -- 172.21.15.114:0/774879441 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa980003520 con 0x7fa99c0b34a0 2024-05-27T09:37:09.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.905+0000 7fa9a1200640 1 -- 172.21.15.114:0/774879441 >> v1:172.21.15.163:6789/0 conn(0x7fa99c0c8580 legacy=0x7fa99c06eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:09.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.905+0000 7fa9a1200640 1 -- 172.21.15.114:0/774879441 >> v1:172.21.15.114:6789/0 conn(0x7fa99c0b34a0 legacy=0x7fa99c159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:09.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.905+0000 7fa9a1200640 1 -- 172.21.15.114:0/774879441 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99c158a40 con 0x7fa99c06f600 2024-05-27T09:37:09.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.906+0000 7fa9a1200640 1 --2- 172.21.15.114:0/774879441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa99c06f600 0x7fa99c06fa10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa994009d90 tx=0x7fa99402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=374f0a19ef04f58c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:09.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.907+0000 7fa99aa00640 1 -- 172.21.15.114:0/774879441 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa994007740 con 0x7fa99c06f600 2024-05-27T09:37:09.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.907+0000 7fa99aa00640 1 -- 172.21.15.114:0/774879441 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9940078e0 con 0x7fa99c06f600 2024-05-27T09:37:09.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.907+0000 7fa99aa00640 1 -- 172.21.15.114:0/774879441 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa994007cb0 con 0x7fa99c06f600 2024-05-27T09:37:09.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.907+0000 7fa9a1c00640 1 -- 172.21.15.114:0/774879441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa99c06f600 msgr2=0x7fa99c06fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:09.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.907+0000 7fa9a1c00640 1 --2- 172.21.15.114:0/774879441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa99c06f600 0x7fa99c06fa10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa994009d90 tx=0x7fa99402a660 comp rx=0 tx=0).stop 2024-05-27T09:37:09.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.908+0000 7fa9a1c00640 1 -- 172.21.15.114:0/774879441 shutdown_connections 2024-05-27T09:37:09.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.908+0000 7fa9a1c00640 1 --2- 172.21.15.114:0/774879441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa99c06f600 0x7fa99c06fa10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:09.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.908+0000 7fa9a1c00640 1 -- 172.21.15.114:0/774879441 >> 172.21.15.114:0/774879441 conn(0x7fa99c0ca320 msgr2=0x7fa99c0ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:09.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.908+0000 7fa9a1c00640 1 -- 172.21.15.114:0/774879441 shutdown_connections 2024-05-27T09:37:09.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.908+0000 7fa9a1c00640 1 -- 172.21.15.114:0/774879441 wait complete. 2024-05-27T09:37:09.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.910+0000 7fa9a1c00640 1 Processor -- start 2024-05-27T09:37:09.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.910+0000 7fa9a1c00640 1 -- start start 2024-05-27T09:37:09.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.911+0000 7fa9a1c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa99c06f600 0x7fa99c14c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:09.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.911+0000 7fa9a1c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa99c0b34a0 0x7fa99c1f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:09.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.911+0000 7fa9a1c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa99c0c8580 0x7fa99c146c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:09.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.911+0000 7fa9a1c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c0c3250 con 0x7fa99c0b34a0 2024-05-27T09:37:09.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.911+0000 7fa9a1c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c158db0 con 0x7fa99c06f600 2024-05-27T09:37:09.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.911+0000 7fa9a1c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa99c06ff50 con 0x7fa99c0c8580 2024-05-27T09:37:09.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.912+0000 7fa99b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa99c0b34a0 0x7fa99c1f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:09.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.912+0000 7fa99be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa99c06f600 0x7fa99c14c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:09.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.912+0000 7fa9a1200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa99c0c8580 0x7fa99c146c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:09.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.912+0000 7fa99be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa99c06f600 0x7fa99c14c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37276/0 (socket says 172.21.15.114:37276) 2024-05-27T09:37:09.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.912+0000 7fa99be00640 1 -- 172.21.15.114:0/837250168 learned_addr learned my addr 172.21.15.114:0/837250168 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:09.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.912+0000 7fa99be00640 1 -- 172.21.15.114:0/837250168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa99c0c8580 msgr2=0x7fa99c146c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:09.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.912+0000 7fa99be00640 1 --2- 172.21.15.114:0/837250168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa99c0c8580 0x7fa99c146c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:09.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.912+0000 7fa99be00640 1 -- 172.21.15.114:0/837250168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa99c0b34a0 msgr2=0x7fa99c1f4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:09.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.912+0000 7fa99be00640 1 --2- 172.21.15.114:0/837250168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa99c0b34a0 0x7fa99c1f4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:09.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.912+0000 7fa99be00640 1 -- 172.21.15.114:0/837250168 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99c149f60 con 0x7fa99c06f600 2024-05-27T09:37:09.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.913+0000 7fa99be00640 1 --2- 172.21.15.114:0/837250168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa99c06f600 0x7fa99c14c370 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00c3c0 tx=0x7fa98c00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:09.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.913+0000 7fa993e00640 1 -- 172.21.15.114:0/837250168 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c017070 con 0x7fa99c06f600 2024-05-27T09:37:09.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.913+0000 7fa993e00640 1 -- 172.21.15.114:0/837250168 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa98c005d00 con 0x7fa99c06f600 2024-05-27T09:37:09.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.913+0000 7fa9a1c00640 1 -- 172.21.15.114:0/837250168 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa99c14a240 con 0x7fa99c06f600 2024-05-27T09:37:09.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.913+0000 7fa993e00640 1 -- 172.21.15.114:0/837250168 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c007310 con 0x7fa99c06f600 2024-05-27T09:37:09.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.913+0000 7fa9a1c00640 1 -- 172.21.15.114:0/837250168 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa99c1f4910 con 0x7fa99c06f600 2024-05-27T09:37:09.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.914+0000 7fa9a1c00640 1 -- 172.21.15.114:0/837250168 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa99c050860 con 0x7fa99c06f600 2024-05-27T09:37:09.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.915+0000 7fa993e00640 1 -- 172.21.15.114:0/837250168 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa98c01a070 con 0x7fa99c06f600 2024-05-27T09:37:09.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.916+0000 7fa993e00640 1 --2- 172.21.15.114:0/837250168 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa97c0762d0 0x7fa97c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:09.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.917+0000 7fa993e00640 1 -- 172.21.15.114:0/837250168 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6168+0+0 (secure 0 0 0) 0x7fa98c096ae0 con 0x7fa99c06f600 2024-05-27T09:37:09.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.917+0000 7fa99b400640 1 --2- 172.21.15.114:0/837250168 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa97c0762d0 0x7fa97c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:09.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.917+0000 7fa99b400640 1 --2- 172.21.15.114:0/837250168 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa97c0762d0 0x7fa97c0787e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa99c1e6180 tx=0x7fa988009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:09.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:09.921+0000 7fa993e00640 1 -- 172.21.15.114:0/837250168 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa98c05fb80 con 0x7fa99c06f600 2024-05-27T09:37:10.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:10.173+0000 7fa9a1c00640 1 -- 172.21.15.114:0/837250168 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa99c152460 con 0x7fa99c06f600 2024-05-27T09:37:10.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:10.174+0000 7fa993e00640 1 -- 172.21.15.114:0/837250168 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+6890 (secure 0 0 0) 0x7fa98c0681c0 con 0x7fa99c06f600 2024-05-27T09:37:10.176 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:37:10.176 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5072,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/1057092553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":1057092553},{"type":"v1","addr":"172.21.15.163:6839","nonce":1057092553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:36:01.014181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:active","state_seq":26,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5160,5086,4875]},"id":1}]} 2024-05-27T09:37:10.176 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 42 2024-05-27T09:37:10.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:10.177+0000 7fa9a1c00640 1 -- 172.21.15.114:0/837250168 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa97c0762d0 msgr2=0x7fa97c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:10.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:10.177+0000 7fa9a1c00640 1 --2- 172.21.15.114:0/837250168 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa97c0762d0 0x7fa97c0787e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa99c1e6180 tx=0x7fa988009040 comp rx=0 tx=0).stop 2024-05-27T09:37:10.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:10.177+0000 7fa9a1c00640 1 -- 172.21.15.114:0/837250168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa99c06f600 msgr2=0x7fa99c14c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:10.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:10.177+0000 7fa9a1c00640 1 --2- 172.21.15.114:0/837250168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa99c06f600 0x7fa99c14c370 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00c3c0 tx=0x7fa98c00c880 comp rx=0 tx=0).stop 2024-05-27T09:37:10.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:10.177+0000 7fa9a1c00640 1 -- 172.21.15.114:0/837250168 shutdown_connections 2024-05-27T09:37:10.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:10.177+0000 7fa9a1c00640 1 --2- 172.21.15.114:0/837250168 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa97c0762d0 0x7fa97c0787e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:10.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:10.177+0000 7fa9a1c00640 1 --2- 172.21.15.114:0/837250168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa99c0c8580 0x7fa99c146c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:10.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:10.177+0000 7fa9a1c00640 1 --2- 172.21.15.114:0/837250168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa99c0b34a0 0x7fa99c1f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:10.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:10.177+0000 7fa9a1c00640 1 --2- 172.21.15.114:0/837250168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa99c06f600 0x7fa99c14c370 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:10.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:10.177+0000 7fa9a1c00640 1 -- 172.21.15.114:0/837250168 >> 172.21.15.114:0/837250168 conn(0x7fa99c0ca320 msgr2=0x7fa99c0cb620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:10.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:10.178+0000 7fa9a1c00640 1 -- 172.21.15.114:0/837250168 shutdown_connections 2024-05-27T09:37:10.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:10.178+0000 7fa9a1c00640 1 -- 172.21.15.114:0/837250168 wait complete. 2024-05-27T09:37:10.207 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-05-27T09:37:12.207 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:37:12.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.490+0000 7f7ea7200640 1 Processor -- start 2024-05-27T09:37:12.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.491+0000 7f7ea7200640 1 -- start start 2024-05-27T09:37:12.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.493+0000 7f7ea7200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7ea00c8780 0x7f7ea015fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:12.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.493+0000 7f7ea7200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ea00b75b0 0x7f7ea00d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:12.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.493+0000 7f7ea7200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ea00cd1e0 con 0x7f7ea00b46e0 2024-05-27T09:37:12.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.493+0000 7f7ea7200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f7ea00cd4c0 con 0x7f7ea00c8780 2024-05-27T09:37:12.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.493+0000 7f7ea7200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea00c7f80 con 0x7f7ea00b75b0 2024-05-27T09:37:12.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.493+0000 7f7ea5e00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7ea00c8780 0x7f7ea015fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:12.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.493+0000 7f7ea5400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ea00b75b0 0x7f7ea00d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:12.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.493+0000 7f7ea5e00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7ea00c8780 0x7f7ea015fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37290/0 (socket says 172.21.15.114:37290) 2024-05-27T09:37:12.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.493+0000 7f7ea5e00640 1 -- 172.21.15.114:0/1148422893 learned_addr learned my addr 172.21.15.114:0/1148422893 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:12.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.494+0000 7f7ea5400640 1 -- 172.21.15.114:0/1148422893 >> v2:172.21.15.163:3300/0 conn(0x7f7ea00c8780 msgr2=0x7f7ea015fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:12.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.494+0000 7f7ea5400640 1 --2- 172.21.15.114:0/1148422893 >> v2:172.21.15.163:3300/0 conn(0x7f7ea00c8780 0x7f7ea015fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:12.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.494+0000 7f7ea5400640 1 -- 172.21.15.114:0/1148422893 >> v1:172.21.15.163:6789/0 conn(0x7f7ea00b46e0 legacy=0x7f7ea00ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:12.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.494+0000 7f7ea5400640 1 -- 172.21.15.114:0/1148422893 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea0160160 con 0x7f7ea00b75b0 2024-05-27T09:37:12.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.494+0000 7f7ea5e00640 1 --2- 172.21.15.114:0/1148422893 >> v2:172.21.15.163:3300/0 conn(0x7f7ea00c8780 0x7f7ea015fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:12.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.494+0000 7f7ea4a00640 1 -- 172.21.15.114:0/1148422893 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 531334595 0 0) 0x7f7ea00cd1e0 con 0x7f7ea00b46e0 2024-05-27T09:37:12.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.495+0000 7f7ea5400640 1 --2- 172.21.15.114:0/1148422893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ea00b75b0 0x7f7ea00d1e50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7e900097a0 tx=0x7f7e9002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51fbe579993edc50 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:12.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.495+0000 7f7ea4a00640 1 -- 172.21.15.114:0/1148422893 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e90037070 con 0x7f7ea00b75b0 2024-05-27T09:37:12.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.495+0000 7f7ea4a00640 1 -- 172.21.15.114:0/1148422893 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e90032440 con 0x7f7ea00b75b0 2024-05-27T09:37:12.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.495+0000 7f7ea4a00640 1 -- 172.21.15.114:0/1148422893 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e90032810 con 0x7f7ea00b75b0 2024-05-27T09:37:12.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.496+0000 7f7ea7200640 1 -- 172.21.15.114:0/1148422893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ea00b75b0 msgr2=0x7f7ea00d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:12.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.496+0000 7f7ea7200640 1 --2- 172.21.15.114:0/1148422893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ea00b75b0 0x7f7ea00d1e50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7e900097a0 tx=0x7f7e9002a250 comp rx=0 tx=0).stop 2024-05-27T09:37:12.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.496+0000 7f7ea7200640 1 -- 172.21.15.114:0/1148422893 shutdown_connections 2024-05-27T09:37:12.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.496+0000 7f7ea7200640 1 --2- 172.21.15.114:0/1148422893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ea00b75b0 0x7f7ea00d1e50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:12.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.496+0000 7f7ea7200640 1 --2- 172.21.15.114:0/1148422893 >> v2:172.21.15.163:3300/0 conn(0x7f7ea00c8780 0x7f7ea015fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:12.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.496+0000 7f7ea7200640 1 -- 172.21.15.114:0/1148422893 >> 172.21.15.114:0/1148422893 conn(0x7f7ea006e040 msgr2=0x7f7ea006e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:12.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.496+0000 7f7ea7200640 1 -- 172.21.15.114:0/1148422893 shutdown_connections 2024-05-27T09:37:12.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.496+0000 7f7ea7200640 1 -- 172.21.15.114:0/1148422893 wait complete. 2024-05-27T09:37:12.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.499+0000 7f7ea7200640 1 Processor -- start 2024-05-27T09:37:12.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.499+0000 7f7ea7200640 1 -- start start 2024-05-27T09:37:12.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.501+0000 7f7ea7200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ea00b46e0 0x7f7ea01f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:12.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.501+0000 7f7ea7200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ea00b75b0 0x7f7ea01f6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:12.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.501+0000 7f7ea7200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ea00c8780 0x7f7ea01f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:12.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.501+0000 7f7ea7200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea00c6800 con 0x7f7ea00c8780 2024-05-27T09:37:12.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.501+0000 7f7ea7200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea00cdf30 con 0x7f7ea00b46e0 2024-05-27T09:37:12.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.501+0000 7f7ea7200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea0157550 con 0x7f7ea00b75b0 2024-05-27T09:37:12.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.502+0000 7f7ea5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ea00b46e0 0x7f7ea01f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:12.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.502+0000 7f7ea5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ea00b46e0 0x7f7ea01f5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:37292/0 (socket says 172.21.15.114:37292) 2024-05-27T09:37:12.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.502+0000 7f7ea5e00640 1 -- 172.21.15.114:0/1080061136 learned_addr learned my addr 172.21.15.114:0/1080061136 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:12.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.502+0000 7f7ea5400640 1 --2- 172.21.15.114:0/1080061136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ea00b75b0 0x7f7ea01f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:12.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.502+0000 7f7ea6800640 1 --2- 172.21.15.114:0/1080061136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ea00c8780 0x7f7ea01f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:12.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.502+0000 7f7ea5e00640 1 -- 172.21.15.114:0/1080061136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ea00b75b0 msgr2=0x7f7ea01f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:12.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.502+0000 7f7ea5e00640 1 --2- 172.21.15.114:0/1080061136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ea00b75b0 0x7f7ea01f6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:12.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.502+0000 7f7ea5e00640 1 -- 172.21.15.114:0/1080061136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ea00c8780 msgr2=0x7f7ea01f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:12.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.502+0000 7f7ea5e00640 1 --2- 172.21.15.114:0/1080061136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ea00c8780 0x7f7ea01f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:12.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.502+0000 7f7ea5e00640 1 -- 172.21.15.114:0/1080061136 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea01543d0 con 0x7f7ea00b46e0 2024-05-27T09:37:12.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.502+0000 7f7ea5400640 1 --2- 172.21.15.114:0/1080061136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ea00b75b0 0x7f7ea01f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:12.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.502+0000 7f7ea5e00640 1 --2- 172.21.15.114:0/1080061136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ea00b46e0 0x7f7ea01f5c20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400e900 tx=0x7f7e9400edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:12.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.503+0000 7f7e9a000640 1 -- 172.21.15.114:0/1080061136 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94017070 con 0x7f7ea00b46e0 2024-05-27T09:37:12.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.503+0000 7f7e9a000640 1 -- 172.21.15.114:0/1080061136 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9400ca00 con 0x7f7ea00b46e0 2024-05-27T09:37:12.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.503+0000 7f7e9a000640 1 -- 172.21.15.114:0/1080061136 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9400cdf0 con 0x7f7ea00b46e0 2024-05-27T09:37:12.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.503+0000 7f7ea7200640 1 -- 172.21.15.114:0/1080061136 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea0154670 con 0x7f7ea00b46e0 2024-05-27T09:37:12.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.503+0000 7f7ea7200640 1 -- 172.21.15.114:0/1080061136 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea0159dc0 con 0x7f7ea00b46e0 2024-05-27T09:37:12.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.504+0000 7f7ea7200640 1 -- 172.21.15.114:0/1080061136 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea0050860 con 0x7f7ea00b46e0 2024-05-27T09:37:12.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.505+0000 7f7e9a000640 1 -- 172.21.15.114:0/1080061136 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7e9401a070 con 0x7f7ea00b46e0 2024-05-27T09:37:12.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.506+0000 7f7e9a000640 1 --2- 172.21.15.114:0/1080061136 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7e84076300 0x7f7e84078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:12.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.507+0000 7f7e9a000640 1 -- 172.21.15.114:0/1080061136 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6168+0+0 (secure 0 0 0) 0x7f7e94097a10 con 0x7f7ea00b46e0 2024-05-27T09:37:12.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.507+0000 7f7ea5400640 1 --2- 172.21.15.114:0/1080061136 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7e84076300 0x7f7e84078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:12.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.507+0000 7f7ea5400640 1 --2- 172.21.15.114:0/1080061136 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7e84076300 0x7f7e84078810 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7e90034000 tx=0x7f7e9002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:12.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.512+0000 7f7e9a000640 1 -- 172.21.15.114:0/1080061136 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7e94060ab0 con 0x7f7ea00b46e0 2024-05-27T09:37:12.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.786+0000 7f7ea7200640 1 -- 172.21.15.114:0/1080061136 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ea00cdd10 con 0x7f7ea00b46e0 2024-05-27T09:37:12.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.787+0000 7f7e9a000640 1 -- 172.21.15.114:0/1080061136 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+6890 (secure 0 0 0) 0x7f7e94064db0 con 0x7f7ea00b46e0 2024-05-27T09:37:12.789 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:37:12.789 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5072,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/1057092553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":1057092553},{"type":"v1","addr":"172.21.15.163:6839","nonce":1057092553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:36:01.014181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:active","state_seq":26,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5160,5086,4875]},"id":1}]} 2024-05-27T09:37:12.789 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 42 2024-05-27T09:37:12.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.791+0000 7f7ea7200640 1 -- 172.21.15.114:0/1080061136 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7e84076300 msgr2=0x7f7e84078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:12.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.791+0000 7f7ea7200640 1 --2- 172.21.15.114:0/1080061136 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7e84076300 0x7f7e84078810 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7e90034000 tx=0x7f7e9002e000 comp rx=0 tx=0).stop 2024-05-27T09:37:12.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.791+0000 7f7ea7200640 1 -- 172.21.15.114:0/1080061136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ea00b46e0 msgr2=0x7f7ea01f5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:12.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.791+0000 7f7ea7200640 1 --2- 172.21.15.114:0/1080061136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ea00b46e0 0x7f7ea01f5c20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400e900 tx=0x7f7e9400edc0 comp rx=0 tx=0).stop 2024-05-27T09:37:12.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.791+0000 7f7ea7200640 1 -- 172.21.15.114:0/1080061136 shutdown_connections 2024-05-27T09:37:12.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.791+0000 7f7ea7200640 1 --2- 172.21.15.114:0/1080061136 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f7e84076300 0x7f7e84078810 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:12.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.791+0000 7f7ea7200640 1 --2- 172.21.15.114:0/1080061136 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ea00c8780 0x7f7ea01f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:12.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.791+0000 7f7ea7200640 1 --2- 172.21.15.114:0/1080061136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ea00b75b0 0x7f7ea01f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:12.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.791+0000 7f7ea7200640 1 --2- 172.21.15.114:0/1080061136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ea00b46e0 0x7f7ea01f5c20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:12.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.791+0000 7f7ea7200640 1 -- 172.21.15.114:0/1080061136 >> 172.21.15.114:0/1080061136 conn(0x7f7ea006e040 msgr2=0x7f7ea00b68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:12.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.791+0000 7f7ea7200640 1 -- 172.21.15.114:0/1080061136 shutdown_connections 2024-05-27T09:37:12.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:12.791+0000 7f7ea7200640 1 -- 172.21.15.114:0/1080061136 wait complete. 2024-05-27T09:37:12.819 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-05-27T09:37:14.821 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:37:15.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.090+0000 7f0f0f000640 1 Processor -- start 2024-05-27T09:37:15.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.090+0000 7f0f0f000640 1 -- start start 2024-05-27T09:37:15.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.092+0000 7f0f0f000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f081534c0 0x7f0f080c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:15.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.092+0000 7f0f0f000640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0f080c8630 0x7f0f08074140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:15.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.092+0000 7f0f0f000640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f080c96d0 con 0x7f0f080c8220 2024-05-27T09:37:15.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.092+0000 7f0f0f000640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f0f080c2f40 con 0x7f0f080c8630 2024-05-27T09:37:15.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.092+0000 7f0f0f000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f0f08159190 con 0x7f0f081534c0 2024-05-27T09:37:15.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.093+0000 7f0f0e600640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0f080c8630 0x7f0f08074140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:15.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.093+0000 7f0f0dc00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f081534c0 0x7f0f080c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:15.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.093+0000 7f0f0e600640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0f080c8630 0x7f0f08074140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33312/0 (socket says 172.21.15.114:33312) 2024-05-27T09:37:15.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.093+0000 7f0f0e600640 1 -- 172.21.15.114:0/1932107152 learned_addr learned my addr 172.21.15.114:0/1932107152 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:15.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.093+0000 7f0f0dc00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f081534c0 0x7f0f080c2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:39384/0 (socket says 172.21.15.114:39384) 2024-05-27T09:37:15.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.093+0000 7f0f0dc00640 1 -- 172.21.15.114:0/1932107152 >> v2:172.21.15.114:3300/0 conn(0x7f0f080c8630 msgr2=0x7f0f08074140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:15.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.093+0000 7f0f0dc00640 1 --2- 172.21.15.114:0/1932107152 >> v2:172.21.15.114:3300/0 conn(0x7f0f080c8630 0x7f0f08074140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:15.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.093+0000 7f0f0dc00640 1 -- 172.21.15.114:0/1932107152 >> v1:172.21.15.114:6789/0 conn(0x7f0f080c8220 legacy=0x7f0f08159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:15.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.093+0000 7f0f0dc00640 1 -- 172.21.15.114:0/1932107152 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f080c9970 con 0x7f0f081534c0 2024-05-27T09:37:15.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.093+0000 7f0f07e00640 1 -- 172.21.15.114:0/1932107152 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3681985398 0 0) 0x7f0f080c96d0 con 0x7f0f080c8220 2024-05-27T09:37:15.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.093+0000 7f0f0e600640 1 --2- 172.21.15.114:0/1932107152 >> v2:172.21.15.114:3300/0 conn(0x7f0f080c8630 0x7f0f08074140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:15.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.094+0000 7f0f0dc00640 1 --2- 172.21.15.114:0/1932107152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f081534c0 0x7f0f080c2a00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40097a0 tx=0x7f0ef402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=587e58f5bc0cfa7b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:15.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.095+0000 7f0f07e00640 1 -- 172.21.15.114:0/1932107152 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4037070 con 0x7f0f081534c0 2024-05-27T09:37:15.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.095+0000 7f0f07e00640 1 -- 172.21.15.114:0/1932107152 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef4032440 con 0x7f0f081534c0 2024-05-27T09:37:15.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.095+0000 7f0f07e00640 1 -- 172.21.15.114:0/1932107152 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4032810 con 0x7f0f081534c0 2024-05-27T09:37:15.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.095+0000 7f0f0f000640 1 -- 172.21.15.114:0/1932107152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f081534c0 msgr2=0x7f0f080c2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:15.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.095+0000 7f0f0f000640 1 --2- 172.21.15.114:0/1932107152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f081534c0 0x7f0f080c2a00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40097a0 tx=0x7f0ef402a180 comp rx=0 tx=0).stop 2024-05-27T09:37:15.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.095+0000 7f0f0f000640 1 -- 172.21.15.114:0/1932107152 shutdown_connections 2024-05-27T09:37:15.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.095+0000 7f0f0f000640 1 --2- 172.21.15.114:0/1932107152 >> v2:172.21.15.114:3300/0 conn(0x7f0f080c8630 0x7f0f08074140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:15.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.095+0000 7f0f0f000640 1 --2- 172.21.15.114:0/1932107152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f081534c0 0x7f0f080c2a00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:15.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.095+0000 7f0f0f000640 1 -- 172.21.15.114:0/1932107152 >> 172.21.15.114:0/1932107152 conn(0x7f0f080ca320 msgr2=0x7f0f080ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:15.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.095+0000 7f0f0f000640 1 -- 172.21.15.114:0/1932107152 shutdown_connections 2024-05-27T09:37:15.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.096+0000 7f0f0f000640 1 -- 172.21.15.114:0/1932107152 wait complete. 2024-05-27T09:37:15.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.098+0000 7f0f0f000640 1 Processor -- start 2024-05-27T09:37:15.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.098+0000 7f0f0f000640 1 -- start start 2024-05-27T09:37:15.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.099+0000 7f0f0f000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f080c8220 0x7f0f081269a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:15.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.099+0000 7f0f0f000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f080c8630 0x7f0f081f6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:15.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.099+0000 7f0f0f000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f081534c0 0x7f0f08155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:15.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.099+0000 7f0f0f000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0f080c9f30 con 0x7f0f081534c0 2024-05-27T09:37:15.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.099+0000 7f0f0f000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081538d0 con 0x7f0f080c8630 2024-05-27T09:37:15.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.099+0000 7f0f0f000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f0f081593e0 con 0x7f0f080c8220 2024-05-27T09:37:15.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.100+0000 7f0f0e600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f081534c0 0x7f0f08155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:15.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.100+0000 7f0f0dc00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f080c8220 0x7f0f081269a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:15.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.100+0000 7f0f0d200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f080c8630 0x7f0f081f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:15.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.100+0000 7f0f0e600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f081534c0 0x7f0f08155e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33322/0 (socket says 172.21.15.114:33322) 2024-05-27T09:37:15.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.100+0000 7f0f0e600640 1 -- 172.21.15.114:0/1124880989 learned_addr learned my addr 172.21.15.114:0/1124880989 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:15.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.100+0000 7f0f0dc00640 1 -- 172.21.15.114:0/1124880989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f080c8630 msgr2=0x7f0f081f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:15.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.100+0000 7f0f0dc00640 1 --2- 172.21.15.114:0/1124880989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f080c8630 0x7f0f081f6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:15.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.100+0000 7f0f0dc00640 1 -- 172.21.15.114:0/1124880989 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f081534c0 msgr2=0x7f0f08155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:15.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.100+0000 7f0f0dc00640 1 --2- 172.21.15.114:0/1124880989 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f081534c0 0x7f0f08155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:15.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.100+0000 7f0f0dc00640 1 -- 172.21.15.114:0/1124880989 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f081f66a0 con 0x7f0f080c8220 2024-05-27T09:37:15.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.100+0000 7f0f0e600640 1 --2- 172.21.15.114:0/1124880989 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f081534c0 0x7f0f08155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:15.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.100+0000 7f0f0d200640 1 --2- 172.21.15.114:0/1124880989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f080c8630 0x7f0f081f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:15.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.101+0000 7f0f0dc00640 1 --2- 172.21.15.114:0/1124880989 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f080c8220 0x7f0f081269a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4031000 tx=0x7f0ef40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:15.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.101+0000 7f0f05600640 1 -- 172.21.15.114:0/1124880989 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4041020 con 0x7f0f080c8220 2024-05-27T09:37:15.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.101+0000 7f0f05600640 1 -- 172.21.15.114:0/1124880989 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef4030700 con 0x7f0f080c8220 2024-05-27T09:37:15.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.101+0000 7f0f05600640 1 -- 172.21.15.114:0/1124880989 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4037040 con 0x7f0f080c8220 2024-05-27T09:37:15.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.101+0000 7f0f0f000640 1 -- 172.21.15.114:0/1124880989 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f080b1650 con 0x7f0f080c8220 2024-05-27T09:37:15.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.101+0000 7f0f0f000640 1 -- 172.21.15.114:0/1124880989 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f080b1c10 con 0x7f0f080c8220 2024-05-27T09:37:15.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.102+0000 7f0f0f000640 1 -- 172.21.15.114:0/1124880989 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f08050860 con 0x7f0f080c8220 2024-05-27T09:37:15.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.103+0000 7f0f05600640 1 -- 172.21.15.114:0/1124880989 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0ef4044020 con 0x7f0f080c8220 2024-05-27T09:37:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.104+0000 7f0f05600640 1 --2- 172.21.15.114:0/1124880989 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0ee8076300 0x7f0ee8078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.104+0000 7f0f05600640 1 -- 172.21.15.114:0/1124880989 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(49..49 src has 1..49) ==== 6168+0+0 (secure 0 0 0) 0x7f0ef40b7f30 con 0x7f0f080c8220 2024-05-27T09:37:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.104+0000 7f0f0d200640 1 --2- 172.21.15.114:0/1124880989 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0ee8076300 0x7f0ee8078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:15.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.105+0000 7f0f0d200640 1 --2- 172.21.15.114:0/1124880989 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0ee8076300 0x7f0ee8078810 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0f08157230 tx=0x7f0efc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:15.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.109+0000 7f0f05600640 1 -- 172.21.15.114:0/1124880989 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0ef404b030 con 0x7f0f080c8220 2024-05-27T09:37:15.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.336+0000 7f0f0f000640 1 -- 172.21.15.114:0/1124880989 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f080c40f0 con 0x7f0f080c8220 2024-05-27T09:37:15.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.337+0000 7f0f05600640 1 -- 172.21.15.114:0/1124880989 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+6890 (secure 0 0 0) 0x7f0ef4080fd0 con 0x7f0f080c8220 2024-05-27T09:37:15.339 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:37:15.339 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":5072,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/1057092553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":1057092553},{"type":"v1","addr":"172.21.15.163:6839","nonce":1057092553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:36:01.014181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":4875,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4875":{"gid":4875,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":25,"addr":"172.21.15.114:6835/2747052034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6833","nonce":2747052034},{"type":"v1","addr":"172.21.15.114:6835","nonce":2747052034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:active","state_seq":26,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5160,5086,4875]},"id":1}]} 2024-05-27T09:37:15.339 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 42 2024-05-27T09:37:15.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.339+0000 7f0f0f000640 1 -- 172.21.15.114:0/1124880989 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0ee8076300 msgr2=0x7f0ee8078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:15.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.339+0000 7f0f0f000640 1 --2- 172.21.15.114:0/1124880989 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0ee8076300 0x7f0ee8078810 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0f08157230 tx=0x7f0efc009040 comp rx=0 tx=0).stop 2024-05-27T09:37:15.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.339+0000 7f0f0f000640 1 -- 172.21.15.114:0/1124880989 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f080c8220 msgr2=0x7f0f081269a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:15.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.339+0000 7f0f0f000640 1 --2- 172.21.15.114:0/1124880989 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f080c8220 0x7f0f081269a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4031000 tx=0x7f0ef40046c0 comp rx=0 tx=0).stop 2024-05-27T09:37:15.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.340+0000 7f0f0f000640 1 -- 172.21.15.114:0/1124880989 shutdown_connections 2024-05-27T09:37:15.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.340+0000 7f0f0f000640 1 --2- 172.21.15.114:0/1124880989 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0ee8076300 0x7f0ee8078810 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:15.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.340+0000 7f0f0f000640 1 --2- 172.21.15.114:0/1124880989 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f081534c0 0x7f0f08155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:15.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.340+0000 7f0f0f000640 1 --2- 172.21.15.114:0/1124880989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f080c8630 0x7f0f081f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:15.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.340+0000 7f0f0f000640 1 --2- 172.21.15.114:0/1124880989 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f080c8220 0x7f0f081269a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:15.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.340+0000 7f0f0f000640 1 -- 172.21.15.114:0/1124880989 >> 172.21.15.114:0/1124880989 conn(0x7f0f080ca320 msgr2=0x7f0f080c72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:15.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.340+0000 7f0f0f000640 1 -- 172.21.15.114:0/1124880989 shutdown_connections 2024-05-27T09:37:15.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:15.340+0000 7f0f0f000640 1 -- 172.21.15.114:0/1124880989 wait complete. 2024-05-27T09:37:15.368 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-05-27T09:37:17.370 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:37:17.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.653+0000 7f40dd400640 1 Processor -- start 2024-05-27T09:37:17.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.654+0000 7f40dd400640 1 -- start start 2024-05-27T09:37:17.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.656+0000 7f40dd400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f40d8154820 0x7f40d80ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:17.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.656+0000 7f40dd400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f40d80ca810 0x7f40d815dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:17.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.656+0000 7f40dd400640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40d80cac20 con 0x7f40d8074a60 2024-05-27T09:37:17.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.656+0000 7f40dd400640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f40d8154c30 con 0x7f40d8154820 2024-05-27T09:37:17.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.656+0000 7f40dd400640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f40d815db60 con 0x7f40d80ca810 2024-05-27T09:37:17.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.656+0000 7f40cde00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f40d8154820 0x7f40d80ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:17.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.656+0000 7f40d7200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f40d80ca810 0x7f40d815dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:17.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.656+0000 7f40d6800640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f40d8074a60 0x7f40d8073260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:36024/0 (socket says 172.21.15.114:36024) 2024-05-27T09:37:17.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.656+0000 7f40d6800640 1 -- 172.21.15.114:0/1243503931 learned_addr learned my addr 172.21.15.114:0/1243503931 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:17.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.657+0000 7f40d7200640 1 -- 172.21.15.114:0/1243503931 >> v2:172.21.15.114:3300/0 conn(0x7f40d8154820 msgr2=0x7f40d80ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:17.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.657+0000 7f40d7200640 1 --2- 172.21.15.114:0/1243503931 >> v2:172.21.15.114:3300/0 conn(0x7f40d8154820 0x7f40d80ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:17.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.657+0000 7f40d7200640 1 -- 172.21.15.114:0/1243503931 >> v1:172.21.15.163:6789/0 conn(0x7f40d8074a60 legacy=0x7f40d8073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:17.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.657+0000 7f40d7200640 1 -- 172.21.15.114:0/1243503931 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d815a810 con 0x7f40d80ca810 2024-05-27T09:37:17.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.657+0000 7f40d5e00640 1 -- 172.21.15.114:0/1243503931 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3990678293 0 0) 0x7f40d80cac20 con 0x7f40d8074a60 2024-05-27T09:37:17.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.658+0000 7f40d7200640 1 --2- 172.21.15.114:0/1243503931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40d80ca810 0x7f40d815dff0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f40d80cb440 tx=0x7f40d002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad0e85c574a97a4b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:17.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.659+0000 7f40d5e00640 1 -- 172.21.15.114:0/1243503931 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d0007740 con 0x7f40d80ca810 2024-05-27T09:37:17.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.659+0000 7f40d5e00640 1 -- 172.21.15.114:0/1243503931 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40d00078e0 con 0x7f40d80ca810 2024-05-27T09:37:17.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.659+0000 7f40d5e00640 1 -- 172.21.15.114:0/1243503931 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d0007cb0 con 0x7f40d80ca810 2024-05-27T09:37:17.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.659+0000 7f40dd400640 1 -- 172.21.15.114:0/1243503931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40d80ca810 msgr2=0x7f40d815dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:17.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.659+0000 7f40dd400640 1 --2- 172.21.15.114:0/1243503931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40d80ca810 0x7f40d815dff0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f40d80cb440 tx=0x7f40d002a710 comp rx=0 tx=0).stop 2024-05-27T09:37:17.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.659+0000 7f40dd400640 1 -- 172.21.15.114:0/1243503931 shutdown_connections 2024-05-27T09:37:17.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.659+0000 7f40dd400640 1 --2- 172.21.15.114:0/1243503931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40d80ca810 0x7f40d815dff0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:17.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.659+0000 7f40dd400640 1 --2- 172.21.15.114:0/1243503931 >> v2:172.21.15.114:3300/0 conn(0x7f40d8154820 0x7f40d80ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:17.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.659+0000 7f40dd400640 1 -- 172.21.15.114:0/1243503931 >> 172.21.15.114:0/1243503931 conn(0x7f40d806f6e0 msgr2=0x7f40d8070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:17.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.659+0000 7f40dd400640 1 -- 172.21.15.114:0/1243503931 shutdown_connections 2024-05-27T09:37:17.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.660+0000 7f40dd400640 1 -- 172.21.15.114:0/1243503931 wait complete. 2024-05-27T09:37:17.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.662+0000 7f40dd400640 1 Processor -- start 2024-05-27T09:37:17.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.662+0000 7f40dd400640 1 -- start start 2024-05-27T09:37:17.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.663+0000 7f40dd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40d8074a60 0x7f40d81f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:17.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.663+0000 7f40dd400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40d80ca810 0x7f40d8159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:17.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.663+0000 7f40dd400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40d8154820 0x7f40d81472a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:17.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.663+0000 7f40dd400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f40d815b0b0 con 0x7f40d8074a60 2024-05-27T09:37:17.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.663+0000 7f40dd400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f40d815add0 con 0x7f40d80ca810 2024-05-27T09:37:17.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.663+0000 7f40dd400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f40d815b390 con 0x7f40d8154820 2024-05-27T09:37:17.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.664+0000 7f40d6800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40d8074a60 0x7f40d81f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:17.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.664+0000 7f40d6800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40d8074a60 0x7f40d81f4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33338/0 (socket says 172.21.15.114:33338) 2024-05-27T09:37:17.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.664+0000 7f40d6800640 1 -- 172.21.15.114:0/359122408 learned_addr learned my addr 172.21.15.114:0/359122408 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:17.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.664+0000 7f40d7200640 1 --2- 172.21.15.114:0/359122408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40d8154820 0x7f40d81472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:17.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.664+0000 7f40cde00640 1 --2- 172.21.15.114:0/359122408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40d80ca810 0x7f40d8159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:17.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.664+0000 7f40d6800640 1 -- 172.21.15.114:0/359122408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40d8154820 msgr2=0x7f40d81472a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:17.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.664+0000 7f40d6800640 1 --2- 172.21.15.114:0/359122408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40d8154820 0x7f40d81472a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:17.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.664+0000 7f40d6800640 1 -- 172.21.15.114:0/359122408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40d80ca810 msgr2=0x7f40d8159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:17.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.664+0000 7f40d6800640 1 --2- 172.21.15.114:0/359122408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40d80ca810 0x7f40d8159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:17.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.664+0000 7f40d6800640 1 -- 172.21.15.114:0/359122408 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d81f0b00 con 0x7f40d8074a60 2024-05-27T09:37:17.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.664+0000 7f40d6800640 1 --2- 172.21.15.114:0/359122408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40d8074a60 0x7f40d81f4bb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f40c800c3e0 tx=0x7f40c800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:17.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.665+0000 7f40cf400640 1 -- 172.21.15.114:0/359122408 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c80074e0 con 0x7f40d8074a60 2024-05-27T09:37:17.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.665+0000 7f40cf400640 1 -- 172.21.15.114:0/359122408 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c80047e0 con 0x7f40d8074a60 2024-05-27T09:37:17.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.665+0000 7f40dd400640 1 -- 172.21.15.114:0/359122408 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d81f4110 con 0x7f40d8074a60 2024-05-27T09:37:17.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.665+0000 7f40cf400640 1 -- 172.21.15.114:0/359122408 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8004bd0 con 0x7f40d8074a60 2024-05-27T09:37:17.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.665+0000 7f40dd400640 1 -- 172.21.15.114:0/359122408 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d81f43b0 con 0x7f40d8074a60 2024-05-27T09:37:17.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.666+0000 7f40dd400640 1 -- 172.21.15.114:0/359122408 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40d806a380 con 0x7f40d8074a60 2024-05-27T09:37:17.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.667+0000 7f40cf400640 1 -- 172.21.15.114:0/359122408 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f40c8011070 con 0x7f40d8074a60 2024-05-27T09:37:17.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.668+0000 7f40cf400640 1 --2- 172.21.15.114:0/359122408 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f40b40762d0 0x7f40b40787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:17.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.668+0000 7f40cf400640 1 -- 172.21.15.114:0/359122408 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7f40c8097300 con 0x7f40d8074a60 2024-05-27T09:37:17.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.669+0000 7f40cde00640 1 --2- 172.21.15.114:0/359122408 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f40b40762d0 0x7f40b40787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:17.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.669+0000 7f40cde00640 1 --2- 172.21.15.114:0/359122408 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f40b40762d0 0x7f40b40787e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f40d80c7d20 tx=0x7f40c0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:17.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.674+0000 7f40cf400640 1 -- 172.21.15.114:0/359122408 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f40c8060350 con 0x7f40d8074a60 2024-05-27T09:37:17.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.920+0000 7f40dd400640 1 -- 172.21.15.114:0/359122408 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40d8157e90 con 0x7f40d8074a60 2024-05-27T09:37:17.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.921+0000 7f40cf400640 1 -- 172.21.15.114:0/359122408 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6056 (secure 0 0 0) 0x7f40c80970c0 con 0x7f40d8074a60 2024-05-27T09:37:17.923 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:37:17.923 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:37:15.517199+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":5072,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5072":{"gid":5072,"name":"f","rank":1,"incarnation":43,"state":"up:replay","state_seq":1,"addr":"172.21.15.163:6839/1057092553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":1057092553},{"type":"v1","addr":"172.21.15.163:6839","nonce":1057092553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:active","state_seq":26,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086,5160]},"id":1}]} 2024-05-27T09:37:17.924 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 43 2024-05-27T09:37:17.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.924+0000 7f40dd400640 1 -- 172.21.15.114:0/359122408 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f40b40762d0 msgr2=0x7f40b40787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:17.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.924+0000 7f40dd400640 1 --2- 172.21.15.114:0/359122408 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f40b40762d0 0x7f40b40787e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f40d80c7d20 tx=0x7f40c0009190 comp rx=0 tx=0).stop 2024-05-27T09:37:17.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.924+0000 7f40dd400640 1 -- 172.21.15.114:0/359122408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40d8074a60 msgr2=0x7f40d81f4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:17.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.924+0000 7f40dd400640 1 --2- 172.21.15.114:0/359122408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40d8074a60 0x7f40d81f4bb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f40c800c3e0 tx=0x7f40c800c8a0 comp rx=0 tx=0).stop 2024-05-27T09:37:17.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.924+0000 7f40dd400640 1 -- 172.21.15.114:0/359122408 shutdown_connections 2024-05-27T09:37:17.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.924+0000 7f40dd400640 1 --2- 172.21.15.114:0/359122408 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f40b40762d0 0x7f40b40787e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:17.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.924+0000 7f40dd400640 1 --2- 172.21.15.114:0/359122408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40d8154820 0x7f40d81472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:17.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.924+0000 7f40dd400640 1 --2- 172.21.15.114:0/359122408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40d80ca810 0x7f40d8159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:17.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.924+0000 7f40dd400640 1 --2- 172.21.15.114:0/359122408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40d8074a60 0x7f40d81f4bb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:17.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.924+0000 7f40dd400640 1 -- 172.21.15.114:0/359122408 >> 172.21.15.114:0/359122408 conn(0x7f40d806f6e0 msgr2=0x7f40d814a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:17.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.925+0000 7f40dd400640 1 -- 172.21.15.114:0/359122408 shutdown_connections 2024-05-27T09:37:17.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:17.925+0000 7f40dd400640 1 -- 172.21.15.114:0/359122408 wait complete. 2024-05-27T09:37:17.953 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2024-05-27T09:37:17.953 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-05-27T09:37:17.953 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:37:18.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.223+0000 7f4782200640 1 Processor -- start 2024-05-27T09:37:18.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.223+0000 7f4782200640 1 -- start start 2024-05-27T09:37:18.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.225+0000 7f4782200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f477c1593b0 0x7f477c142720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:18.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.225+0000 7f4782200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f477c0739a0 con 0x7f477c0c2990 2024-05-27T09:37:18.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.225+0000 7f4782200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f477c073c80 con 0x7f477c073260 2024-05-27T09:37:18.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.225+0000 7f4782200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f477c074d20 con 0x7f477c1593b0 2024-05-27T09:37:18.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.225+0000 7f477be00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f477c0c2990 0x7f477c163d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57164/0 (socket says 172.21.15.114:57164) 2024-05-27T09:37:18.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.225+0000 7f477be00640 1 -- 172.21.15.114:0/4124620622 learned_addr learned my addr 172.21.15.114:0/4124620622 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:18.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.225+0000 7f477b400640 1 --2- 172.21.15.114:0/4124620622 >> v2:172.21.15.163:3300/0 conn(0x7f477c1593b0 0x7f477c142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:18.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.226+0000 7f477b400640 1 -- 172.21.15.114:0/4124620622 >> v1:172.21.15.163:6789/0 conn(0x7f477c073260 legacy=0x7f477c15a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:18.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.226+0000 7f477b400640 1 -- 172.21.15.114:0/4124620622 >> v1:172.21.15.114:6789/0 conn(0x7f477c0c2990 legacy=0x7f477c163d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:18.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.226+0000 7f477b400640 1 -- 172.21.15.114:0/4124620622 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477c073f60 con 0x7f477c1593b0 2024-05-27T09:37:18.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.226+0000 7f477aa00640 1 -- 172.21.15.114:0/4124620622 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3585282993 0 0) 0x7f477c0739a0 con 0x7f477c0c2990 2024-05-27T09:37:18.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.227+0000 7f477b400640 1 --2- 172.21.15.114:0/4124620622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f477c1593b0 0x7f477c142720 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f476c0097a0 tx=0x7f476c02a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9afc5a206f13196a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:18.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.227+0000 7f477aa00640 1 -- 172.21.15.114:0/4124620622 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c037070 con 0x7f477c1593b0 2024-05-27T09:37:18.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.227+0000 7f477aa00640 1 -- 172.21.15.114:0/4124620622 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f476c032640 con 0x7f477c1593b0 2024-05-27T09:37:18.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.227+0000 7f477aa00640 1 -- 172.21.15.114:0/4124620622 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c032a10 con 0x7f477c1593b0 2024-05-27T09:37:18.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.228+0000 7f4782200640 1 -- 172.21.15.114:0/4124620622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f477c1593b0 msgr2=0x7f477c142720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:18.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.228+0000 7f4782200640 1 --2- 172.21.15.114:0/4124620622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f477c1593b0 0x7f477c142720 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f476c0097a0 tx=0x7f476c02a2c0 comp rx=0 tx=0).stop 2024-05-27T09:37:18.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.228+0000 7f4782200640 1 -- 172.21.15.114:0/4124620622 shutdown_connections 2024-05-27T09:37:18.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.228+0000 7f4782200640 1 --2- 172.21.15.114:0/4124620622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f477c1593b0 0x7f477c142720 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:18.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.228+0000 7f4782200640 1 -- 172.21.15.114:0/4124620622 >> 172.21.15.114:0/4124620622 conn(0x7f477c06f6e0 msgr2=0x7f477c070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:18.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.228+0000 7f4782200640 1 -- 172.21.15.114:0/4124620622 shutdown_connections 2024-05-27T09:37:18.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.228+0000 7f4782200640 1 -- 172.21.15.114:0/4124620622 wait complete. 2024-05-27T09:37:18.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.230+0000 7f4782200640 1 Processor -- start 2024-05-27T09:37:18.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.231+0000 7f4782200640 1 -- start start 2024-05-27T09:37:18.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.232+0000 7f4782200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f477c073260 0x7f477c1f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:18.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.232+0000 7f4782200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f477c0c2990 0x7f477c0ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:18.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.232+0000 7f4782200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f477c1593b0 0x7f477c1f3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:18.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.232+0000 7f4782200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f477c074800 con 0x7f477c1593b0 2024-05-27T09:37:18.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.232+0000 7f4782200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f477c074520 con 0x7f477c0c2990 2024-05-27T09:37:18.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.232+0000 7f4782200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f477c073670 con 0x7f477c073260 2024-05-27T09:37:18.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.232+0000 7f477be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f477c1593b0 0x7f477c1f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:18.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.232+0000 7f477b400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f477c073260 0x7f477c1f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:18.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.232+0000 7f477be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f477c1593b0 0x7f477c1f3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33344/0 (socket says 172.21.15.114:33344) 2024-05-27T09:37:18.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.232+0000 7f477be00640 1 -- 172.21.15.114:0/1994804441 learned_addr learned my addr 172.21.15.114:0/1994804441 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:18.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.232+0000 7f4772a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f477c0c2990 0x7f477c0ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:18.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.233+0000 7f477b400640 1 -- 172.21.15.114:0/1994804441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f477c0c2990 msgr2=0x7f477c0ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:18.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.233+0000 7f477b400640 1 --2- 172.21.15.114:0/1994804441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f477c0c2990 0x7f477c0ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:18.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.233+0000 7f477b400640 1 -- 172.21.15.114:0/1994804441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f477c1593b0 msgr2=0x7f477c1f3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:18.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.233+0000 7f477b400640 1 --2- 172.21.15.114:0/1994804441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f477c1593b0 0x7f477c1f3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:18.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.233+0000 7f477b400640 1 -- 172.21.15.114:0/1994804441 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477c0ca540 con 0x7f477c073260 2024-05-27T09:37:18.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.233+0000 7f477be00640 1 --2- 172.21.15.114:0/1994804441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f477c1593b0 0x7f477c1f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:18.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.233+0000 7f477b400640 1 --2- 172.21.15.114:0/1994804441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f477c073260 0x7f477c1f2c90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f476c031000 tx=0x7f476c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:18.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.233+0000 7f4773e00640 1 -- 172.21.15.114:0/1994804441 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c037040 con 0x7f477c073260 2024-05-27T09:37:18.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.233+0000 7f4773e00640 1 -- 172.21.15.114:0/1994804441 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f476c030770 con 0x7f477c073260 2024-05-27T09:37:18.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.234+0000 7f4782200640 1 -- 172.21.15.114:0/1994804441 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f477c1e8c20 con 0x7f477c073260 2024-05-27T09:37:18.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.234+0000 7f4773e00640 1 -- 172.21.15.114:0/1994804441 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c030b60 con 0x7f477c073260 2024-05-27T09:37:18.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.234+0000 7f4782200640 1 -- 172.21.15.114:0/1994804441 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f477c1f6d70 con 0x7f477c073260 2024-05-27T09:37:18.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.235+0000 7f4782200640 1 -- 172.21.15.114:0/1994804441 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f477c0692a0 con 0x7f477c073260 2024-05-27T09:37:18.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.236+0000 7f4773e00640 1 -- 172.21.15.114:0/1994804441 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f476c044030 con 0x7f477c073260 2024-05-27T09:37:18.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.237+0000 7f4773e00640 1 --2- 172.21.15.114:0/1994804441 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f47580762d0 0x7f47580787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:18.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.237+0000 7f4773e00640 1 -- 172.21.15.114:0/1994804441 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7f476c0b7870 con 0x7f477c073260 2024-05-27T09:37:18.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.237+0000 7f4772a00640 1 --2- 172.21.15.114:0/1994804441 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f47580762d0 0x7f47580787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:18.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.238+0000 7f4772a00640 1 --2- 172.21.15.114:0/1994804441 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f47580762d0 0x7f47580787e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4764000f60 tx=0x7f4764009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:18.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.242+0000 7f4773e00640 1 -- 172.21.15.114:0/1994804441 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f477c0692a0 con 0x7f477c073260 2024-05-27T09:37:18.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.487+0000 7f4782200640 1 -- 172.21.15.114:0/1994804441 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f477c06a380 con 0x7f477c073260 2024-05-27T09:37:18.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.489+0000 7f4773e00640 1 -- 172.21.15.114:0/1994804441 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6056 (secure 0 0 0) 0x7f476c04b060 con 0x7f477c073260 2024-05-27T09:37:18.491 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:37:18.492 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:37:15.517199+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":5072,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5072":{"gid":5072,"name":"f","rank":1,"incarnation":43,"state":"up:replay","state_seq":1,"addr":"172.21.15.163:6839/1057092553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":1057092553},{"type":"v1","addr":"172.21.15.163:6839","nonce":1057092553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:active","state_seq":26,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086,5160]},"id":1}]} 2024-05-27T09:37:18.492 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 43 2024-05-27T09:37:18.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.491+0000 7f4782200640 1 -- 172.21.15.114:0/1994804441 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f47580762d0 msgr2=0x7f47580787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:18.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.491+0000 7f4782200640 1 --2- 172.21.15.114:0/1994804441 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f47580762d0 0x7f47580787e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4764000f60 tx=0x7f4764009040 comp rx=0 tx=0).stop 2024-05-27T09:37:18.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.491+0000 7f4782200640 1 -- 172.21.15.114:0/1994804441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f477c073260 msgr2=0x7f477c1f2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:18.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.491+0000 7f4782200640 1 --2- 172.21.15.114:0/1994804441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f477c073260 0x7f477c1f2c90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f476c031000 tx=0x7f476c0046c0 comp rx=0 tx=0).stop 2024-05-27T09:37:18.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.491+0000 7f4782200640 1 -- 172.21.15.114:0/1994804441 shutdown_connections 2024-05-27T09:37:18.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.491+0000 7f4782200640 1 --2- 172.21.15.114:0/1994804441 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f47580762d0 0x7f47580787e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:18.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.491+0000 7f4782200640 1 --2- 172.21.15.114:0/1994804441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f477c1593b0 0x7f477c1f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:18.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.491+0000 7f4782200640 1 --2- 172.21.15.114:0/1994804441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f477c0c2990 0x7f477c0ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:18.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.491+0000 7f4782200640 1 --2- 172.21.15.114:0/1994804441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f477c073260 0x7f477c1f2c90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:18.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.491+0000 7f4782200640 1 -- 172.21.15.114:0/1994804441 >> 172.21.15.114:0/1994804441 conn(0x7f477c06f6e0 msgr2=0x7f477c152fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:18.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.491+0000 7f4782200640 1 -- 172.21.15.114:0/1994804441 shutdown_connections 2024-05-27T09:37:18.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:18.491+0000 7f4782200640 1 -- 172.21.15.114:0/1994804441 wait complete. 2024-05-27T09:37:18.521 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 43, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-05-27T09:27:18.079214+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 43, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5072": { "addr": "172.21.15.163:6839/1057092553", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6837", "nonce": 1057092553, "type": "v2" }, { "addr": "172.21.15.163:6839", "nonce": 1057092553, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5072, "incarnation": 43, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:replay", "state_seq": 1 }, "gid_5086": { "addr": "172.21.15.114:6834/1371544448", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6832", "nonce": 1371544448, "type": "v2" }, { "addr": "172.21.15.114:6834", "nonce": 1371544448, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5086, "incarnation": 31, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 27 }, "gid_5160": { "addr": "172.21.15.114:6837/1128209239", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6836", "nonce": 1128209239, "type": "v2" }, { "addr": "172.21.15.114:6837", "nonce": 1128209239, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5160, "incarnation": 37, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 26 } }, "last_failure": 0, "last_failure_osd_epoch": 50, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-05-27T09:37:15.517199+0000", "qdb_cluster": [ 5086, 5160 ], "qdb_leader": 5086, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5086, "mds_1": 5072, "mds_2": 5160 } } } ], "standbys": [ { "addr": "172.21.15.163:6835/3513522318", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 3513522318, "type": "v2" }, { "addr": "172.21.15.163:6835", "nonce": 3513522318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6838/1753596112", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6836", "nonce": 1753596112, "type": "v2" }, { "addr": "172.21.15.163:6838", "nonce": 1753596112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-05-27T09:37:20.523 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:37:20.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.779+0000 7f5be0a00640 1 Processor -- start 2024-05-27T09:37:20.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.779+0000 7f5be0a00640 1 -- start start 2024-05-27T09:37:20.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.780+0000 7f5be0a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5bdc0741f0 0x7f5bdc0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:20.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.780+0000 7f5be0a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5bdc1547d0 0x7f5bdc15ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:20.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.780+0000 7f5be0a00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5bdc074840 0x7f5bdc06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:20.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.780+0000 7f5be0a00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f5bdc0703d0 con 0x7f5bdc0741f0 2024-05-27T09:37:20.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.781+0000 7f5be0a00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f5bdc15b260 con 0x7f5bdc074840 2024-05-27T09:37:20.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.781+0000 7f5be0a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f5bdc06fe50 con 0x7f5bdc1547d0 2024-05-27T09:37:20.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.781+0000 7f5bdb400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5bdc0741f0 0x7f5bdc0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:20.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.781+0000 7f5bdb400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5bdc0741f0 0x7f5bdc0c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33360/0 (socket says 172.21.15.114:33360) 2024-05-27T09:37:20.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.781+0000 7f5bdb400640 1 -- 172.21.15.114:0/3201969886 learned_addr learned my addr 172.21.15.114:0/3201969886 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:20.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.781+0000 7f5bdaa00640 1 --2- 172.21.15.114:0/3201969886 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5bdc1547d0 0x7f5bdc15ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:20.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.781+0000 7f5bdbe00640 1 --2- 172.21.15.114:0/3201969886 >> v2:172.21.15.163:3300/0 conn(0x7f5bdc074840 0x7f5bdc06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:20.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.781+0000 7f5bdaa00640 1 -- 172.21.15.114:0/3201969886 >> v2:172.21.15.163:3300/0 conn(0x7f5bdc074840 msgr2=0x7f5bdc06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:20.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.781+0000 7f5bdaa00640 1 --2- 172.21.15.114:0/3201969886 >> v2:172.21.15.163:3300/0 conn(0x7f5bdc074840 0x7f5bdc06edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:20.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.781+0000 7f5bdaa00640 1 -- 172.21.15.114:0/3201969886 >> v2:172.21.15.114:3300/0 conn(0x7f5bdc0741f0 msgr2=0x7f5bdc0c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:20.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.781+0000 7f5bdaa00640 1 --2- 172.21.15.114:0/3201969886 >> v2:172.21.15.114:3300/0 conn(0x7f5bdc0741f0 0x7f5bdc0c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:20.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.781+0000 7f5bdaa00640 1 -- 172.21.15.114:0/3201969886 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bdc0700f0 con 0x7f5bdc1547d0 2024-05-27T09:37:20.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.781+0000 7f5bdbe00640 1 --2- 172.21.15.114:0/3201969886 >> v2:172.21.15.163:3300/0 conn(0x7f5bdc074840 0x7f5bdc06edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:20.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.782+0000 7f5bdb400640 1 --2- 172.21.15.114:0/3201969886 >> v2:172.21.15.114:3300/0 conn(0x7f5bdc0741f0 0x7f5bdc0c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:20.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.783+0000 7f5bdaa00640 1 --2- 172.21.15.114:0/3201969886 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5bdc1547d0 0x7f5bdc15ad20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0b31e0 tx=0x7f5bc802a1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d7467f914dd27ad server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:20.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.783+0000 7f5bda000640 1 -- 172.21.15.114:0/3201969886 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8037070 con 0x7f5bdc1547d0 2024-05-27T09:37:20.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.783+0000 7f5bda000640 1 -- 172.21.15.114:0/3201969886 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc8032440 con 0x7f5bdc1547d0 2024-05-27T09:37:20.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.783+0000 7f5bda000640 1 -- 172.21.15.114:0/3201969886 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8032810 con 0x7f5bdc1547d0 2024-05-27T09:37:20.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.784+0000 7f5be0a00640 1 -- 172.21.15.114:0/3201969886 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5bdc1547d0 msgr2=0x7f5bdc15ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:20.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.784+0000 7f5be0a00640 1 --2- 172.21.15.114:0/3201969886 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5bdc1547d0 0x7f5bdc15ad20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0b31e0 tx=0x7f5bc802a1d0 comp rx=0 tx=0).stop 2024-05-27T09:37:20.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.784+0000 7f5be0a00640 1 -- 172.21.15.114:0/3201969886 shutdown_connections 2024-05-27T09:37:20.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.784+0000 7f5be0a00640 1 --2- 172.21.15.114:0/3201969886 >> v2:172.21.15.163:3300/0 conn(0x7f5bdc074840 0x7f5bdc06edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:20.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.784+0000 7f5be0a00640 1 --2- 172.21.15.114:0/3201969886 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5bdc1547d0 0x7f5bdc15ad20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:20.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.784+0000 7f5be0a00640 1 --2- 172.21.15.114:0/3201969886 >> v2:172.21.15.114:3300/0 conn(0x7f5bdc0741f0 0x7f5bdc0c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:20.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.784+0000 7f5be0a00640 1 -- 172.21.15.114:0/3201969886 >> 172.21.15.114:0/3201969886 conn(0x7f5bdc0ca320 msgr2=0x7f5bdc0ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:20.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.784+0000 7f5be0a00640 1 -- 172.21.15.114:0/3201969886 shutdown_connections 2024-05-27T09:37:20.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.784+0000 7f5be0a00640 1 -- 172.21.15.114:0/3201969886 wait complete. 2024-05-27T09:37:20.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.786+0000 7f5be0a00640 1 Processor -- start 2024-05-27T09:37:20.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.787+0000 7f5be0a00640 1 -- start start 2024-05-27T09:37:20.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.789+0000 7f5be0a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bdc0741f0 0x7f5bdc155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:20.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.789+0000 7f5be0a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5bdc074840 0x7f5bdc1f2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:20.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.789+0000 7f5be0a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bdc1547d0 0x7f5bdc147390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:20.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.789+0000 7f5be0a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5bdc073790 con 0x7f5bdc0741f0 2024-05-27T09:37:20.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.789+0000 7f5be0a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5bdc0734b0 con 0x7f5bdc1547d0 2024-05-27T09:37:20.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.789+0000 7f5be0a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f5bdc073cb0 con 0x7f5bdc074840 2024-05-27T09:37:20.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.789+0000 7f5bdb400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bdc0741f0 0x7f5bdc155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:20.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.789+0000 7f5bdb400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bdc0741f0 0x7f5bdc155b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33368/0 (socket says 172.21.15.114:33368) 2024-05-27T09:37:20.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.789+0000 7f5bdb400640 1 -- 172.21.15.114:0/173058318 learned_addr learned my addr 172.21.15.114:0/173058318 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:20.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.790+0000 7f5bdaa00640 1 --2- 172.21.15.114:0/173058318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5bdc074840 0x7f5bdc1f2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:20.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.790+0000 7f5bdbe00640 1 --2- 172.21.15.114:0/173058318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bdc1547d0 0x7f5bdc147390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:20.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.790+0000 7f5bdaa00640 1 -- 172.21.15.114:0/173058318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bdc1547d0 msgr2=0x7f5bdc147390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:20.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.790+0000 7f5bdaa00640 1 --2- 172.21.15.114:0/173058318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bdc1547d0 0x7f5bdc147390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:20.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.790+0000 7f5bdaa00640 1 -- 172.21.15.114:0/173058318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bdc0741f0 msgr2=0x7f5bdc155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:20.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.790+0000 7f5bdaa00640 1 --2- 172.21.15.114:0/173058318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bdc0741f0 0x7f5bdc155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:20.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.790+0000 7f5bdaa00640 1 -- 172.21.15.114:0/173058318 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bdc14d200 con 0x7f5bdc074840 2024-05-27T09:37:20.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.790+0000 7f5bdb400640 1 --2- 172.21.15.114:0/173058318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bdc0741f0 0x7f5bdc155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:20.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.790+0000 7f5bdaa00640 1 --2- 172.21.15.114:0/173058318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5bdc074840 0x7f5bdc1f2fd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8031000 tx=0x7f5bc80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:20.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.791+0000 7f5bd3400640 1 -- 172.21.15.114:0/173058318 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8041020 con 0x7f5bdc074840 2024-05-27T09:37:20.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.791+0000 7f5bd3400640 1 -- 172.21.15.114:0/173058318 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc803d640 con 0x7f5bdc074840 2024-05-27T09:37:20.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.791+0000 7f5be0a00640 1 -- 172.21.15.114:0/173058318 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bdc14d4a0 con 0x7f5bdc074840 2024-05-27T09:37:20.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.791+0000 7f5bd3400640 1 -- 172.21.15.114:0/173058318 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8037040 con 0x7f5bdc074840 2024-05-27T09:37:20.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.791+0000 7f5be0a00640 1 -- 172.21.15.114:0/173058318 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bdc0c4110 con 0x7f5bdc074840 2024-05-27T09:37:20.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.793+0000 7f5be0a00640 1 -- 172.21.15.114:0/173058318 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bdc0c3c50 con 0x7f5bdc074840 2024-05-27T09:37:20.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.793+0000 7f5bd3400640 1 -- 172.21.15.114:0/173058318 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5bc8044020 con 0x7f5bdc074840 2024-05-27T09:37:20.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.794+0000 7f5bd3400640 1 --2- 172.21.15.114:0/173058318 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5bbc0762d0 0x7f5bbc0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:20.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.794+0000 7f5bd3400640 1 -- 172.21.15.114:0/173058318 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7f5bc80b7c10 con 0x7f5bdc074840 2024-05-27T09:37:20.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.795+0000 7f5bdb400640 1 --2- 172.21.15.114:0/173058318 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5bbc0762d0 0x7f5bbc0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:20.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.795+0000 7f5bdb400640 1 --2- 172.21.15.114:0/173058318 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5bbc0762d0 0x7f5bbc0787e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4009fd0 tx=0x7f5bd400d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:20.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:20.799+0000 7f5bd3400640 1 -- 172.21.15.114:0/173058318 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5bdc0c3c50 con 0x7f5bdc074840 2024-05-27T09:37:21.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:21.062+0000 7f5be0a00640 1 -- 172.21.15.114:0/173058318 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5bdc0767d0 con 0x7f5bdc074840 2024-05-27T09:37:21.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:21.063+0000 7f5bd3400640 1 -- 172.21.15.114:0/173058318 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6056 (secure 0 0 0) 0x7f5bc804b070 con 0x7f5bdc074840 2024-05-27T09:37:21.065 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:37:21.065 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:37:15.517199+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":5072,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5072":{"gid":5072,"name":"f","rank":1,"incarnation":43,"state":"up:replay","state_seq":1,"addr":"172.21.15.163:6839/1057092553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":1057092553},{"type":"v1","addr":"172.21.15.163:6839","nonce":1057092553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:active","state_seq":26,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086,5160]},"id":1}]} 2024-05-27T09:37:21.065 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 43 2024-05-27T09:37:21.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:21.065+0000 7f5be0a00640 1 -- 172.21.15.114:0/173058318 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5bbc0762d0 msgr2=0x7f5bbc0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:21.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:21.065+0000 7f5be0a00640 1 --2- 172.21.15.114:0/173058318 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5bbc0762d0 0x7f5bbc0787e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4009fd0 tx=0x7f5bd400d040 comp rx=0 tx=0).stop 2024-05-27T09:37:21.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:21.065+0000 7f5be0a00640 1 -- 172.21.15.114:0/173058318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5bdc074840 msgr2=0x7f5bdc1f2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:21.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:21.065+0000 7f5be0a00640 1 --2- 172.21.15.114:0/173058318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5bdc074840 0x7f5bdc1f2fd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8031000 tx=0x7f5bc80046c0 comp rx=0 tx=0).stop 2024-05-27T09:37:21.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:21.066+0000 7f5be0a00640 1 -- 172.21.15.114:0/173058318 shutdown_connections 2024-05-27T09:37:21.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:21.066+0000 7f5be0a00640 1 --2- 172.21.15.114:0/173058318 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f5bbc0762d0 0x7f5bbc0787e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:21.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:21.066+0000 7f5be0a00640 1 --2- 172.21.15.114:0/173058318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bdc1547d0 0x7f5bdc147390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:21.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:21.066+0000 7f5be0a00640 1 --2- 172.21.15.114:0/173058318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5bdc074840 0x7f5bdc1f2fd0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:21.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:21.066+0000 7f5be0a00640 1 --2- 172.21.15.114:0/173058318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5bdc0741f0 0x7f5bdc155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:21.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:21.066+0000 7f5be0a00640 1 -- 172.21.15.114:0/173058318 >> 172.21.15.114:0/173058318 conn(0x7f5bdc0ca320 msgr2=0x7f5bdc152080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:21.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:21.066+0000 7f5be0a00640 1 -- 172.21.15.114:0/173058318 shutdown_connections 2024-05-27T09:37:21.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:21.066+0000 7f5be0a00640 1 -- 172.21.15.114:0/173058318 wait complete. 2024-05-27T09:37:21.089 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:37:21.300 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:37:21.308 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:37:23.090 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:37:23.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.373+0000 7fb34e400640 1 Processor -- start 2024-05-27T09:37:23.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.373+0000 7fb34e400640 1 -- start start 2024-05-27T09:37:23.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.375+0000 7fb34e400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb3480c2a00 0x7fb348159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:23.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.375+0000 7fb34e400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fb348159e40 0x7fb348074140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:23.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.376+0000 7fb34e400640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3480c96d0 con 0x7fb3481534c0 2024-05-27T09:37:23.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.376+0000 7fb34e400640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fb348159190 con 0x7fb348159e40 2024-05-27T09:37:23.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.376+0000 7fb34e400640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fb3481538d0 con 0x7fb3480c2a00 2024-05-27T09:37:23.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.376+0000 7fb34d000640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fb3481534c0 0x7fb3480c8220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:57180/0 (socket says 172.21.15.114:57180) 2024-05-27T09:37:23.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.376+0000 7fb34d000640 1 -- 172.21.15.114:0/1742598962 learned_addr learned my addr 172.21.15.114:0/1742598962 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:23.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.376+0000 7fb34da00640 1 --2- 172.21.15.114:0/1742598962 >> v2:172.21.15.114:3300/0 conn(0x7fb348159e40 0x7fb348074140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:23.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.376+0000 7fb347e00640 1 --2- 172.21.15.114:0/1742598962 >> v2:172.21.15.163:3300/0 conn(0x7fb3480c2a00 0x7fb348159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:23.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.376+0000 7fb34da00640 1 -- 172.21.15.114:0/1742598962 >> v2:172.21.15.163:3300/0 conn(0x7fb3480c2a00 msgr2=0x7fb348159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:23.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.376+0000 7fb34da00640 1 --2- 172.21.15.114:0/1742598962 >> v2:172.21.15.163:3300/0 conn(0x7fb3480c2a00 0x7fb348159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:23.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.376+0000 7fb34da00640 1 -- 172.21.15.114:0/1742598962 >> v1:172.21.15.114:6789/0 conn(0x7fb3481534c0 legacy=0x7fb3480c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:23.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.376+0000 7fb34da00640 1 -- 172.21.15.114:0/1742598962 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3480c8930 con 0x7fb348159e40 2024-05-27T09:37:23.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.376+0000 7fb347400640 1 -- 172.21.15.114:0/1742598962 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3798449164 0 0) 0x7fb3480c96d0 con 0x7fb3481534c0 2024-05-27T09:37:23.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.378+0000 7fb34da00640 1 --2- 172.21.15.114:0/1742598962 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb348159e40 0x7fb348074140 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb348071be0 tx=0x7fb34002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7113c677ce7d983 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:23.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.378+0000 7fb347400640 1 -- 172.21.15.114:0/1742598962 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb340007740 con 0x7fb348159e40 2024-05-27T09:37:23.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.378+0000 7fb347400640 1 -- 172.21.15.114:0/1742598962 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3400078e0 con 0x7fb348159e40 2024-05-27T09:37:23.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.378+0000 7fb347400640 1 -- 172.21.15.114:0/1742598962 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb340007cb0 con 0x7fb348159e40 2024-05-27T09:37:23.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.379+0000 7fb34e400640 1 -- 172.21.15.114:0/1742598962 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb348159e40 msgr2=0x7fb348074140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:23.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.379+0000 7fb34e400640 1 --2- 172.21.15.114:0/1742598962 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb348159e40 0x7fb348074140 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb348071be0 tx=0x7fb34002a680 comp rx=0 tx=0).stop 2024-05-27T09:37:23.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.379+0000 7fb34e400640 1 -- 172.21.15.114:0/1742598962 shutdown_connections 2024-05-27T09:37:23.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.379+0000 7fb34e400640 1 --2- 172.21.15.114:0/1742598962 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb348159e40 0x7fb348074140 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:23.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.379+0000 7fb34e400640 1 --2- 172.21.15.114:0/1742598962 >> v2:172.21.15.163:3300/0 conn(0x7fb3480c2a00 0x7fb348159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:23.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.379+0000 7fb34e400640 1 -- 172.21.15.114:0/1742598962 >> 172.21.15.114:0/1742598962 conn(0x7fb3480ca320 msgr2=0x7fb3480ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:23.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.379+0000 7fb34e400640 1 -- 172.21.15.114:0/1742598962 shutdown_connections 2024-05-27T09:37:23.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.379+0000 7fb34e400640 1 -- 172.21.15.114:0/1742598962 wait complete. 2024-05-27T09:37:23.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.381+0000 7fb34e400640 1 Processor -- start 2024-05-27T09:37:23.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.381+0000 7fb34e400640 1 -- start start 2024-05-27T09:37:23.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.383+0000 7fb34e400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3480c2a00 0x7fb348153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:23.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.383+0000 7fb34e400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3481534c0 0x7fb3481f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:23.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.383+0000 7fb34e400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb348159e40 0x7fb3480c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:23.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.383+0000 7fb34e400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb3480c9f30 con 0x7fb3481534c0 2024-05-27T09:37:23.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.383+0000 7fb34e400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb3480c9c50 con 0x7fb3480c2a00 2024-05-27T09:37:23.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.383+0000 7fb34e400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fb3481593e0 con 0x7fb348159e40 2024-05-27T09:37:23.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.383+0000 7fb347e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3481534c0 0x7fb3481f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:23.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.383+0000 7fb347e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3481534c0 0x7fb3481f3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33388/0 (socket says 172.21.15.114:33388) 2024-05-27T09:37:23.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.383+0000 7fb347e00640 1 -- 172.21.15.114:0/1203161985 learned_addr learned my addr 172.21.15.114:0/1203161985 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:23.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.383+0000 7fb34d000640 1 --2- 172.21.15.114:0/1203161985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3480c2a00 0x7fb348153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:23.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.383+0000 7fb34da00640 1 --2- 172.21.15.114:0/1203161985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb348159e40 0x7fb3480c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:23.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.384+0000 7fb347e00640 1 -- 172.21.15.114:0/1203161985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb348159e40 msgr2=0x7fb3480c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:23.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.384+0000 7fb347e00640 1 --2- 172.21.15.114:0/1203161985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb348159e40 0x7fb3480c6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:23.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.384+0000 7fb347e00640 1 -- 172.21.15.114:0/1203161985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3480c2a00 msgr2=0x7fb348153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:23.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.384+0000 7fb347e00640 1 --2- 172.21.15.114:0/1203161985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3480c2a00 0x7fb348153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:23.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.384+0000 7fb347e00640 1 -- 172.21.15.114:0/1203161985 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3481f3570 con 0x7fb3481534c0 2024-05-27T09:37:23.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.384+0000 7fb347e00640 1 --2- 172.21.15.114:0/1203161985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3481534c0 0x7fb3481f3030 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb33400e870 tx=0x7fb33400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:23.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.384+0000 7fb344c00640 1 -- 172.21.15.114:0/1203161985 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334017070 con 0x7fb3481534c0 2024-05-27T09:37:23.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.384+0000 7fb344c00640 1 -- 172.21.15.114:0/1203161985 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb33400c940 con 0x7fb3481534c0 2024-05-27T09:37:23.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.384+0000 7fb344c00640 1 -- 172.21.15.114:0/1203161985 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33400cd30 con 0x7fb3481534c0 2024-05-27T09:37:23.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.384+0000 7fb34e400640 1 -- 172.21.15.114:0/1203161985 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3481f6290 con 0x7fb3481534c0 2024-05-27T09:37:23.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.384+0000 7fb34e400640 1 -- 172.21.15.114:0/1203161985 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3481f6830 con 0x7fb3481534c0 2024-05-27T09:37:23.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.385+0000 7fb34e400640 1 -- 172.21.15.114:0/1203161985 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb348050860 con 0x7fb3481534c0 2024-05-27T09:37:23.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.387+0000 7fb344c00640 1 -- 172.21.15.114:0/1203161985 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb33401a070 con 0x7fb3481534c0 2024-05-27T09:37:23.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.388+0000 7fb344c00640 1 --2- 172.21.15.114:0/1203161985 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb3280762d0 0x7fb3280787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:23.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.389+0000 7fb344c00640 1 -- 172.21.15.114:0/1203161985 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7fb334097670 con 0x7fb3481534c0 2024-05-27T09:37:23.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.389+0000 7fb34d000640 1 --2- 172.21.15.114:0/1203161985 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb3280762d0 0x7fb3280787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:23.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.390+0000 7fb34d000640 1 --2- 172.21.15.114:0/1203161985 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb3280762d0 0x7fb3280787e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb33c007fd0 tx=0x7fb33c007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:23.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.394+0000 7fb344c00640 1 -- 172.21.15.114:0/1203161985 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb3340606c0 con 0x7fb3481534c0 2024-05-27T09:37:23.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.642+0000 7fb34e400640 1 -- 172.21.15.114:0/1203161985 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb348152760 con 0x7fb3481534c0 2024-05-27T09:37:23.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.643+0000 7fb344c00640 1 -- 172.21.15.114:0/1203161985 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+6058 (secure 0 0 0) 0x7fb3340649c0 con 0x7fb3481534c0 2024-05-27T09:37:23.645 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:37:23.645 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:37:20.541620+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":5072,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5072":{"gid":5072,"name":"f","rank":1,"incarnation":43,"state":"up:resolve","state_seq":21,"addr":"172.21.15.163:6839/1057092553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":1057092553},{"type":"v1","addr":"172.21.15.163:6839","nonce":1057092553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:active","state_seq":26,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5160,5086]},"id":1}]} 2024-05-27T09:37:23.645 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 44 2024-05-27T09:37:23.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.646+0000 7fb34e400640 1 -- 172.21.15.114:0/1203161985 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb3280762d0 msgr2=0x7fb3280787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:23.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.646+0000 7fb34e400640 1 --2- 172.21.15.114:0/1203161985 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb3280762d0 0x7fb3280787e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb33c007fd0 tx=0x7fb33c007de0 comp rx=0 tx=0).stop 2024-05-27T09:37:23.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.646+0000 7fb34e400640 1 -- 172.21.15.114:0/1203161985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3481534c0 msgr2=0x7fb3481f3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:23.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.646+0000 7fb34e400640 1 --2- 172.21.15.114:0/1203161985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3481534c0 0x7fb3481f3030 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb33400e870 tx=0x7fb33400ed30 comp rx=0 tx=0).stop 2024-05-27T09:37:23.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.646+0000 7fb34e400640 1 -- 172.21.15.114:0/1203161985 shutdown_connections 2024-05-27T09:37:23.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.646+0000 7fb34e400640 1 --2- 172.21.15.114:0/1203161985 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb3280762d0 0x7fb3280787e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:23.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.646+0000 7fb34e400640 1 --2- 172.21.15.114:0/1203161985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb348159e40 0x7fb3480c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:23.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.646+0000 7fb34e400640 1 --2- 172.21.15.114:0/1203161985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3481534c0 0x7fb3481f3030 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:23.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.646+0000 7fb34e400640 1 --2- 172.21.15.114:0/1203161985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3480c2a00 0x7fb348153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:23.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.646+0000 7fb34e400640 1 -- 172.21.15.114:0/1203161985 >> 172.21.15.114:0/1203161985 conn(0x7fb3480ca320 msgr2=0x7fb34814f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:23.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.647+0000 7fb34e400640 1 -- 172.21.15.114:0/1203161985 shutdown_connections 2024-05-27T09:37:23.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:23.647+0000 7fb34e400640 1 -- 172.21.15.114:0/1203161985 wait complete. 2024-05-27T09:37:23.675 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:37:25.676 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:37:25.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.947+0000 7fc7f1000640 1 Processor -- start 2024-05-27T09:37:25.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.947+0000 7fc7f1000640 1 -- start start 2024-05-27T09:37:25.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.949+0000 7fc7f1000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc7ec142720 0x7fc7ec073260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:25.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.949+0000 7fc7f1000640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc7ec0c2990 0x7fc7ec0c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:25.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.949+0000 7fc7f1000640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7ec0739a0 con 0x7fc7ec1593b0 2024-05-27T09:37:25.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.949+0000 7fc7f1000640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc7ec074d20 con 0x7fc7ec0c2990 2024-05-27T09:37:25.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.949+0000 7fc7f1000640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fc7ec142b30 con 0x7fc7ec142720 2024-05-27T09:37:25.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.949+0000 7fc7e9e00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fc7ec1593b0 0x7fc7ec15a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:48106/0 (socket says 172.21.15.114:48106) 2024-05-27T09:37:25.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.949+0000 7fc7e9e00640 1 -- 172.21.15.114:0/853001322 learned_addr learned my addr 172.21.15.114:0/853001322 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:25.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.949+0000 7fc7ea800640 1 --2- 172.21.15.114:0/853001322 >> v2:172.21.15.114:3300/0 conn(0x7fc7ec0c2990 0x7fc7ec0c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:25.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.950+0000 7fc7e9400640 1 --2- 172.21.15.114:0/853001322 >> v2:172.21.15.163:3300/0 conn(0x7fc7ec142720 0x7fc7ec073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:25.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.950+0000 7fc7e8a00640 1 -- 172.21.15.114:0/853001322 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 844857782 0 0) 0x7fc7ec0739a0 con 0x7fc7ec1593b0 2024-05-27T09:37:25.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.950+0000 7fc7ea800640 1 -- 172.21.15.114:0/853001322 >> v2:172.21.15.163:3300/0 conn(0x7fc7ec142720 msgr2=0x7fc7ec073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:25.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.950+0000 7fc7ea800640 1 --2- 172.21.15.114:0/853001322 >> v2:172.21.15.163:3300/0 conn(0x7fc7ec142720 0x7fc7ec073260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:25.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.950+0000 7fc7ea800640 1 -- 172.21.15.114:0/853001322 >> v1:172.21.15.114:6789/0 conn(0x7fc7ec1593b0 legacy=0x7fc7ec15a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:25.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.950+0000 7fc7ea800640 1 -- 172.21.15.114:0/853001322 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ec073c80 con 0x7fc7ec0c2990 2024-05-27T09:37:25.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.951+0000 7fc7ea800640 1 --2- 172.21.15.114:0/853001322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7ec0c2990 0x7fc7ec0c2da0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0caf80 tx=0x7fc7e402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41b4e9ee9228ad6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:25.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.951+0000 7fc7e8a00640 1 -- 172.21.15.114:0/853001322 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4007740 con 0x7fc7ec0c2990 2024-05-27T09:37:25.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.951+0000 7fc7e8a00640 1 -- 172.21.15.114:0/853001322 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7e40078e0 con 0x7fc7ec0c2990 2024-05-27T09:37:25.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.951+0000 7fc7e8a00640 1 -- 172.21.15.114:0/853001322 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4007cb0 con 0x7fc7ec0c2990 2024-05-27T09:37:25.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.952+0000 7fc7f1000640 1 -- 172.21.15.114:0/853001322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7ec0c2990 msgr2=0x7fc7ec0c2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:25.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.952+0000 7fc7f1000640 1 --2- 172.21.15.114:0/853001322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7ec0c2990 0x7fc7ec0c2da0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0caf80 tx=0x7fc7e402a710 comp rx=0 tx=0).stop 2024-05-27T09:37:25.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.952+0000 7fc7f1000640 1 -- 172.21.15.114:0/853001322 shutdown_connections 2024-05-27T09:37:25.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.952+0000 7fc7f1000640 1 --2- 172.21.15.114:0/853001322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7ec0c2990 0x7fc7ec0c2da0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:25.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.952+0000 7fc7f1000640 1 --2- 172.21.15.114:0/853001322 >> v2:172.21.15.163:3300/0 conn(0x7fc7ec142720 0x7fc7ec073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:25.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.952+0000 7fc7f1000640 1 -- 172.21.15.114:0/853001322 >> 172.21.15.114:0/853001322 conn(0x7fc7ec06f6e0 msgr2=0x7fc7ec070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:25.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.952+0000 7fc7f1000640 1 -- 172.21.15.114:0/853001322 shutdown_connections 2024-05-27T09:37:25.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.952+0000 7fc7f1000640 1 -- 172.21.15.114:0/853001322 wait complete. 2024-05-27T09:37:25.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.954+0000 7fc7f1000640 1 Processor -- start 2024-05-27T09:37:25.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.955+0000 7fc7f1000640 1 -- start start 2024-05-27T09:37:25.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.956+0000 7fc7f1000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7ec0c2990 0x7fc7ec0c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:25.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.956+0000 7fc7f1000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7ec142720 0x7fc7ec15aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:25.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.956+0000 7fc7f1000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7ec1593b0 0x7fc7ec15b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:25.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.956+0000 7fc7f1000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec074520 con 0x7fc7ec1593b0 2024-05-27T09:37:25.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.956+0000 7fc7f1000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec074240 con 0x7fc7ec142720 2024-05-27T09:37:25.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.956+0000 7fc7f1000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fc7ec074800 con 0x7fc7ec0c2990 2024-05-27T09:37:25.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.956+0000 7fc7e9400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7ec142720 0x7fc7ec15aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:25.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.956+0000 7fc7ea800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7ec1593b0 0x7fc7ec15b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:25.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.956+0000 7fc7e9e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7ec0c2990 0x7fc7ec0c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:25.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.956+0000 7fc7e9400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7ec142720 0x7fc7ec15aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:52640/0 (socket says 172.21.15.114:52640) 2024-05-27T09:37:25.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.957+0000 7fc7e9400640 1 -- 172.21.15.114:0/4090728974 learned_addr learned my addr 172.21.15.114:0/4090728974 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:25.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.957+0000 7fc7e9400640 1 -- 172.21.15.114:0/4090728974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7ec0c2990 msgr2=0x7fc7ec0c99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:25.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.957+0000 7fc7e9400640 1 --2- 172.21.15.114:0/4090728974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7ec0c2990 0x7fc7ec0c99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:25.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.957+0000 7fc7e9400640 1 -- 172.21.15.114:0/4090728974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7ec1593b0 msgr2=0x7fc7ec15b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:25.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.957+0000 7fc7e9400640 1 --2- 172.21.15.114:0/4090728974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7ec1593b0 0x7fc7ec15b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:25.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.957+0000 7fc7e9400640 1 -- 172.21.15.114:0/4090728974 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ec153d30 con 0x7fc7ec142720 2024-05-27T09:37:25.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.957+0000 7fc7e9e00640 1 --2- 172.21.15.114:0/4090728974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7ec0c2990 0x7fc7ec0c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:25.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.957+0000 7fc7e9400640 1 --2- 172.21.15.114:0/4090728974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7ec142720 0x7fc7ec15aeb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800e870 tx=0x7fc7d800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:25.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.958+0000 7fc7e2000640 1 -- 172.21.15.114:0/4090728974 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8017070 con 0x7fc7ec142720 2024-05-27T09:37:25.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.958+0000 7fc7e2000640 1 -- 172.21.15.114:0/4090728974 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d800c940 con 0x7fc7ec142720 2024-05-27T09:37:25.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.958+0000 7fc7f1000640 1 -- 172.21.15.114:0/4090728974 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ec1542b0 con 0x7fc7ec142720 2024-05-27T09:37:25.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.958+0000 7fc7e2000640 1 -- 172.21.15.114:0/4090728974 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d800cd30 con 0x7fc7ec142720 2024-05-27T09:37:25.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.958+0000 7fc7f1000640 1 -- 172.21.15.114:0/4090728974 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ec1eea00 con 0x7fc7ec142720 2024-05-27T09:37:25.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.959+0000 7fc7f1000640 1 -- 172.21.15.114:0/4090728974 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7ec1ee550 con 0x7fc7ec142720 2024-05-27T09:37:25.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.960+0000 7fc7e2000640 1 -- 172.21.15.114:0/4090728974 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc7d801a070 con 0x7fc7ec142720 2024-05-27T09:37:25.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.961+0000 7fc7e2000640 1 --2- 172.21.15.114:0/4090728974 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc7cc0762d0 0x7fc7cc0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:25.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.961+0000 7fc7e2000640 1 -- 172.21.15.114:0/4090728974 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7fc7d80982d0 con 0x7fc7ec142720 2024-05-27T09:37:25.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.962+0000 7fc7e9e00640 1 --2- 172.21.15.114:0/4090728974 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc7cc0762d0 0x7fc7cc0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:25.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.962+0000 7fc7e9e00640 1 --2- 172.21.15.114:0/4090728974 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc7cc0762d0 0x7fc7cc0787e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc007fd0 tx=0x7fc7dc007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:25.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:25.965+0000 7fc7e2000640 1 -- 172.21.15.114:0/4090728974 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc7d8061310 con 0x7fc7ec142720 2024-05-27T09:37:26.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:26.210+0000 7fc7f1000640 1 -- 172.21.15.114:0/4090728974 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7ec06a380 con 0x7fc7ec142720 2024-05-27T09:37:26.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:26.212+0000 7fc7e2000640 1 -- 172.21.15.114:0/4090728974 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+6058 (secure 0 0 0) 0x7fc7d8065610 con 0x7fc7ec142720 2024-05-27T09:37:26.213 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:37:26.213 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:37:20.541620+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":5072,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5072":{"gid":5072,"name":"f","rank":1,"incarnation":43,"state":"up:resolve","state_seq":21,"addr":"172.21.15.163:6839/1057092553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":1057092553},{"type":"v1","addr":"172.21.15.163:6839","nonce":1057092553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:active","state_seq":26,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5160,5086]},"id":1}]} 2024-05-27T09:37:26.214 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 44 2024-05-27T09:37:26.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:26.214+0000 7fc7f1000640 1 -- 172.21.15.114:0/4090728974 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc7cc0762d0 msgr2=0x7fc7cc0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:26.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:26.214+0000 7fc7f1000640 1 --2- 172.21.15.114:0/4090728974 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc7cc0762d0 0x7fc7cc0787e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc007fd0 tx=0x7fc7dc007de0 comp rx=0 tx=0).stop 2024-05-27T09:37:26.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:26.214+0000 7fc7f1000640 1 -- 172.21.15.114:0/4090728974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7ec142720 msgr2=0x7fc7ec15aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:26.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:26.214+0000 7fc7f1000640 1 --2- 172.21.15.114:0/4090728974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7ec142720 0x7fc7ec15aeb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800e870 tx=0x7fc7d800ed30 comp rx=0 tx=0).stop 2024-05-27T09:37:26.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:26.214+0000 7fc7f1000640 1 -- 172.21.15.114:0/4090728974 shutdown_connections 2024-05-27T09:37:26.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:26.214+0000 7fc7f1000640 1 --2- 172.21.15.114:0/4090728974 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc7cc0762d0 0x7fc7cc0787e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:26.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:26.214+0000 7fc7f1000640 1 --2- 172.21.15.114:0/4090728974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc7ec1593b0 0x7fc7ec15b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:26.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:26.214+0000 7fc7f1000640 1 --2- 172.21.15.114:0/4090728974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7ec142720 0x7fc7ec15aeb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:26.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:26.214+0000 7fc7f1000640 1 --2- 172.21.15.114:0/4090728974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7ec0c2990 0x7fc7ec0c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:26.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:26.214+0000 7fc7f1000640 1 -- 172.21.15.114:0/4090728974 >> 172.21.15.114:0/4090728974 conn(0x7fc7ec06f6e0 msgr2=0x7fc7ec158bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:26.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:26.215+0000 7fc7f1000640 1 -- 172.21.15.114:0/4090728974 shutdown_connections 2024-05-27T09:37:26.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:26.215+0000 7fc7f1000640 1 -- 172.21.15.114:0/4090728974 wait complete. 2024-05-27T09:37:26.242 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:37:28.243 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:37:28.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.526+0000 7f6abf200640 1 Processor -- start 2024-05-27T09:37:28.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.527+0000 7f6abf200640 1 -- start start 2024-05-27T09:37:28.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.529+0000 7f6abf200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ac00741f0 0x7f6ac00c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:28.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.529+0000 7f6abf200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6ac0074840 0x7f6ac015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:28.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.529+0000 7f6abf200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ac006fe50 con 0x7f6ac01547d0 2024-05-27T09:37:28.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.529+0000 7f6abf200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f6ac00703d0 con 0x7f6ac0074840 2024-05-27T09:37:28.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.529+0000 7f6abf200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f6ac015b260 con 0x7f6ac00741f0 2024-05-27T09:37:28.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.529+0000 7f6abde00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ac00741f0 0x7f6ac00c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:28.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.529+0000 7f6abe800640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6ac0074840 0x7f6ac015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:28.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.529+0000 7f6abde00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ac00741f0 0x7f6ac00c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:52822/0 (socket says 172.21.15.114:52822) 2024-05-27T09:37:28.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.529+0000 7f6abde00640 1 -- 172.21.15.114:0/3273962913 learned_addr learned my addr 172.21.15.114:0/3273962913 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:28.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.530+0000 7f6abde00640 1 -- 172.21.15.114:0/3273962913 >> v2:172.21.15.163:3300/0 conn(0x7f6ac0074840 msgr2=0x7f6ac015ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:28.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.530+0000 7f6abde00640 1 --2- 172.21.15.114:0/3273962913 >> v2:172.21.15.163:3300/0 conn(0x7f6ac0074840 0x7f6ac015ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.530+0000 7f6abde00640 1 -- 172.21.15.114:0/3273962913 >> v1:172.21.15.114:6789/0 conn(0x7f6ac01547d0 legacy=0x7f6ac006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:28.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.530+0000 7f6abde00640 1 -- 172.21.15.114:0/3273962913 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac00700f0 con 0x7f6ac00741f0 2024-05-27T09:37:28.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.531+0000 7f6abde00640 1 --2- 172.21.15.114:0/3273962913 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ac00741f0 0x7f6ac00c63a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009870 tx=0x7f6ab002a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8627afd3967462ca server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:28.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.531+0000 7f6abca00640 1 -- 172.21.15.114:0/3273962913 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0037070 con 0x7f6ac00741f0 2024-05-27T09:37:28.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.531+0000 7f6abca00640 1 -- 172.21.15.114:0/3273962913 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ab002ae50 con 0x7f6ac00741f0 2024-05-27T09:37:28.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.531+0000 7f6abca00640 1 -- 172.21.15.114:0/3273962913 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0032860 con 0x7f6ac00741f0 2024-05-27T09:37:28.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.532+0000 7f6abf200640 1 -- 172.21.15.114:0/3273962913 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ac00741f0 msgr2=0x7f6ac00c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:28.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.532+0000 7f6abf200640 1 --2- 172.21.15.114:0/3273962913 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ac00741f0 0x7f6ac00c63a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009870 tx=0x7f6ab002a300 comp rx=0 tx=0).stop 2024-05-27T09:37:28.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.532+0000 7f6abf200640 1 -- 172.21.15.114:0/3273962913 shutdown_connections 2024-05-27T09:37:28.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.532+0000 7f6abf200640 1 --2- 172.21.15.114:0/3273962913 >> v2:172.21.15.163:3300/0 conn(0x7f6ac0074840 0x7f6ac015ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.532+0000 7f6abf200640 1 --2- 172.21.15.114:0/3273962913 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ac00741f0 0x7f6ac00c63a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.532+0000 7f6abf200640 1 -- 172.21.15.114:0/3273962913 >> 172.21.15.114:0/3273962913 conn(0x7f6ac00ca320 msgr2=0x7f6ac00ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:28.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.532+0000 7f6abf200640 1 -- 172.21.15.114:0/3273962913 shutdown_connections 2024-05-27T09:37:28.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.532+0000 7f6abf200640 1 -- 172.21.15.114:0/3273962913 wait complete. 2024-05-27T09:37:28.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.534+0000 7f6abf200640 1 Processor -- start 2024-05-27T09:37:28.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.534+0000 7f6abf200640 1 -- start start 2024-05-27T09:37:28.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.536+0000 7f6abf200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ac00741f0 0x7f6ac01597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:28.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.536+0000 7f6abf200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ac0074840 0x7f6ac014a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:28.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.536+0000 7f6abf200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ac01547d0 0x7f6ac00c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:28.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.536+0000 7f6abf200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac0073790 con 0x7f6ac0074840 2024-05-27T09:37:28.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.536+0000 7f6abf200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac00734b0 con 0x7f6ac00741f0 2024-05-27T09:37:28.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.536+0000 7f6abf200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f6ac006f500 con 0x7f6ac01547d0 2024-05-27T09:37:28.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.536+0000 7f6abd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ac0074840 0x7f6ac014a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:28.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.536+0000 7f6abd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ac0074840 0x7f6ac014a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45196/0 (socket says 172.21.15.114:45196) 2024-05-27T09:37:28.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.536+0000 7f6abd400640 1 -- 172.21.15.114:0/3098211776 learned_addr learned my addr 172.21.15.114:0/3098211776 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:28.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.536+0000 7f6abe800640 1 --2- 172.21.15.114:0/3098211776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ac01547d0 0x7f6ac00c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:28.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.536+0000 7f6abde00640 1 --2- 172.21.15.114:0/3098211776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ac00741f0 0x7f6ac01597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:28.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.537+0000 7f6abd400640 1 -- 172.21.15.114:0/3098211776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ac01547d0 msgr2=0x7f6ac00c7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:28.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.537+0000 7f6abd400640 1 --2- 172.21.15.114:0/3098211776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ac01547d0 0x7f6ac00c7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.537+0000 7f6abd400640 1 -- 172.21.15.114:0/3098211776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ac00741f0 msgr2=0x7f6ac01597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:28.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.537+0000 7f6abd400640 1 --2- 172.21.15.114:0/3098211776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ac00741f0 0x7f6ac01597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.537+0000 7f6abd400640 1 -- 172.21.15.114:0/3098211776 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac00c2840 con 0x7f6ac0074840 2024-05-27T09:37:28.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.537+0000 7f6abd400640 1 --2- 172.21.15.114:0/3098211776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ac0074840 0x7f6ac014a3a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800c760 tx=0x7f6aa800cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:28.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.537+0000 7f6ab6000640 1 -- 172.21.15.114:0/3098211776 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8016020 con 0x7f6ac0074840 2024-05-27T09:37:28.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.537+0000 7f6ab6000640 1 -- 172.21.15.114:0/3098211776 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aa8006d70 con 0x7f6ac0074840 2024-05-27T09:37:28.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.537+0000 7f6abf200640 1 -- 172.21.15.114:0/3098211776 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ac0159ce0 con 0x7f6ac0074840 2024-05-27T09:37:28.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.537+0000 7f6ab6000640 1 -- 172.21.15.114:0/3098211776 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8003bd0 con 0x7f6ac0074840 2024-05-27T09:37:28.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.538+0000 7f6abf200640 1 -- 172.21.15.114:0/3098211776 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ac01f3830 con 0x7f6ac0074840 2024-05-27T09:37:28.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.539+0000 7f6abf200640 1 -- 172.21.15.114:0/3098211776 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ac00b3250 con 0x7f6ac0074840 2024-05-27T09:37:28.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.540+0000 7f6ab6000640 1 -- 172.21.15.114:0/3098211776 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6aa80050d0 con 0x7f6ac0074840 2024-05-27T09:37:28.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.541+0000 7f6ab6000640 1 --2- 172.21.15.114:0/3098211776 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6a9c076300 0x7f6a9c078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:28.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.541+0000 7f6ab6000640 1 -- 172.21.15.114:0/3098211776 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7f6aa8097280 con 0x7f6ac0074840 2024-05-27T09:37:28.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.541+0000 7f6abde00640 1 --2- 172.21.15.114:0/3098211776 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6a9c076300 0x7f6a9c078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:28.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.542+0000 7f6abde00640 1 --2- 172.21.15.114:0/3098211776 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6a9c076300 0x7f6a9c078810 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0034000 tx=0x7f6ab002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:28.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.547+0000 7f6ab6000640 1 -- 172.21.15.114:0/3098211776 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6aa8020020 con 0x7f6ac0074840 2024-05-27T09:37:28.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.797+0000 7f6abf200640 1 -- 172.21.15.114:0/3098211776 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ac0157cb0 con 0x7f6ac0074840 2024-05-27T09:37:28.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.798+0000 7f6ab6000640 1 -- 172.21.15.114:0/3098211776 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+6060 (secure 0 0 0) 0x7f6aa80602e0 con 0x7f6ac0074840 2024-05-27T09:37:28.799 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:37:28.800 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:37:26.283162+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":5072,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5072":{"gid":5072,"name":"f","rank":1,"incarnation":43,"state":"up:reconnect","state_seq":23,"addr":"172.21.15.163:6839/1057092553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":1057092553},{"type":"v1","addr":"172.21.15.163:6839","nonce":1057092553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:active","state_seq":26,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5086,5160]},"id":1}]} 2024-05-27T09:37:28.800 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 45 2024-05-27T09:37:28.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.801+0000 7f6abf200640 1 -- 172.21.15.114:0/3098211776 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6a9c076300 msgr2=0x7f6a9c078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:28.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.801+0000 7f6abf200640 1 --2- 172.21.15.114:0/3098211776 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6a9c076300 0x7f6a9c078810 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0034000 tx=0x7f6ab002e000 comp rx=0 tx=0).stop 2024-05-27T09:37:28.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.801+0000 7f6abf200640 1 -- 172.21.15.114:0/3098211776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ac0074840 msgr2=0x7f6ac014a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:28.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.801+0000 7f6abf200640 1 --2- 172.21.15.114:0/3098211776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ac0074840 0x7f6ac014a3a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800c760 tx=0x7f6aa800cb60 comp rx=0 tx=0).stop 2024-05-27T09:37:28.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.801+0000 7f6abf200640 1 -- 172.21.15.114:0/3098211776 shutdown_connections 2024-05-27T09:37:28.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.801+0000 7f6abf200640 1 --2- 172.21.15.114:0/3098211776 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f6a9c076300 0x7f6a9c078810 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.801+0000 7f6abf200640 1 --2- 172.21.15.114:0/3098211776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ac01547d0 0x7f6ac00c7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.801+0000 7f6abf200640 1 --2- 172.21.15.114:0/3098211776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ac0074840 0x7f6ac014a3a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.801+0000 7f6abf200640 1 --2- 172.21.15.114:0/3098211776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ac00741f0 0x7f6ac01597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.801+0000 7f6abf200640 1 -- 172.21.15.114:0/3098211776 >> 172.21.15.114:0/3098211776 conn(0x7f6ac00ca320 msgr2=0x7f6ac01539d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:28.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.801+0000 7f6abf200640 1 -- 172.21.15.114:0/3098211776 shutdown_connections 2024-05-27T09:37:28.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:28.801+0000 7f6abf200640 1 -- 172.21.15.114:0/3098211776 wait complete. 2024-05-27T09:37:28.830 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:37:30.831 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:37:31.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.110+0000 7ffb82600640 1 Processor -- start 2024-05-27T09:37:31.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.110+0000 7ffb82600640 1 -- start start 2024-05-27T09:37:31.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.112+0000 7ffb82600640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffb7c0c2a00 0x7ffb7c159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:31.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.112+0000 7ffb82600640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ffb7c159e40 0x7ffb7c074140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:31.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.112+0000 7ffb82600640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb7c0c96d0 con 0x7ffb7c1534c0 2024-05-27T09:37:31.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.112+0000 7ffb82600640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7ffb7c159190 con 0x7ffb7c159e40 2024-05-27T09:37:31.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.112+0000 7ffb82600640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7ffb7c1538d0 con 0x7ffb7c0c2a00 2024-05-27T09:37:31.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.113+0000 7ffb81c00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ffb7c159e40 0x7ffb7c074140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:31.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.113+0000 7ffb81c00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ffb7c159e40 0x7ffb7c074140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45198/0 (socket says 172.21.15.114:45198) 2024-05-27T09:37:31.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.113+0000 7ffb81c00640 1 -- 172.21.15.114:0/1759407146 learned_addr learned my addr 172.21.15.114:0/1759407146 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:31.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.113+0000 7ffb73e00640 1 --2- 172.21.15.114:0/1759407146 >> v2:172.21.15.163:3300/0 conn(0x7ffb7c0c2a00 0x7ffb7c159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:31.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.113+0000 7ffb81c00640 1 -- 172.21.15.114:0/1759407146 >> v2:172.21.15.163:3300/0 conn(0x7ffb7c0c2a00 msgr2=0x7ffb7c159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:31.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.113+0000 7ffb81c00640 1 --2- 172.21.15.114:0/1759407146 >> v2:172.21.15.163:3300/0 conn(0x7ffb7c0c2a00 0x7ffb7c159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:31.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.113+0000 7ffb81c00640 1 -- 172.21.15.114:0/1759407146 >> v1:172.21.15.163:6789/0 conn(0x7ffb7c1534c0 legacy=0x7ffb7c0c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:31.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.113+0000 7ffb81c00640 1 -- 172.21.15.114:0/1759407146 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7c0c8930 con 0x7ffb7c159e40 2024-05-27T09:37:31.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.114+0000 7ffb81c00640 1 --2- 172.21.15.114:0/1759407146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb7c159e40 0x7ffb7c074140 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c071be0 tx=0x7ffb7402c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef0412a02cf5d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:31.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.114+0000 7ffb7be00640 1 -- 172.21.15.114:0/1759407146 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7400b2e0 con 0x7ffb7c159e40 2024-05-27T09:37:31.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.114+0000 7ffb7be00640 1 -- 172.21.15.114:0/1759407146 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb74037070 con 0x7ffb7c159e40 2024-05-27T09:37:31.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.114+0000 7ffb7be00640 1 -- 172.21.15.114:0/1759407146 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7400b480 con 0x7ffb7c159e40 2024-05-27T09:37:31.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.115+0000 7ffb82600640 1 -- 172.21.15.114:0/1759407146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb7c159e40 msgr2=0x7ffb7c074140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:31.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.115+0000 7ffb82600640 1 --2- 172.21.15.114:0/1759407146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb7c159e40 0x7ffb7c074140 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c071be0 tx=0x7ffb7402c940 comp rx=0 tx=0).stop 2024-05-27T09:37:31.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.115+0000 7ffb82600640 1 -- 172.21.15.114:0/1759407146 shutdown_connections 2024-05-27T09:37:31.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.115+0000 7ffb82600640 1 --2- 172.21.15.114:0/1759407146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb7c159e40 0x7ffb7c074140 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:31.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.115+0000 7ffb82600640 1 --2- 172.21.15.114:0/1759407146 >> v2:172.21.15.163:3300/0 conn(0x7ffb7c0c2a00 0x7ffb7c159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:31.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.115+0000 7ffb82600640 1 -- 172.21.15.114:0/1759407146 >> 172.21.15.114:0/1759407146 conn(0x7ffb7c0ca320 msgr2=0x7ffb7c0ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:31.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.115+0000 7ffb82600640 1 -- 172.21.15.114:0/1759407146 shutdown_connections 2024-05-27T09:37:31.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.115+0000 7ffb82600640 1 -- 172.21.15.114:0/1759407146 wait complete. 2024-05-27T09:37:31.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.118+0000 7ffb82600640 1 Processor -- start 2024-05-27T09:37:31.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.118+0000 7ffb82600640 1 -- start start 2024-05-27T09:37:31.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.119+0000 7ffb82600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb7c0c2a00 0x7ffb7c156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:31.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.119+0000 7ffb82600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb7c1534c0 0x7ffb7c149c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:31.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.119+0000 7ffb82600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb7c159e40 0x7ffb7c14a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:31.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.119+0000 7ffb82600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c0c9f30 con 0x7ffb7c1534c0 2024-05-27T09:37:31.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.119+0000 7ffb82600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c0c9c50 con 0x7ffb7c159e40 2024-05-27T09:37:31.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.119+0000 7ffb82600640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7ffb7c1593e0 con 0x7ffb7c0c2a00 2024-05-27T09:37:31.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.120+0000 7ffb73e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb7c1534c0 0x7ffb7c149c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:31.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.120+0000 7ffb81200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb7c0c2a00 0x7ffb7c156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:31.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.120+0000 7ffb81c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb7c159e40 0x7ffb7c14a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:31.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.120+0000 7ffb81200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb7c0c2a00 0x7ffb7c156dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:52844/0 (socket says 172.21.15.114:52844) 2024-05-27T09:37:31.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.120+0000 7ffb81200640 1 -- 172.21.15.114:0/1819132027 learned_addr learned my addr 172.21.15.114:0/1819132027 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:31.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.120+0000 7ffb81200640 1 -- 172.21.15.114:0/1819132027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb7c159e40 msgr2=0x7ffb7c14a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:31.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.120+0000 7ffb81200640 1 --2- 172.21.15.114:0/1819132027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb7c159e40 0x7ffb7c14a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:31.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.120+0000 7ffb81200640 1 -- 172.21.15.114:0/1819132027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb7c1534c0 msgr2=0x7ffb7c149c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:31.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.120+0000 7ffb81200640 1 --2- 172.21.15.114:0/1819132027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb7c1534c0 0x7ffb7c149c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:31.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.120+0000 7ffb81200640 1 -- 172.21.15.114:0/1819132027 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7c14e9e0 con 0x7ffb7c0c2a00 2024-05-27T09:37:31.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.120+0000 7ffb73e00640 1 --2- 172.21.15.114:0/1819132027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb7c1534c0 0x7ffb7c149c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:31.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.120+0000 7ffb81200640 1 --2- 172.21.15.114:0/1819132027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb7c0c2a00 0x7ffb7c156dc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00c3d0 tx=0x7ffb6c00c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:31.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.121+0000 7ffb79600640 1 -- 172.21.15.114:0/1819132027 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c017070 con 0x7ffb7c0c2a00 2024-05-27T09:37:31.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.121+0000 7ffb79600640 1 -- 172.21.15.114:0/1819132027 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb6c0052d0 con 0x7ffb7c0c2a00 2024-05-27T09:37:31.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.121+0000 7ffb79600640 1 -- 172.21.15.114:0/1819132027 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c003830 con 0x7ffb7c0c2a00 2024-05-27T09:37:31.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.121+0000 7ffb82600640 1 -- 172.21.15.114:0/1819132027 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb7c157300 con 0x7ffb7c0c2a00 2024-05-27T09:37:31.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.121+0000 7ffb82600640 1 -- 172.21.15.114:0/1819132027 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb7c148bb0 con 0x7ffb7c0c2a00 2024-05-27T09:37:31.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.122+0000 7ffb82600640 1 -- 172.21.15.114:0/1819132027 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb7c050860 con 0x7ffb7c0c2a00 2024-05-27T09:37:31.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.123+0000 7ffb79600640 1 -- 172.21.15.114:0/1819132027 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ffb6c01a070 con 0x7ffb7c0c2a00 2024-05-27T09:37:31.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.124+0000 7ffb79600640 1 --2- 172.21.15.114:0/1819132027 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ffb58076300 0x7ffb58078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:31.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.124+0000 7ffb79600640 1 -- 172.21.15.114:0/1819132027 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7ffb6c096ef0 con 0x7ffb7c0c2a00 2024-05-27T09:37:31.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.125+0000 7ffb73e00640 1 --2- 172.21.15.114:0/1819132027 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ffb58076300 0x7ffb58078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:31.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.125+0000 7ffb73e00640 1 --2- 172.21.15.114:0/1819132027 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ffb58076300 0x7ffb58078810 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c06f060 tx=0x7ffb64009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:31.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.128+0000 7ffb79600640 1 -- 172.21.15.114:0/1819132027 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ffb6c03a080 con 0x7ffb7c0c2a00 2024-05-27T09:37:31.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.375+0000 7ffb82600640 1 -- 172.21.15.114:0/1819132027 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb7c0c40f0 con 0x7ffb7c0c2a00 2024-05-27T09:37:31.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.376+0000 7ffb79600640 1 -- 172.21.15.114:0/1819132027 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+6057 (secure 0 0 0) 0x7ffb6c05ff30 con 0x7ffb7c0c2a00 2024-05-27T09:37:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:37:31.376 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:37:31.305177+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":5072,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5072":{"gid":5072,"name":"f","rank":1,"incarnation":43,"state":"up:rejoin","state_seq":25,"addr":"172.21.15.163:6839/1057092553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":1057092553},{"type":"v1","addr":"172.21.15.163:6839","nonce":1057092553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:active","state_seq":26,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5160,5086]},"id":1}]} 2024-05-27T09:37:31.377 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 46 2024-05-27T09:37:31.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.378+0000 7ffb82600640 1 -- 172.21.15.114:0/1819132027 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ffb58076300 msgr2=0x7ffb58078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:31.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.378+0000 7ffb82600640 1 --2- 172.21.15.114:0/1819132027 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ffb58076300 0x7ffb58078810 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c06f060 tx=0x7ffb64009190 comp rx=0 tx=0).stop 2024-05-27T09:37:31.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.378+0000 7ffb82600640 1 -- 172.21.15.114:0/1819132027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb7c0c2a00 msgr2=0x7ffb7c156dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:31.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.378+0000 7ffb82600640 1 --2- 172.21.15.114:0/1819132027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb7c0c2a00 0x7ffb7c156dc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00c3d0 tx=0x7ffb6c00c890 comp rx=0 tx=0).stop 2024-05-27T09:37:31.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.379+0000 7ffb82600640 1 -- 172.21.15.114:0/1819132027 shutdown_connections 2024-05-27T09:37:31.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.379+0000 7ffb82600640 1 --2- 172.21.15.114:0/1819132027 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ffb58076300 0x7ffb58078810 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:31.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.379+0000 7ffb82600640 1 --2- 172.21.15.114:0/1819132027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb7c159e40 0x7ffb7c14a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:31.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.379+0000 7ffb82600640 1 --2- 172.21.15.114:0/1819132027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb7c1534c0 0x7ffb7c149c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:31.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.379+0000 7ffb82600640 1 --2- 172.21.15.114:0/1819132027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb7c0c2a00 0x7ffb7c156dc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:31.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.379+0000 7ffb82600640 1 -- 172.21.15.114:0/1819132027 >> 172.21.15.114:0/1819132027 conn(0x7ffb7c0ca320 msgr2=0x7ffb7c14d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:31.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.379+0000 7ffb82600640 1 -- 172.21.15.114:0/1819132027 shutdown_connections 2024-05-27T09:37:31.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:31.379+0000 7ffb82600640 1 -- 172.21.15.114:0/1819132027 wait complete. 2024-05-27T09:37:31.408 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 46, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-05-27T09:27:18.079214+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 46, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5072": { "addr": "172.21.15.163:6839/1057092553", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6837", "nonce": 1057092553, "type": "v2" }, { "addr": "172.21.15.163:6839", "nonce": 1057092553, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5072, "incarnation": 43, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:rejoin", "state_seq": 25 }, "gid_5086": { "addr": "172.21.15.114:6834/1371544448", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6832", "nonce": 1371544448, "type": "v2" }, { "addr": "172.21.15.114:6834", "nonce": 1371544448, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5086, "incarnation": 31, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 27 }, "gid_5160": { "addr": "172.21.15.114:6837/1128209239", "addrs": { "addrvec": [ { "addr": "172.21.15.114:6836", "nonce": 1128209239, "type": "v2" }, { "addr": "172.21.15.114:6837", "nonce": 1128209239, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5160, "incarnation": 37, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 26 } }, "last_failure": 0, "last_failure_osd_epoch": 50, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-05-27T09:37:31.305177+0000", "qdb_cluster": [ 5160, 5086 ], "qdb_leader": 5086, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5086, "mds_1": 5072, "mds_2": 5160 } } } ], "standbys": [ { "addr": "172.21.15.163:6835/3513522318", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 3513522318, "type": "v2" }, { "addr": "172.21.15.163:6835", "nonce": 3513522318, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6838/1753596112", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6836", "nonce": 1753596112, "type": "v2" }, { "addr": "172.21.15.163:6838", "nonce": 1753596112, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-05-27T09:37:33.409 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:37:33.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.695+0000 7fc66b200640 1 Processor -- start 2024-05-27T09:37:33.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.695+0000 7fc66b200640 1 -- start start 2024-05-27T09:37:33.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.697+0000 7fc66b200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc66c1547d0 0x7fc66c0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:33.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.697+0000 7fc66b200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc66c074840 0x7fc66c15ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:33.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.697+0000 7fc66b200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc66c06fe50 con 0x7fc66c0741f0 2024-05-27T09:37:33.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.697+0000 7fc66b200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fc66c0703d0 con 0x7fc66c1547d0 2024-05-27T09:37:33.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.697+0000 7fc66b200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fc66c15b260 con 0x7fc66c074840 2024-05-27T09:37:33.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.697+0000 7fc661400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc66c1547d0 0x7fc66c0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:33.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.697+0000 7fc661400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fc66c1547d0 0x7fc66c0c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45208/0 (socket says 172.21.15.114:45208) 2024-05-27T09:37:33.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.697+0000 7fc66a800640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc66c074840 0x7fc66c15ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:33.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.697+0000 7fc661400640 1 -- 172.21.15.114:0/2653477270 learned_addr learned my addr 172.21.15.114:0/2653477270 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:33.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.698+0000 7fc66a800640 1 -- 172.21.15.114:0/2653477270 >> v2:172.21.15.114:3300/0 conn(0x7fc66c1547d0 msgr2=0x7fc66c0c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:33.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.698+0000 7fc66a800640 1 --2- 172.21.15.114:0/2653477270 >> v2:172.21.15.114:3300/0 conn(0x7fc66c1547d0 0x7fc66c0c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:33.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.698+0000 7fc66a800640 1 -- 172.21.15.114:0/2653477270 >> v1:172.21.15.163:6789/0 conn(0x7fc66c0741f0 legacy=0x7fc66c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:33.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.698+0000 7fc66a800640 1 -- 172.21.15.114:0/2653477270 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc66c0700f0 con 0x7fc66c074840 2024-05-27T09:37:33.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.698+0000 7fc669400640 1 -- 172.21.15.114:0/2653477270 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4008747315 0 0) 0x7fc66c06fe50 con 0x7fc66c0741f0 2024-05-27T09:37:33.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.698+0000 7fc661400640 1 --2- 172.21.15.114:0/2653477270 >> v2:172.21.15.114:3300/0 conn(0x7fc66c1547d0 0x7fc66c0c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:33.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.699+0000 7fc66a800640 1 --2- 172.21.15.114:0/2653477270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc66c074840 0x7fc66c15ad20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc66c070940 tx=0x7fc66402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ce2ebb6f8382dc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:33.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.699+0000 7fc669400640 1 -- 172.21.15.114:0/2653477270 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc664007740 con 0x7fc66c074840 2024-05-27T09:37:33.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.699+0000 7fc669400640 1 -- 172.21.15.114:0/2653477270 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6640078e0 con 0x7fc66c074840 2024-05-27T09:37:33.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.699+0000 7fc669400640 1 -- 172.21.15.114:0/2653477270 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc664007cb0 con 0x7fc66c074840 2024-05-27T09:37:33.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.700+0000 7fc66b200640 1 -- 172.21.15.114:0/2653477270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc66c074840 msgr2=0x7fc66c15ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:33.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.700+0000 7fc66b200640 1 --2- 172.21.15.114:0/2653477270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc66c074840 0x7fc66c15ad20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc66c070940 tx=0x7fc66402a710 comp rx=0 tx=0).stop 2024-05-27T09:37:33.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.700+0000 7fc66b200640 1 -- 172.21.15.114:0/2653477270 shutdown_connections 2024-05-27T09:37:33.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.700+0000 7fc66b200640 1 --2- 172.21.15.114:0/2653477270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc66c074840 0x7fc66c15ad20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:33.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.700+0000 7fc66b200640 1 --2- 172.21.15.114:0/2653477270 >> v2:172.21.15.114:3300/0 conn(0x7fc66c1547d0 0x7fc66c0c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:33.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.700+0000 7fc66b200640 1 -- 172.21.15.114:0/2653477270 >> 172.21.15.114:0/2653477270 conn(0x7fc66c0ca320 msgr2=0x7fc66c0ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:33.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.700+0000 7fc66b200640 1 -- 172.21.15.114:0/2653477270 shutdown_connections 2024-05-27T09:37:33.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.701+0000 7fc66b200640 1 -- 172.21.15.114:0/2653477270 wait complete. 2024-05-27T09:37:33.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.703+0000 7fc66b200640 1 Processor -- start 2024-05-27T09:37:33.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.703+0000 7fc66b200640 1 -- start start 2024-05-27T09:37:33.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.705+0000 7fc66b200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc66c0741f0 0x7fc66c15b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:33.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.705+0000 7fc66b200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc66c074840 0x7fc66c1f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:33.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.705+0000 7fc66b200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc66c1547d0 0x7fc66c0c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:33.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.705+0000 7fc66b200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c073790 con 0x7fc66c0741f0 2024-05-27T09:37:33.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.705+0000 7fc66b200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c0734b0 con 0x7fc66c074840 2024-05-27T09:37:33.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.705+0000 7fc66b200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fc66c06f500 con 0x7fc66c1547d0 2024-05-27T09:37:33.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.706+0000 7fc669e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc66c0741f0 0x7fc66c15b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:33.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.706+0000 7fc669e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc66c0741f0 0x7fc66c15b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45222/0 (socket says 172.21.15.114:45222) 2024-05-27T09:37:33.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.706+0000 7fc669e00640 1 -- 172.21.15.114:0/2055297910 learned_addr learned my addr 172.21.15.114:0/2055297910 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:33.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.706+0000 7fc66a800640 1 --2- 172.21.15.114:0/2055297910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc66c1547d0 0x7fc66c0c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:33.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.706+0000 7fc661400640 1 --2- 172.21.15.114:0/2055297910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc66c074840 0x7fc66c1f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:33.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.706+0000 7fc669e00640 1 -- 172.21.15.114:0/2055297910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc66c1547d0 msgr2=0x7fc66c0c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:33.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.706+0000 7fc669e00640 1 --2- 172.21.15.114:0/2055297910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc66c1547d0 0x7fc66c0c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:33.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.706+0000 7fc669e00640 1 -- 172.21.15.114:0/2055297910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc66c074840 msgr2=0x7fc66c1f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:33.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.706+0000 7fc669e00640 1 --2- 172.21.15.114:0/2055297910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc66c074840 0x7fc66c1f4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:33.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.706+0000 7fc669e00640 1 -- 172.21.15.114:0/2055297910 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc66c15bf00 con 0x7fc66c0741f0 2024-05-27T09:37:33.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.706+0000 7fc661400640 1 --2- 172.21.15.114:0/2055297910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc66c074840 0x7fc66c1f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:33.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.707+0000 7fc669e00640 1 --2- 172.21.15.114:0/2055297910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc66c0741f0 0x7fc66c15b9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00c370 tx=0x7fc65c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:33.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.707+0000 7fc662a00640 1 -- 172.21.15.114:0/2055297910 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c017070 con 0x7fc66c0741f0 2024-05-27T09:37:33.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.707+0000 7fc662a00640 1 -- 172.21.15.114:0/2055297910 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc65c0052d0 con 0x7fc66c0741f0 2024-05-27T09:37:33.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.707+0000 7fc662a00640 1 -- 172.21.15.114:0/2055297910 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c003850 con 0x7fc66c0741f0 2024-05-27T09:37:33.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.707+0000 7fc66b200640 1 -- 172.21.15.114:0/2055297910 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc66c15dd70 con 0x7fc66c0741f0 2024-05-27T09:37:33.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.707+0000 7fc66b200640 1 -- 172.21.15.114:0/2055297910 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc66c15e050 con 0x7fc66c0741f0 2024-05-27T09:37:33.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.708+0000 7fc66b200640 1 -- 172.21.15.114:0/2055297910 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc66c0b3370 con 0x7fc66c0741f0 2024-05-27T09:37:33.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.709+0000 7fc662a00640 1 -- 172.21.15.114:0/2055297910 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc65c01a070 con 0x7fc66c0741f0 2024-05-27T09:37:33.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.710+0000 7fc662a00640 1 --2- 172.21.15.114:0/2055297910 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc6480762d0 0x7fc6480787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:33.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.711+0000 7fc662a00640 1 -- 172.21.15.114:0/2055297910 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7fc65c096b20 con 0x7fc66c0741f0 2024-05-27T09:37:33.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.711+0000 7fc661400640 1 --2- 172.21.15.114:0/2055297910 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc6480762d0 0x7fc6480787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:33.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.712+0000 7fc661400640 1 --2- 172.21.15.114:0/2055297910 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc6480762d0 0x7fc6480787e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc654009630 tx=0x7fc654009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:33.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.715+0000 7fc662a00640 1 -- 172.21.15.114:0/2055297910 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc65c05fb80 con 0x7fc66c0741f0 2024-05-27T09:37:33.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.962+0000 7fc66b200640 1 -- 172.21.15.114:0/2055297910 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc66c0c6510 con 0x7fc66c0741f0 2024-05-27T09:37:33.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.964+0000 7fc662a00640 1 -- 172.21.15.114:0/2055297910 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+6057 (secure 0 0 0) 0x7fc65c068250 con 0x7fc66c0741f0 2024-05-27T09:37:33.964 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:37:33.965 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:37:31.305177+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":5072,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5072":{"gid":5072,"name":"f","rank":1,"incarnation":43,"state":"up:rejoin","state_seq":25,"addr":"172.21.15.163:6839/1057092553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":1057092553},{"type":"v1","addr":"172.21.15.163:6839","nonce":1057092553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:active","state_seq":26,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5160,5086]},"id":1}]} 2024-05-27T09:37:33.965 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 46 2024-05-27T09:37:33.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.966+0000 7fc66b200640 1 -- 172.21.15.114:0/2055297910 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc6480762d0 msgr2=0x7fc6480787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:33.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.966+0000 7fc66b200640 1 --2- 172.21.15.114:0/2055297910 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc6480762d0 0x7fc6480787e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc654009630 tx=0x7fc654009240 comp rx=0 tx=0).stop 2024-05-27T09:37:33.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.966+0000 7fc66b200640 1 -- 172.21.15.114:0/2055297910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc66c0741f0 msgr2=0x7fc66c15b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:33.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.966+0000 7fc66b200640 1 --2- 172.21.15.114:0/2055297910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc66c0741f0 0x7fc66c15b9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00c370 tx=0x7fc65c00c830 comp rx=0 tx=0).stop 2024-05-27T09:37:33.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.967+0000 7fc66b200640 1 -- 172.21.15.114:0/2055297910 shutdown_connections 2024-05-27T09:37:33.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.967+0000 7fc66b200640 1 --2- 172.21.15.114:0/2055297910 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fc6480762d0 0x7fc6480787e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:33.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.967+0000 7fc66b200640 1 --2- 172.21.15.114:0/2055297910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc66c1547d0 0x7fc66c0c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:33.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.967+0000 7fc66b200640 1 --2- 172.21.15.114:0/2055297910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc66c074840 0x7fc66c1f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:33.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.967+0000 7fc66b200640 1 --2- 172.21.15.114:0/2055297910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc66c0741f0 0x7fc66c15b9c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:33.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.967+0000 7fc66b200640 1 -- 172.21.15.114:0/2055297910 >> 172.21.15.114:0/2055297910 conn(0x7fc66c0ca320 msgr2=0x7fc66c150ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:33.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.967+0000 7fc66b200640 1 -- 172.21.15.114:0/2055297910 shutdown_connections 2024-05-27T09:37:33.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:33.967+0000 7fc66b200640 1 -- 172.21.15.114:0/2055297910 wait complete. 2024-05-27T09:37:33.995 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-05-27T09:37:35.996 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:37:36.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.274+0000 7f8545000640 1 Processor -- start 2024-05-27T09:37:36.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.274+0000 7f8545000640 1 -- start start 2024-05-27T09:37:36.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.275+0000 7f8545000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85400c9170 0x7f8540152f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:36.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.276+0000 7f8545000640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85400c42f0 con 0x7f85400c8370 2024-05-27T09:37:36.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.276+0000 7f8545000640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85400c45d0 con 0x7f85400ca400 2024-05-27T09:37:36.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.276+0000 7f8545000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f85400c9b40 con 0x7f85400c9170 2024-05-27T09:37:36.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.276+0000 7f8535400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f85400c8370 0x7f854015c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56780/0 (socket says 172.21.15.114:56780) 2024-05-27T09:37:36.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.276+0000 7f8535400640 1 -- 172.21.15.114:0/258850686 learned_addr learned my addr 172.21.15.114:0/258850686 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:36.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.276+0000 7f853de00640 1 --2- 172.21.15.114:0/258850686 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85400c9170 0x7f8540152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:36.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.276+0000 7f853d400640 1 -- 172.21.15.114:0/258850686 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1330551095 0 0) 0x7f85400c42f0 con 0x7f85400c8370 2024-05-27T09:37:36.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.276+0000 7f853d400640 1 -- 172.21.15.114:0/258850686 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8524003440 con 0x7f85400c8370 2024-05-27T09:37:36.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.276+0000 7f853d400640 1 -- 172.21.15.114:0/258850686 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2554201914 0 0) 0x7f85400c45d0 con 0x7f85400ca400 2024-05-27T09:37:36.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.276+0000 7f853d400640 1 -- 172.21.15.114:0/258850686 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85400c42f0 con 0x7f85400ca400 2024-05-27T09:37:36.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.277+0000 7f853de00640 1 -- 172.21.15.114:0/258850686 >> v1:172.21.15.163:6789/0 conn(0x7f85400ca400 legacy=0x7f85401634e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:36.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.277+0000 7f853de00640 1 -- 172.21.15.114:0/258850686 >> v1:172.21.15.114:6789/0 conn(0x7f85400c8370 legacy=0x7f854015c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:36.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.277+0000 7f853de00640 1 -- 172.21.15.114:0/258850686 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85400c48b0 con 0x7f85400c9170 2024-05-27T09:37:36.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.278+0000 7f853de00640 1 --2- 172.21.15.114:0/258850686 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85400c9170 0x7f8540152f00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f85300097d0 tx=0x7f853002a2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd319efa14b5f9c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:36.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.278+0000 7f853d400640 1 -- 172.21.15.114:0/258850686 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530037070 con 0x7f85400c9170 2024-05-27T09:37:36.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.278+0000 7f853d400640 1 -- 172.21.15.114:0/258850686 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8530032640 con 0x7f85400c9170 2024-05-27T09:37:36.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.278+0000 7f853d400640 1 -- 172.21.15.114:0/258850686 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85300329f0 con 0x7f85400c9170 2024-05-27T09:37:36.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.279+0000 7f8545000640 1 -- 172.21.15.114:0/258850686 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85400c9170 msgr2=0x7f8540152f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:36.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.279+0000 7f8545000640 1 --2- 172.21.15.114:0/258850686 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85400c9170 0x7f8540152f00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f85300097d0 tx=0x7f853002a2c0 comp rx=0 tx=0).stop 2024-05-27T09:37:36.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.279+0000 7f8545000640 1 -- 172.21.15.114:0/258850686 shutdown_connections 2024-05-27T09:37:36.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.279+0000 7f8545000640 1 --2- 172.21.15.114:0/258850686 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85400c9170 0x7f8540152f00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:36.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.279+0000 7f8545000640 1 -- 172.21.15.114:0/258850686 >> 172.21.15.114:0/258850686 conn(0x7f854006f6e0 msgr2=0x7f8540070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:36.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.279+0000 7f8545000640 1 -- 172.21.15.114:0/258850686 shutdown_connections 2024-05-27T09:37:36.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.279+0000 7f8545000640 1 -- 172.21.15.114:0/258850686 wait complete. 2024-05-27T09:37:36.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.281+0000 7f8545000640 1 Processor -- start 2024-05-27T09:37:36.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.281+0000 7f8545000640 1 -- start start 2024-05-27T09:37:36.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.283+0000 7f8545000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85400c8370 0x7f85400cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:36.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.283+0000 7f8545000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85400c9170 0x7f85401ee960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:36.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.283+0000 7f8545000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85400ca400 0x7f85400cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:36.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.283+0000 7f8545000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f85400c9fd0 con 0x7f85400c9170 2024-05-27T09:37:36.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.283+0000 7f8545000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f85400c2b20 con 0x7f85400c8370 2024-05-27T09:37:36.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.283+0000 7f8545000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f85400ca150 con 0x7f85400ca400 2024-05-27T09:37:36.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.283+0000 7f8535400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85400c9170 0x7f85401ee960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:36.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.283+0000 7f8535400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85400c9170 0x7f85401ee960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39716/0 (socket says 172.21.15.114:39716) 2024-05-27T09:37:36.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.283+0000 7f8535400640 1 -- 172.21.15.114:0/3187348349 learned_addr learned my addr 172.21.15.114:0/3187348349 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:36.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.283+0000 7f853de00640 1 --2- 172.21.15.114:0/3187348349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85400c8370 0x7f85400cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:36.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.283+0000 7f853e800640 1 --2- 172.21.15.114:0/3187348349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85400ca400 0x7f85400cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:36.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.283+0000 7f8535400640 1 -- 172.21.15.114:0/3187348349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85400ca400 msgr2=0x7f85400cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:36.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.283+0000 7f8535400640 1 --2- 172.21.15.114:0/3187348349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85400ca400 0x7f85400cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:36.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.283+0000 7f8535400640 1 -- 172.21.15.114:0/3187348349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85400c8370 msgr2=0x7f85400cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:36.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.283+0000 7f8535400640 1 --2- 172.21.15.114:0/3187348349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85400c8370 0x7f85400cc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:36.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.283+0000 7f8535400640 1 -- 172.21.15.114:0/3187348349 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85400cdd60 con 0x7f85400c9170 2024-05-27T09:37:36.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.283+0000 7f853e800640 1 --2- 172.21.15.114:0/3187348349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85400ca400 0x7f85400cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:37:36.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.284+0000 7f8535400640 1 --2- 172.21.15.114:0/3187348349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85400c9170 0x7f85401ee960 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f852800c390 tx=0x7f8528001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:36.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.284+0000 7f8536a00640 1 -- 172.21.15.114:0/3187348349 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528017070 con 0x7f85400c9170 2024-05-27T09:37:36.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.284+0000 7f8536a00640 1 -- 172.21.15.114:0/3187348349 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8528005ce0 con 0x7f85400c9170 2024-05-27T09:37:36.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.284+0000 7f8536a00640 1 -- 172.21.15.114:0/3187348349 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85280072f0 con 0x7f85400c9170 2024-05-27T09:37:36.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.284+0000 7f8545000640 1 -- 172.21.15.114:0/3187348349 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85401f20f0 con 0x7f85400c9170 2024-05-27T09:37:36.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.284+0000 7f8545000640 1 -- 172.21.15.114:0/3187348349 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85401f26b0 con 0x7f85400c9170 2024-05-27T09:37:36.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.286+0000 7f8545000640 1 -- 172.21.15.114:0/3187348349 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85400692a0 con 0x7f85400c9170 2024-05-27T09:37:36.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.287+0000 7f8536a00640 1 -- 172.21.15.114:0/3187348349 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f852801a070 con 0x7f85400c9170 2024-05-27T09:37:36.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.288+0000 7f8536a00640 1 --2- 172.21.15.114:0/3187348349 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f851c0762d0 0x7f851c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:36.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.288+0000 7f8536a00640 1 -- 172.21.15.114:0/3187348349 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7f8528096aa0 con 0x7f85400c9170 2024-05-27T09:37:36.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.288+0000 7f853de00640 1 --2- 172.21.15.114:0/3187348349 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f851c0762d0 0x7f851c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:36.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.289+0000 7f853de00640 1 --2- 172.21.15.114:0/3187348349 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f851c0762d0 0x7f851c0787e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8530034000 tx=0x7f853002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:36.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.293+0000 7f8536a00640 1 -- 172.21.15.114:0/3187348349 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f85400692a0 con 0x7f85400c9170 2024-05-27T09:37:36.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.543+0000 7f8545000640 1 -- 172.21.15.114:0/3187348349 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85400c2dd0 con 0x7f85400c9170 2024-05-27T09:37:36.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.545+0000 7f8536a00640 1 -- 172.21.15.114:0/3187348349 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+6062 (secure 0 0 0) 0x7f85400c2dd0 con 0x7f85400c9170 2024-05-27T09:37:36.545 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:37:36.546 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:37:36.353158+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":5072,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5072":{"gid":5072,"name":"f","rank":1,"incarnation":43,"state":"up:active","state_seq":27,"addr":"172.21.15.163:6839/1057092553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":1057092553},{"type":"v1","addr":"172.21.15.163:6839","nonce":1057092553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:active","state_seq":26,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5072,5086,5160]},"id":1}]} 2024-05-27T09:37:36.546 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 47 2024-05-27T09:37:36.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.548+0000 7f8545000640 1 -- 172.21.15.114:0/3187348349 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f851c0762d0 msgr2=0x7f851c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:36.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.548+0000 7f8545000640 1 --2- 172.21.15.114:0/3187348349 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f851c0762d0 0x7f851c0787e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8530034000 tx=0x7f853002e000 comp rx=0 tx=0).stop 2024-05-27T09:37:36.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.548+0000 7f8545000640 1 -- 172.21.15.114:0/3187348349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85400c9170 msgr2=0x7f85401ee960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:36.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.548+0000 7f8545000640 1 --2- 172.21.15.114:0/3187348349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85400c9170 0x7f85401ee960 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f852800c390 tx=0x7f8528001e70 comp rx=0 tx=0).stop 2024-05-27T09:37:36.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.548+0000 7f8545000640 1 -- 172.21.15.114:0/3187348349 shutdown_connections 2024-05-27T09:37:36.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.548+0000 7f8545000640 1 --2- 172.21.15.114:0/3187348349 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f851c0762d0 0x7f851c0787e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:36.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.548+0000 7f8545000640 1 --2- 172.21.15.114:0/3187348349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85400ca400 0x7f85400cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:36.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.548+0000 7f8545000640 1 --2- 172.21.15.114:0/3187348349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85400c9170 0x7f85401ee960 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:36.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.548+0000 7f8545000640 1 --2- 172.21.15.114:0/3187348349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85400c8370 0x7f85400cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:36.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.548+0000 7f8545000640 1 -- 172.21.15.114:0/3187348349 >> 172.21.15.114:0/3187348349 conn(0x7f854006f6e0 msgr2=0x7f8540070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:36.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.548+0000 7f8545000640 1 -- 172.21.15.114:0/3187348349 shutdown_connections 2024-05-27T09:37:36.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.548+0000 7f8545000640 1 -- 172.21.15.114:0/3187348349 wait complete. 2024-05-27T09:37:36.576 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=1, replacing gid=4875 2024-05-27T09:37:36.576 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.a 2024-05-27T09:37:36.576 ERROR:tasks.mds_thrash.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks/mds_thrash.py", line 121, in _run self.do_thrash() File "/home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks/mds_thrash.py", line 314, in do_thrash self.sleep_unless_stopped(delay) File "/home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-05-27T09:37:36.583 INFO:tasks.mds_thrash:done joining 2024-05-27T09:37:36.583 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-27T09:37:36.592 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-27T09:37:36.593 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:37:36.593 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T09:37:36.648 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-05-27T09:37:36.648 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T09:37:36.699 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-27T09:37:36.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.978+0000 7f4e11800640 1 Processor -- start 2024-05-27T09:37:36.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.979+0000 7f4e11800640 1 -- start start 2024-05-27T09:37:36.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.980+0000 7f4e11800640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4e0c074a60 0x7f4e0c154820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:36.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.980+0000 7f4e11800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e0c0ca9e0 0x7f4e0c073670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:36.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.980+0000 7f4e11800640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e0c15a810 con 0x7f4e0c073260 2024-05-27T09:37:36.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.980+0000 7f4e11800640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f4e0c15db60 con 0x7f4e0c074a60 2024-05-27T09:37:36.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.981+0000 7f4e11800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f4e0c15aaf0 con 0x7f4e0c0ca9e0 2024-05-27T09:37:36.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.981+0000 7f4e03e00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f4e0c073260 0x7f4e0c0ca2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56786/0 (socket says 172.21.15.114:56786) 2024-05-27T09:37:36.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.981+0000 7f4e03e00640 1 -- 172.21.15.114:0/352031297 learned_addr learned my addr 172.21.15.114:0/352031297 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:36.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.981+0000 7f4e0b400640 1 --2- 172.21.15.114:0/352031297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e0c0ca9e0 0x7f4e0c073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:36.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.981+0000 7f4e0aa00640 1 --2- 172.21.15.114:0/352031297 >> v2:172.21.15.163:3300/0 conn(0x7f4e0c074a60 0x7f4e0c154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:36.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.981+0000 7f4e0a000640 1 -- 172.21.15.114:0/352031297 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3930877360 0 0) 0x7f4e0c15a810 con 0x7f4e0c073260 2024-05-27T09:37:36.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.981+0000 7f4e0a000640 1 -- 172.21.15.114:0/352031297 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4df0003520 con 0x7f4e0c073260 2024-05-27T09:37:36.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.981+0000 7f4e0aa00640 1 -- 172.21.15.114:0/352031297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e0c0ca9e0 msgr2=0x7f4e0c073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:36.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.981+0000 7f4e0aa00640 1 --2- 172.21.15.114:0/352031297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e0c0ca9e0 0x7f4e0c073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:36.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.982+0000 7f4e0aa00640 1 -- 172.21.15.114:0/352031297 >> v1:172.21.15.114:6789/0 conn(0x7f4e0c073260 legacy=0x7f4e0c0ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:36.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.982+0000 7f4e0aa00640 1 -- 172.21.15.114:0/352031297 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0c15add0 con 0x7f4e0c074a60 2024-05-27T09:37:36.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.982+0000 7f4e0b400640 1 --2- 172.21.15.114:0/352031297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e0c0ca9e0 0x7f4e0c073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:36.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.983+0000 7f4e0aa00640 1 --2- 172.21.15.114:0/352031297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e0c074a60 0x7f4e0c154820 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc0097a0 tx=0x7f4dfc02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4adc49275d1daf4e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:36.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.983+0000 7f4e0a000640 1 -- 172.21.15.114:0/352031297 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc037070 con 0x7f4e0c074a60 2024-05-27T09:37:36.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.983+0000 7f4e0a000640 1 -- 172.21.15.114:0/352031297 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dfc032440 con 0x7f4e0c074a60 2024-05-27T09:37:36.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.983+0000 7f4e0a000640 1 -- 172.21.15.114:0/352031297 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc0327f0 con 0x7f4e0c074a60 2024-05-27T09:37:36.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.984+0000 7f4e11800640 1 -- 172.21.15.114:0/352031297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e0c074a60 msgr2=0x7f4e0c154820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:36.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.984+0000 7f4e11800640 1 --2- 172.21.15.114:0/352031297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e0c074a60 0x7f4e0c154820 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc0097a0 tx=0x7f4dfc02a210 comp rx=0 tx=0).stop 2024-05-27T09:37:36.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.984+0000 7f4e11800640 1 -- 172.21.15.114:0/352031297 shutdown_connections 2024-05-27T09:37:36.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.984+0000 7f4e11800640 1 --2- 172.21.15.114:0/352031297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e0c0ca9e0 0x7f4e0c073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:36.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.984+0000 7f4e11800640 1 --2- 172.21.15.114:0/352031297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e0c074a60 0x7f4e0c154820 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:36.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.984+0000 7f4e11800640 1 -- 172.21.15.114:0/352031297 >> 172.21.15.114:0/352031297 conn(0x7f4e0c06f6e0 msgr2=0x7f4e0c070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:36.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.984+0000 7f4e11800640 1 -- 172.21.15.114:0/352031297 shutdown_connections 2024-05-27T09:37:36.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.984+0000 7f4e11800640 1 -- 172.21.15.114:0/352031297 wait complete. 2024-05-27T09:37:36.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.986+0000 7f4e11800640 1 Processor -- start 2024-05-27T09:37:36.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.987+0000 7f4e11800640 1 -- start start 2024-05-27T09:37:36.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.988+0000 7f4e11800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e0c073260 0x7f4e0c15bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:36.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.988+0000 7f4e11800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e0c074a60 0x7f4e0c0c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:36.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.988+0000 7f4e11800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e0c0ca9e0 0x7f4e0c15c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:36.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.988+0000 7f4e11800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c15a4e0 con 0x7f4e0c073260 2024-05-27T09:37:36.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.988+0000 7f4e11800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c15b390 con 0x7f4e0c0ca9e0 2024-05-27T09:37:36.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.988+0000 7f4e11800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f4e0c15dff0 con 0x7f4e0c074a60 2024-05-27T09:37:36.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.988+0000 7f4e0aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e0c073260 0x7f4e0c15bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:36.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.988+0000 7f4e03e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e0c074a60 0x7f4e0c0c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:36.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.988+0000 7f4e0b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e0c0ca9e0 0x7f4e0c15c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:36.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.988+0000 7f4e03e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e0c074a60 0x7f4e0c0c5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:39278/0 (socket says 172.21.15.114:39278) 2024-05-27T09:37:36.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.988+0000 7f4e0aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e0c073260 0x7f4e0c15bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39732/0 (socket says 172.21.15.114:39732) 2024-05-27T09:37:36.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.988+0000 7f4e0b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e0c0ca9e0 0x7f4e0c15c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:51346/0 (socket says 172.21.15.114:51346) 2024-05-27T09:37:36.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.988+0000 7f4e03e00640 1 -- 172.21.15.114:0/126247350 learned_addr learned my addr 172.21.15.114:0/126247350 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:36.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.989+0000 7f4e03e00640 1 -- 172.21.15.114:0/126247350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e0c0ca9e0 msgr2=0x7f4e0c15c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:36.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.989+0000 7f4e03e00640 1 --2- 172.21.15.114:0/126247350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e0c0ca9e0 0x7f4e0c15c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:36.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.989+0000 7f4e03e00640 1 -- 172.21.15.114:0/126247350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e0c073260 msgr2=0x7f4e0c15bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:36.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.989+0000 7f4e03e00640 1 --2- 172.21.15.114:0/126247350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e0c073260 0x7f4e0c15bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:36.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.989+0000 7f4e03e00640 1 -- 172.21.15.114:0/126247350 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0c152690 con 0x7f4e0c074a60 2024-05-27T09:37:36.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.989+0000 7f4e0aa00640 1 --2- 172.21.15.114:0/126247350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e0c073260 0x7f4e0c15bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:36.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.989+0000 7f4e03e00640 1 --2- 172.21.15.114:0/126247350 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e0c074a60 0x7f4e0c0c5c30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4df400c3f0 tx=0x7f4df400c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:36.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.989+0000 7f4e02a00640 1 -- 172.21.15.114:0/126247350 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df4017070 con 0x7f4e0c074a60 2024-05-27T09:37:36.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.989+0000 7f4e11800640 1 -- 172.21.15.114:0/126247350 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e0c0c4350 con 0x7f4e0c074a60 2024-05-27T09:37:36.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.990+0000 7f4e02a00640 1 -- 172.21.15.114:0/126247350 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4df4005ce0 con 0x7f4e0c074a60 2024-05-27T09:37:36.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.990+0000 7f4e02a00640 1 -- 172.21.15.114:0/126247350 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df40072d0 con 0x7f4e0c074a60 2024-05-27T09:37:36.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.990+0000 7f4e11800640 1 -- 172.21.15.114:0/126247350 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e0c145d30 con 0x7f4e0c074a60 2024-05-27T09:37:36.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.991+0000 7f4e11800640 1 -- 172.21.15.114:0/126247350 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e0c06a380 con 0x7f4e0c074a60 2024-05-27T09:37:36.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.992+0000 7f4e02a00640 1 -- 172.21.15.114:0/126247350 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4df401a070 con 0x7f4e0c074a60 2024-05-27T09:37:36.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.993+0000 7f4e02a00640 1 --2- 172.21.15.114:0/126247350 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4de80762d0 0x7f4de80787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:36.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.993+0000 7f4e02a00640 1 -- 172.21.15.114:0/126247350 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7f4df4096e00 con 0x7f4e0c074a60 2024-05-27T09:37:36.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.993+0000 7f4e0aa00640 1 --2- 172.21.15.114:0/126247350 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4de80762d0 0x7f4de80787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:36.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.994+0000 7f4e0aa00640 1 --2- 172.21.15.114:0/126247350 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4de80762d0 0x7f4de80787e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc034000 tx=0x7f4dfc02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:36.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:36.997+0000 7f4e02a00640 1 -- 172.21.15.114:0/126247350 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4df403a020 con 0x7f4e0c074a60 2024-05-27T09:37:37.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:37.202+0000 7f4e11800640 1 -- 172.21.15.114:0/126247350 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4e0c073ca0 con 0x7f4e0c074a60 2024-05-27T09:37:37.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:37.203+0000 7f4e02a00640 1 -- 172.21.15.114:0/126247350 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 12 entries v50) ==== 81+0+740 (secure 0 0 0) 0x7f4df405fee0 con 0x7f4e0c074a60 2024-05-27T09:37:37.203 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6835/2747052034 2024-05-28T09:37:15.516770+0000 2024-05-27T09:37:37.203 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6833/2747052034 2024-05-28T09:37:15.516770+0000 2024-05-27T09:37:37.204 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.163:6837/3371890103 2024-05-28T09:35:40.437044+0000 2024-05-27T09:37:37.204 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6837/1448950017 2024-05-28T09:34:05.407657+0000 2024-05-27T09:37:37.204 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.163:6839/3371890103 2024-05-28T09:35:40.437044+0000 2024-05-27T09:37:37.204 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6834/657027519 2024-05-28T09:32:30.373988+0000 2024-05-27T09:37:37.204 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6836/1448950017 2024-05-28T09:34:05.407657+0000 2024-05-27T09:37:37.204 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6832/657027519 2024-05-28T09:32:30.373988+0000 2024-05-27T09:37:37.204 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6833/1325619883 2024-05-28T09:31:00.345342+0000 2024-05-27T09:37:37.205 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.163:6839/444766663 2024-05-28T09:29:45.323526+0000 2024-05-27T09:37:37.205 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6835/1325619883 2024-05-28T09:31:00.345342+0000 2024-05-27T09:37:37.205 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.163:6837/444766663 2024-05-28T09:29:45.323526+0000 2024-05-27T09:37:37.206 INFO:teuthology.orchestra.run.smithi114.stderr:listed 12 entries 2024-05-27T09:37:37.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:37.205+0000 7f4e11800640 1 -- 172.21.15.114:0/126247350 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4de80762d0 msgr2=0x7f4de80787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:37.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:37.205+0000 7f4e11800640 1 --2- 172.21.15.114:0/126247350 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4de80762d0 0x7f4de80787e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc034000 tx=0x7f4dfc02e000 comp rx=0 tx=0).stop 2024-05-27T09:37:37.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:37.205+0000 7f4e11800640 1 -- 172.21.15.114:0/126247350 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e0c074a60 msgr2=0x7f4e0c0c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:37.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:37.205+0000 7f4e11800640 1 --2- 172.21.15.114:0/126247350 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e0c074a60 0x7f4e0c0c5c30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4df400c3f0 tx=0x7f4df400c7f0 comp rx=0 tx=0).stop 2024-05-27T09:37:37.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:37.206+0000 7f4e11800640 1 -- 172.21.15.114:0/126247350 shutdown_connections 2024-05-27T09:37:37.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:37.206+0000 7f4e11800640 1 --2- 172.21.15.114:0/126247350 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f4de80762d0 0x7f4de80787e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:37.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:37.206+0000 7f4e11800640 1 --2- 172.21.15.114:0/126247350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e0c0ca9e0 0x7f4e0c15c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:37.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:37.206+0000 7f4e11800640 1 --2- 172.21.15.114:0/126247350 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e0c074a60 0x7f4e0c0c5c30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:37.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:37.206+0000 7f4e11800640 1 --2- 172.21.15.114:0/126247350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e0c073260 0x7f4e0c15bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:37.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:37.206+0000 7f4e11800640 1 -- 172.21.15.114:0/126247350 >> 172.21.15.114:0/126247350 conn(0x7f4e0c06f6e0 msgr2=0x7f4e0c06c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:37.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:37.206+0000 7f4e11800640 1 -- 172.21.15.114:0/126247350 shutdown_connections 2024-05-27T09:37:37.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:37.206+0000 7f4e11800640 1 -- 172.21.15.114:0/126247350 wait complete. 2024-05-27T09:37:37.231 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-27T09:37:37.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:37:37.231 DEBUG:teuthology.orchestra.run.smithi114:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-27T09:37:41.381 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi114.front.sepia.ceph.com 2024-05-27T09:37:41.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:37:41.382 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-27T09:37:41.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:37:41.435 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2024-05-27T09:37:41.482 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-27T09:37:41.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:37:41.483 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T09:37:41.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:37:41.561 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link delete ceph-brx 2024-05-27T09:37:41.673 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-27T09:37:41.683 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-27T09:37:41.683 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T09:37:41.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.935+0000 7f2cbcc00640 1 Processor -- start 2024-05-27T09:37:41.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.936+0000 7f2cbcc00640 1 -- start start 2024-05-27T09:37:41.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.938+0000 7f2cbcc00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2cb8074a80 0x7f2cb80c8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:41.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.938+0000 7f2cbcc00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cb80c9dd0 0x7f2cb80c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:41.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.938+0000 7f2cbcc00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cb80c8470 con 0x7f2cb815ad10 2024-05-27T09:37:41.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.938+0000 7f2cbcc00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f2cb80c8220 con 0x7f2cb8074a80 2024-05-27T09:37:41.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.938+0000 7f2cbcc00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb80c91d0 con 0x7f2cb80c9dd0 2024-05-27T09:37:41.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.938+0000 7f2cb7400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f2cb815ad10 0x7f2cb80c96c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56790/0 (socket says 172.21.15.114:56790) 2024-05-27T09:37:41.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.938+0000 7f2cb7400640 1 -- 172.21.15.114:0/1742639987 learned_addr learned my addr 172.21.15.114:0/1742639987 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:41.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.938+0000 7f2cb7e00640 1 --2- 172.21.15.114:0/1742639987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cb80c9dd0 0x7f2cb80c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:41.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.938+0000 7f2caea00640 1 --2- 172.21.15.114:0/1742639987 >> v2:172.21.15.163:3300/0 conn(0x7f2cb8074a80 0x7f2cb80c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:41.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.939+0000 7f2cb7e00640 1 -- 172.21.15.114:0/1742639987 >> v2:172.21.15.163:3300/0 conn(0x7f2cb8074a80 msgr2=0x7f2cb80c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:41.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.939+0000 7f2cb7e00640 1 --2- 172.21.15.114:0/1742639987 >> v2:172.21.15.163:3300/0 conn(0x7f2cb8074a80 0x7f2cb80c8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:41.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.939+0000 7f2cb7e00640 1 -- 172.21.15.114:0/1742639987 >> v1:172.21.15.114:6789/0 conn(0x7f2cb815ad10 legacy=0x7f2cb80c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:41.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.939+0000 7f2cb7e00640 1 -- 172.21.15.114:0/1742639987 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb806ef90 con 0x7f2cb80c9dd0 2024-05-27T09:37:41.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.939+0000 7f2cb6a00640 1 -- 172.21.15.114:0/1742639987 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1418199956 0 0) 0x7f2cb80c8470 con 0x7f2cb815ad10 2024-05-27T09:37:41.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.940+0000 7f2cb7e00640 1 --2- 172.21.15.114:0/1742639987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cb80c9dd0 0x7f2cb80c8c90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009d90 tx=0x7f2cb002a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7da294c30320927b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:41.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.940+0000 7f2cb6a00640 1 -- 172.21.15.114:0/1742639987 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0007740 con 0x7f2cb80c9dd0 2024-05-27T09:37:41.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.940+0000 7f2cb6a00640 1 -- 172.21.15.114:0/1742639987 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cb00078e0 con 0x7f2cb80c9dd0 2024-05-27T09:37:41.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.940+0000 7f2cb6a00640 1 -- 172.21.15.114:0/1742639987 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0007cb0 con 0x7f2cb80c9dd0 2024-05-27T09:37:41.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.941+0000 7f2cbcc00640 1 -- 172.21.15.114:0/1742639987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cb80c9dd0 msgr2=0x7f2cb80c8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:41.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.941+0000 7f2cbcc00640 1 --2- 172.21.15.114:0/1742639987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cb80c9dd0 0x7f2cb80c8c90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009d90 tx=0x7f2cb002a6f0 comp rx=0 tx=0).stop 2024-05-27T09:37:41.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.941+0000 7f2cbcc00640 1 -- 172.21.15.114:0/1742639987 shutdown_connections 2024-05-27T09:37:41.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.941+0000 7f2cbcc00640 1 --2- 172.21.15.114:0/1742639987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cb80c9dd0 0x7f2cb80c8c90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:41.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.941+0000 7f2cbcc00640 1 --2- 172.21.15.114:0/1742639987 >> v2:172.21.15.163:3300/0 conn(0x7f2cb8074a80 0x7f2cb80c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:41.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.941+0000 7f2cbcc00640 1 -- 172.21.15.114:0/1742639987 >> 172.21.15.114:0/1742639987 conn(0x7f2cb80ca320 msgr2=0x7f2cb80ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:41.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.941+0000 7f2cbcc00640 1 -- 172.21.15.114:0/1742639987 shutdown_connections 2024-05-27T09:37:41.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.941+0000 7f2cbcc00640 1 -- 172.21.15.114:0/1742639987 wait complete. 2024-05-27T09:37:41.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.943+0000 7f2cbcc00640 1 Processor -- start 2024-05-27T09:37:41.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.944+0000 7f2cbcc00640 1 -- start start 2024-05-27T09:37:41.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.945+0000 7f2cbcc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cb8074a80 0x7f2cb81f6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:41.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.945+0000 7f2cbcc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cb80c9dd0 0x7f2cb8161ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:41.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.945+0000 7f2cbcc00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cb815ad10 0x7f2cb80ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:41.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.945+0000 7f2cbcc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb806f830 con 0x7f2cb8074a80 2024-05-27T09:37:41.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.945+0000 7f2cbcc00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb806f550 con 0x7f2cb80c9dd0 2024-05-27T09:37:41.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.945+0000 7f2cbcc00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb8070060 con 0x7f2cb815ad10 2024-05-27T09:37:41.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.945+0000 7f2cb7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cb8074a80 0x7f2cb81f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:41.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.946+0000 7f2cb7e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cb815ad10 0x7f2cb80ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:41.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.946+0000 7f2cb7e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cb815ad10 0x7f2cb80ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:39294/0 (socket says 172.21.15.114:39294) 2024-05-27T09:37:41.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.946+0000 7f2cb7e00640 1 -- 172.21.15.114:0/2678552594 learned_addr learned my addr 172.21.15.114:0/2678552594 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:41.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.946+0000 7f2caea00640 1 --2- 172.21.15.114:0/2678552594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cb80c9dd0 0x7f2cb8161ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:41.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.946+0000 7f2cb7e00640 1 -- 172.21.15.114:0/2678552594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cb80c9dd0 msgr2=0x7f2cb8161ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:41.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.946+0000 7f2cb7e00640 1 --2- 172.21.15.114:0/2678552594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cb80c9dd0 0x7f2cb8161ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:41.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.946+0000 7f2cb7e00640 1 -- 172.21.15.114:0/2678552594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cb8074a80 msgr2=0x7f2cb81f6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:41.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.946+0000 7f2cb7e00640 1 --2- 172.21.15.114:0/2678552594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cb8074a80 0x7f2cb81f6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:41.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.946+0000 7f2cb7e00640 1 -- 172.21.15.114:0/2678552594 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb8161ff0 con 0x7f2cb815ad10 2024-05-27T09:37:41.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.946+0000 7f2cb7400640 1 --2- 172.21.15.114:0/2678552594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cb8074a80 0x7f2cb81f6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:41.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.946+0000 7f2cb7e00640 1 --2- 172.21.15.114:0/2678552594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cb815ad10 0x7f2cb80ccdf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009e80 tx=0x7f2cb0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:41.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.947+0000 7f2cafe00640 1 -- 172.21.15.114:0/2678552594 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0009450 con 0x7f2cb815ad10 2024-05-27T09:37:41.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.947+0000 7f2cafe00640 1 -- 172.21.15.114:0/2678552594 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cb0031bf0 con 0x7f2cb815ad10 2024-05-27T09:37:41.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.947+0000 7f2cafe00640 1 -- 172.21.15.114:0/2678552594 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb003b410 con 0x7f2cb815ad10 2024-05-27T09:37:41.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.947+0000 7f2cbcc00640 1 -- 172.21.15.114:0/2678552594 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb8164c00 con 0x7f2cb815ad10 2024-05-27T09:37:41.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.947+0000 7f2cbcc00640 1 -- 172.21.15.114:0/2678552594 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb8164640 con 0x7f2cb815ad10 2024-05-27T09:37:41.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.948+0000 7f2cbcc00640 1 -- 172.21.15.114:0/2678552594 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb8050860 con 0x7f2cb815ad10 2024-05-27T09:37:41.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.950+0000 7f2cafe00640 1 -- 172.21.15.114:0/2678552594 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2cb003b5b0 con 0x7f2cb815ad10 2024-05-27T09:37:41.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.950+0000 7f2cafe00640 1 --2- 172.21.15.114:0/2678552594 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2c940762d0 0x7f2c940787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:41.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.951+0000 7f2cafe00640 1 -- 172.21.15.114:0/2678552594 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7f2cb00b8a00 con 0x7f2cb815ad10 2024-05-27T09:37:41.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.951+0000 7f2cb7400640 1 --2- 172.21.15.114:0/2678552594 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2c940762d0 0x7f2c940787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:41.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.951+0000 7f2cb7400640 1 --2- 172.21.15.114:0/2678552594 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2c940762d0 0x7f2c940787e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8000f60 tx=0x7f2ca8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:41.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:41.956+0000 7f2cafe00640 1 -- 172.21.15.114:0/2678552594 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2cb004a030 con 0x7f2cb815ad10 2024-05-27T09:37:42.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.157+0000 7f2cbcc00640 1 -- 172.21.15.114:0/2678552594 --> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2cb8074860 con 0x7f2c940762d0 2024-05-27T09:37:42.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.165+0000 7f2cafe00640 1 -- 172.21.15.114:0/2678552594 <== mgr.4108 v2:172.21.15.163:6832/1122018204 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+256663 (secure 0 0 0) 0x7f2cb8074860 con 0x7f2c940762d0 2024-05-27T09:37:42.168 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:37:42.170 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-05-27T09:37:42.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.171+0000 7f2cbcc00640 1 -- 172.21.15.114:0/2678552594 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2c940762d0 msgr2=0x7f2c940787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:42.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.171+0000 7f2cbcc00640 1 --2- 172.21.15.114:0/2678552594 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2c940762d0 0x7f2c940787e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8000f60 tx=0x7f2ca8009040 comp rx=0 tx=0).stop 2024-05-27T09:37:42.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.171+0000 7f2cbcc00640 1 -- 172.21.15.114:0/2678552594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cb815ad10 msgr2=0x7f2cb80ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:42.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.171+0000 7f2cbcc00640 1 --2- 172.21.15.114:0/2678552594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cb815ad10 0x7f2cb80ccdf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009e80 tx=0x7f2cb0004a40 comp rx=0 tx=0).stop 2024-05-27T09:37:42.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.171+0000 7f2cbcc00640 1 -- 172.21.15.114:0/2678552594 shutdown_connections 2024-05-27T09:37:42.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.171+0000 7f2cbcc00640 1 --2- 172.21.15.114:0/2678552594 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2c940762d0 0x7f2c940787e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:42.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.171+0000 7f2cbcc00640 1 --2- 172.21.15.114:0/2678552594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cb815ad10 0x7f2cb80ccdf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:42.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.171+0000 7f2cbcc00640 1 --2- 172.21.15.114:0/2678552594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cb80c9dd0 0x7f2cb8161ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:42.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.171+0000 7f2cbcc00640 1 --2- 172.21.15.114:0/2678552594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2cb8074a80 0x7f2cb81f6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:42.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.171+0000 7f2cbcc00640 1 -- 172.21.15.114:0/2678552594 >> 172.21.15.114:0/2678552594 conn(0x7f2cb80ca320 msgr2=0x7f2cb80763f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:42.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.171+0000 7f2cbcc00640 1 -- 172.21.15.114:0/2678552594 shutdown_connections 2024-05-27T09:37:42.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.171+0000 7f2cbcc00640 1 -- 172.21.15.114:0/2678552594 wait complete. 2024-05-27T09:37:42.199 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-05-27T09:37:40.940503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1141479215,"num_objects":10877,"num_object_clones":0,"num_object_copies":21754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10877,"num_whiteouts":0,"num_read":1810,"num_read_kb":502358,"num_write":25946,"num_write_kb":1569859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":105,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25920,"ondisk_log_size":25920,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2177880,"kb_used_data":1779928,"kb_used_omap":12,"kb_used_meta":397875,"kb_avail":752796840,"statfs":{"total":773094113280,"available":770863964160,"internally_reserved":0,"allocated":1822646272,"data_stored":2283396432,"data_compressed":218513410,"data_compressed_allocated":248946688,"data_compressed_original":767852544,"omap_allocated":12750,"internal_metadata":407424562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1943541,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":166,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003502"},"pg_stats":[{"pgid":"3.3f","version":"50'328","reported_seq":2448,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.380193+0000","last_change":"2024-05-27T09:33:11.447418+0000","last_active":"2024-05-27T09:37:37.380193+0000","last_peered":"2024-05-27T09:37:37.380193+0000","last_clean":"2024-05-27T09:37:37.380193+0000","last_became_active":"2024-05-27T09:27:18.585169+0000","last_became_peered":"2024-05-27T09:27:18.585169+0000","last_unstale":"2024-05-27T09:37:37.380193+0000","last_undegraded":"2024-05-27T09:37:37.380193+0000","last_fullsized":"2024-05-27T09:37:37.380193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:30:18.926000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01102325,"stat_sum":{"num_bytes":23698106,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":17,"num_read_kb":4111,"num_write":320,"num_write_kb":27403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"50'313","reported_seq":2432,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.133411+0000","last_change":"2024-05-27T09:33:11.731804+0000","last_active":"2024-05-27T09:37:20.133411+0000","last_peered":"2024-05-27T09:37:20.133411+0000","last_clean":"2024-05-27T09:37:20.133411+0000","last_became_active":"2024-05-27T09:27:18.566269+0000","last_became_peered":"2024-05-27T09:27:18.566269+0000","last_unstale":"2024-05-27T09:37:20.133411+0000","last_undegraded":"2024-05-27T09:37:20.133411+0000","last_fullsized":"2024-05-27T09:37:20.133411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:30:51.472078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061074720000000001,"stat_sum":{"num_bytes":5460707,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":313,"num_write_kb":13693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"50'342","reported_seq":2469,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.128558+0000","last_change":"2024-05-27T09:33:11.731326+0000","last_active":"2024-05-27T09:37:20.128558+0000","last_peered":"2024-05-27T09:37:20.128558+0000","last_clean":"2024-05-27T09:37:20.128558+0000","last_became_active":"2024-05-27T09:27:18.560620+0000","last_became_peered":"2024-05-27T09:27:18.560620+0000","last_unstale":"2024-05-27T09:37:20.128558+0000","last_undegraded":"2024-05-27T09:37:20.128558+0000","last_fullsized":"2024-05-27T09:37:20.128558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:02:04.553273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005599354,"stat_sum":{"num_bytes":19266359,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":24,"num_read_kb":17715,"num_write":342,"num_write_kb":39437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"50'307","reported_seq":2421,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.378615+0000","last_change":"2024-05-27T09:33:11.450825+0000","last_active":"2024-05-27T09:37:37.378615+0000","last_peered":"2024-05-27T09:37:37.378615+0000","last_clean":"2024-05-27T09:37:37.378615+0000","last_became_active":"2024-05-27T09:27:18.569481+0000","last_became_peered":"2024-05-27T09:27:18.569481+0000","last_unstale":"2024-05-27T09:37:37.378615+0000","last_undegraded":"2024-05-27T09:37:37.378615+0000","last_fullsized":"2024-05-27T09:37:37.378615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:19:26.159843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007726882,"stat_sum":{"num_bytes":6546535,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":298,"num_write_kb":6559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"50'375","reported_seq":2496,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.373663+0000","last_change":"2024-05-27T09:33:11.440268+0000","last_active":"2024-05-27T09:37:37.373663+0000","last_peered":"2024-05-27T09:37:37.373663+0000","last_clean":"2024-05-27T09:37:37.373663+0000","last_became_active":"2024-05-27T09:27:18.567795+0000","last_became_peered":"2024-05-27T09:27:18.567795+0000","last_unstale":"2024-05-27T09:37:37.373663+0000","last_undegraded":"2024-05-27T09:37:37.373663+0000","last_fullsized":"2024-05-27T09:37:37.373663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:31.321140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049759740000000002,"stat_sum":{"num_bytes":10629514,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":371,"num_write_kb":12093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"50'350","reported_seq":2475,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.371402+0000","last_change":"2024-05-27T09:33:11.445483+0000","last_active":"2024-05-27T09:37:37.371402+0000","last_peered":"2024-05-27T09:37:37.371402+0000","last_clean":"2024-05-27T09:37:37.371402+0000","last_became_active":"2024-05-27T09:27:18.573576+0000","last_became_peered":"2024-05-27T09:27:18.573576+0000","last_unstale":"2024-05-27T09:37:37.371402+0000","last_undegraded":"2024-05-27T09:37:37.371402+0000","last_fullsized":"2024-05-27T09:37:37.371402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:45:04.738799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074740400000000004,"stat_sum":{"num_bytes":30579450,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":22,"num_read_kb":8211,"num_write":344,"num_write_kb":38238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"50'360","reported_seq":2481,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.370720+0000","last_change":"2024-05-27T09:33:11.448897+0000","last_active":"2024-05-27T09:37:37.370720+0000","last_peered":"2024-05-27T09:37:37.370720+0000","last_clean":"2024-05-27T09:37:37.370720+0000","last_became_active":"2024-05-27T09:27:18.542593+0000","last_became_peered":"2024-05-27T09:27:18.542593+0000","last_unstale":"2024-05-27T09:37:37.370720+0000","last_undegraded":"2024-05-27T09:37:37.370720+0000","last_fullsized":"2024-05-27T09:37:37.370720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:11:49.487143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014070156,"stat_sum":{"num_bytes":5299988,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":18,"num_read_kb":4113,"num_write":355,"num_write_kb":9461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"50'320","reported_seq":2436,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.234926+0000","last_change":"2024-05-27T09:33:11.732725+0000","last_active":"2024-05-27T09:37:20.234926+0000","last_peered":"2024-05-27T09:37:20.234926+0000","last_clean":"2024-05-27T09:37:20.234926+0000","last_became_active":"2024-05-27T09:27:18.561788+0000","last_became_peered":"2024-05-27T09:27:18.561788+0000","last_unstale":"2024-05-27T09:37:20.234926+0000","last_undegraded":"2024-05-27T09:37:20.234926+0000","last_fullsized":"2024-05-27T09:37:20.234926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:51:29.653992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065132080000000004,"stat_sum":{"num_bytes":27492830,"num_objects":191,"num_object_clones":0,"num_object_copies":382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":191,"num_whiteouts":0,"num_read":13,"num_read_kb":4107,"num_write":319,"num_write_kb":27008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"50'338","reported_seq":2463,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.369696+0000","last_change":"2024-05-27T09:33:11.739498+0000","last_active":"2024-05-27T09:37:37.369696+0000","last_peered":"2024-05-27T09:37:37.369696+0000","last_clean":"2024-05-27T09:37:37.369696+0000","last_became_active":"2024-05-27T09:27:18.582041+0000","last_became_peered":"2024-05-27T09:27:18.582041+0000","last_unstale":"2024-05-27T09:37:37.369696+0000","last_undegraded":"2024-05-27T09:37:37.369696+0000","last_fullsized":"2024-05-27T09:37:37.369696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:29:10.337097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0083382379999999996,"stat_sum":{"num_bytes":9733898,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":22,"num_read_kb":8210,"num_write":328,"num_write_kb":30110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"50'321","reported_seq":2441,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.392171+0000","last_change":"2024-05-27T09:33:11.446829+0000","last_active":"2024-05-27T09:37:37.392171+0000","last_peered":"2024-05-27T09:37:37.392171+0000","last_clean":"2024-05-27T09:37:37.392171+0000","last_became_active":"2024-05-27T09:27:18.574409+0000","last_became_peered":"2024-05-27T09:27:18.574409+0000","last_unstale":"2024-05-27T09:37:37.392171+0000","last_undegraded":"2024-05-27T09:37:37.392171+0000","last_fullsized":"2024-05-27T09:37:37.392171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:41:37.603718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010493662000000001,"stat_sum":{"num_bytes":13833965,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":17,"num_read_kb":4112,"num_write":306,"num_write_kb":13622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"50'319","reported_seq":2444,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.384747+0000","last_change":"2024-05-27T09:33:11.443445+0000","last_active":"2024-05-27T09:37:37.384747+0000","last_peered":"2024-05-27T09:37:37.384747+0000","last_clean":"2024-05-27T09:37:37.384747+0000","last_became_active":"2024-05-27T09:27:18.578679+0000","last_became_peered":"2024-05-27T09:27:18.578679+0000","last_unstale":"2024-05-27T09:37:37.384747+0000","last_undegraded":"2024-05-27T09:37:37.384747+0000","last_fullsized":"2024-05-27T09:37:37.384747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:33:46.683180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080365679999999991,"stat_sum":{"num_bytes":26002241,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":22,"num_read_kb":7043,"num_write":312,"num_write_kb":25555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"50'294","reported_seq":2402,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.218006+0000","last_change":"2024-05-27T09:33:11.625670+0000","last_active":"2024-05-27T09:37:20.218006+0000","last_peered":"2024-05-27T09:37:20.218006+0000","last_clean":"2024-05-27T09:37:20.218006+0000","last_became_active":"2024-05-27T09:28:06.207562+0000","last_became_peered":"2024-05-27T09:28:06.207562+0000","last_unstale":"2024-05-27T09:37:20.218006+0000","last_undegraded":"2024-05-27T09:37:20.218006+0000","last_fullsized":"2024-05-27T09:37:20.218006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:52:32.773329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001018333,"stat_sum":{"num_bytes":11019660,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":12,"num_read_kb":8201,"num_write":293,"num_write_kb":23198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"50'298","reported_seq":2422,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.265350+0000","last_change":"2024-05-27T09:33:11.728591+0000","last_active":"2024-05-27T09:37:20.265350+0000","last_peered":"2024-05-27T09:37:20.265350+0000","last_clean":"2024-05-27T09:37:20.265350+0000","last_became_active":"2024-05-27T09:27:18.561493+0000","last_became_peered":"2024-05-27T09:27:18.561493+0000","last_unstale":"2024-05-27T09:37:20.265350+0000","last_undegraded":"2024-05-27T09:37:20.265350+0000","last_fullsized":"2024-05-27T09:37:20.265350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:22.877083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024650470000000002,"stat_sum":{"num_bytes":14891482,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":21,"num_read_kb":4116,"num_write":298,"num_write_kb":31046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"50'335","reported_seq":2456,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.131958+0000","last_change":"2024-05-27T09:33:11.727658+0000","last_active":"2024-05-27T09:37:20.131958+0000","last_peered":"2024-05-27T09:37:20.131958+0000","last_clean":"2024-05-27T09:37:20.131958+0000","last_became_active":"2024-05-27T09:27:18.562843+0000","last_became_peered":"2024-05-27T09:27:18.562843+0000","last_unstale":"2024-05-27T09:37:20.131958+0000","last_undegraded":"2024-05-27T09:37:20.131958+0000","last_fullsized":"2024-05-27T09:37:20.131958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:31.147919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025802960000000002,"stat_sum":{"num_bytes":6434369,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":335,"num_write_kb":10518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"50'357","reported_seq":2482,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.244846+0000","last_change":"2024-05-27T09:33:11.729192+0000","last_active":"2024-05-27T09:37:20.244846+0000","last_peered":"2024-05-27T09:37:20.244846+0000","last_clean":"2024-05-27T09:37:20.244846+0000","last_became_active":"2024-05-27T09:27:18.566108+0000","last_became_peered":"2024-05-27T09:27:18.566108+0000","last_unstale":"2024-05-27T09:37:20.244846+0000","last_undegraded":"2024-05-27T09:37:20.244846+0000","last_fullsized":"2024-05-27T09:37:20.244846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:39:23.102944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041189030000000001,"stat_sum":{"num_bytes":20283045,"num_objects":201,"num_object_clones":0,"num_object_copies":402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":201,"num_whiteouts":0,"num_read":22,"num_read_kb":6627,"num_write":357,"num_write_kb":19995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"50'369","reported_seq":2493,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.386640+0000","last_change":"2024-05-27T09:33:11.457079+0000","last_active":"2024-05-27T09:37:37.386640+0000","last_peered":"2024-05-27T09:37:37.386640+0000","last_clean":"2024-05-27T09:37:37.386640+0000","last_became_active":"2024-05-27T09:27:18.581719+0000","last_became_peered":"2024-05-27T09:27:18.581719+0000","last_unstale":"2024-05-27T09:37:37.386640+0000","last_undegraded":"2024-05-27T09:37:37.386640+0000","last_fullsized":"2024-05-27T09:37:37.386640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:59:21.546098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.008579415,"stat_sum":{"num_bytes":24362280,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":22,"num_read_kb":5287,"num_write":359,"num_write_kb":31349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"50'366","reported_seq":2491,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.387571+0000","last_change":"2024-05-27T09:33:11.458057+0000","last_active":"2024-05-27T09:37:37.387571+0000","last_peered":"2024-05-27T09:37:37.387571+0000","last_clean":"2024-05-27T09:37:37.387571+0000","last_became_active":"2024-05-27T09:27:18.577193+0000","last_became_peered":"2024-05-27T09:27:18.577193+0000","last_unstale":"2024-05-27T09:37:37.387571+0000","last_undegraded":"2024-05-27T09:37:37.387571+0000","last_fullsized":"2024-05-27T09:37:37.387571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:41:12.903002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079143630000000006,"stat_sum":{"num_bytes":23132412,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":23,"num_read_kb":8093,"num_write":357,"num_write_kb":30967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"50'297","reported_seq":2428,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.284414+0000","last_change":"2024-05-27T09:33:11.626051+0000","last_active":"2024-05-27T09:37:20.284414+0000","last_peered":"2024-05-27T09:37:20.284414+0000","last_clean":"2024-05-27T09:37:20.284414+0000","last_became_active":"2024-05-27T09:27:18.570712+0000","last_became_peered":"2024-05-27T09:27:18.570712+0000","last_unstale":"2024-05-27T09:37:20.284414+0000","last_undegraded":"2024-05-27T09:37:20.284414+0000","last_fullsized":"2024-05-27T09:37:20.284414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:32:58.266174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767629999999999,"stat_sum":{"num_bytes":13973992,"num_objects":138,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":0,"num_read":28,"num_read_kb":8218,"num_write":296,"num_write_kb":21985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"50'323","reported_seq":2443,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.367564+0000","last_change":"2024-05-27T09:33:11.456909+0000","last_active":"2024-05-27T09:37:37.367564+0000","last_peered":"2024-05-27T09:37:37.367564+0000","last_clean":"2024-05-27T09:37:37.367564+0000","last_became_active":"2024-05-27T09:27:18.564643+0000","last_became_peered":"2024-05-27T09:27:18.564643+0000","last_unstale":"2024-05-27T09:37:37.367564+0000","last_undegraded":"2024-05-27T09:37:37.367564+0000","last_fullsized":"2024-05-27T09:37:37.367564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:14:07.864139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0091687629999999999,"stat_sum":{"num_bytes":2873482,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":18,"num_read_kb":4113,"num_write":315,"num_write_kb":11167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"50'345","reported_seq":2469,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.376407+0000","last_change":"2024-05-27T09:33:11.452205+0000","last_active":"2024-05-27T09:37:37.376407+0000","last_peered":"2024-05-27T09:37:37.376407+0000","last_clean":"2024-05-27T09:37:37.376407+0000","last_became_active":"2024-05-27T09:27:18.543777+0000","last_became_peered":"2024-05-27T09:27:18.543777+0000","last_unstale":"2024-05-27T09:37:37.376407+0000","last_undegraded":"2024-05-27T09:37:37.376407+0000","last_fullsized":"2024-05-27T09:37:37.376407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:11:45.868921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012534072,"stat_sum":{"num_bytes":9859903,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":21,"num_read_kb":4116,"num_write":337,"num_write_kb":17999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"50'326","reported_seq":2451,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.216955+0000","last_change":"2024-05-27T09:33:11.732683+0000","last_active":"2024-05-27T09:37:20.216955+0000","last_peered":"2024-05-27T09:37:20.216955+0000","last_clean":"2024-05-27T09:37:20.216955+0000","last_became_active":"2024-05-27T09:27:18.570948+0000","last_became_peered":"2024-05-27T09:27:18.570948+0000","last_unstale":"2024-05-27T09:37:20.216955+0000","last_undegraded":"2024-05-27T09:37:20.216955+0000","last_fullsized":"2024-05-27T09:37:20.216955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:03:03.895209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040840829999999996,"stat_sum":{"num_bytes":5923849,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":325,"num_write_kb":5958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"50'343","reported_seq":2466,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.383214+0000","last_change":"2024-05-27T09:33:11.737682+0000","last_active":"2024-05-27T09:37:37.383214+0000","last_peered":"2024-05-27T09:37:37.383214+0000","last_clean":"2024-05-27T09:37:37.383214+0000","last_became_active":"2024-05-27T09:27:18.543921+0000","last_became_peered":"2024-05-27T09:27:18.543921+0000","last_unstale":"2024-05-27T09:37:37.383214+0000","last_undegraded":"2024-05-27T09:37:37.383214+0000","last_fullsized":"2024-05-27T09:37:37.383214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:47:03.620703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0094259180000000001,"stat_sum":{"num_bytes":14450069,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":20,"num_read_kb":8209,"num_write":332,"num_write_kb":18385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"50'329","reported_seq":2463,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.233562+0000","last_change":"2024-05-27T09:33:11.731031+0000","last_active":"2024-05-27T09:37:20.233562+0000","last_peered":"2024-05-27T09:37:20.233562+0000","last_clean":"2024-05-27T09:37:20.233562+0000","last_became_active":"2024-05-27T09:27:18.575115+0000","last_became_peered":"2024-05-27T09:27:18.575115+0000","last_unstale":"2024-05-27T09:37:20.233562+0000","last_undegraded":"2024-05-27T09:37:20.233562+0000","last_fullsized":"2024-05-27T09:37:20.233562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:56:57.284871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054063330000000001,"stat_sum":{"num_bytes":18656408,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":31,"num_read_kb":19450,"num_write":329,"num_write_kb":38861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"50'353","reported_seq":2474,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.367488+0000","last_change":"2024-05-27T09:33:11.451714+0000","last_active":"2024-05-27T09:37:37.367488+0000","last_peered":"2024-05-27T09:37:37.367488+0000","last_clean":"2024-05-27T09:37:37.367488+0000","last_became_active":"2024-05-27T09:27:18.563991+0000","last_became_peered":"2024-05-27T09:27:18.563991+0000","last_unstale":"2024-05-27T09:37:37.367488+0000","last_undegraded":"2024-05-27T09:37:37.367488+0000","last_fullsized":"2024-05-27T09:37:37.367488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:41:37.963962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009841753,"stat_sum":{"num_bytes":13963475,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":19,"num_read_kb":8208,"num_write":347,"num_write_kb":22013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"50'330","reported_seq":2454,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.234416+0000","last_change":"2024-05-27T09:33:11.732272+0000","last_active":"2024-05-27T09:37:20.234416+0000","last_peered":"2024-05-27T09:37:20.234416+0000","last_clean":"2024-05-27T09:37:20.234416+0000","last_became_active":"2024-05-27T09:27:18.562803+0000","last_became_peered":"2024-05-27T09:27:18.562803+0000","last_unstale":"2024-05-27T09:37:20.234416+0000","last_undegraded":"2024-05-27T09:37:20.234416+0000","last_fullsized":"2024-05-27T09:37:20.234416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:09:56.174365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067443290000000003,"stat_sum":{"num_bytes":23071782,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":21,"num_read_kb":4115,"num_write":328,"num_write_kb":30886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"50'384","reported_seq":2507,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.393993+0000","last_change":"2024-05-27T09:33:11.453888+0000","last_active":"2024-05-27T09:37:37.393993+0000","last_peered":"2024-05-27T09:37:37.393993+0000","last_clean":"2024-05-27T09:37:37.393993+0000","last_became_active":"2024-05-27T09:27:18.590083+0000","last_became_peered":"2024-05-27T09:27:18.590083+0000","last_unstale":"2024-05-27T09:37:37.393993+0000","last_undegraded":"2024-05-27T09:37:37.393993+0000","last_fullsized":"2024-05-27T09:37:37.393993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:00.006136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.008181008,"stat_sum":{"num_bytes":14547405,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":20,"num_read_kb":4115,"num_write":369,"num_write_kb":14397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"50'326","reported_seq":2445,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.169040+0000","last_change":"2024-05-27T09:33:11.728099+0000","last_active":"2024-05-27T09:37:20.169040+0000","last_peered":"2024-05-27T09:37:20.169040+0000","last_clean":"2024-05-27T09:37:20.169040+0000","last_became_active":"2024-05-27T09:27:18.557953+0000","last_became_peered":"2024-05-27T09:27:18.557953+0000","last_unstale":"2024-05-27T09:37:20.169040+0000","last_undegraded":"2024-05-27T09:37:20.169040+0000","last_fullsized":"2024-05-27T09:37:20.169040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:03:03.895209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027117650000000001,"stat_sum":{"num_bytes":18627167,"num_objects":187,"num_object_clones":0,"num_object_copies":374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":187,"num_whiteouts":0,"num_read":16,"num_read_kb":4110,"num_write":325,"num_write_kb":26562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"50'303","reported_seq":2423,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.207979+0000","last_change":"2024-05-27T09:33:11.729430+0000","last_active":"2024-05-27T09:37:20.207979+0000","last_peered":"2024-05-27T09:37:20.207979+0000","last_clean":"2024-05-27T09:37:20.207979+0000","last_became_active":"2024-05-27T09:27:18.562839+0000","last_became_peered":"2024-05-27T09:27:18.562839+0000","last_unstale":"2024-05-27T09:37:20.207979+0000","last_undegraded":"2024-05-27T09:37:20.207979+0000","last_fullsized":"2024-05-27T09:37:20.207979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:21:17.734561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032647399999999999,"stat_sum":{"num_bytes":14533754,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":17,"num_read_kb":11479,"num_write":303,"num_write_kb":21724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"50'344","reported_seq":2467,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.133428+0000","last_change":"2024-05-27T09:33:11.730679+0000","last_active":"2024-05-27T09:37:20.133428+0000","last_peered":"2024-05-27T09:37:20.133428+0000","last_clean":"2024-05-27T09:37:20.133428+0000","last_became_active":"2024-05-27T09:27:18.561475+0000","last_became_peered":"2024-05-27T09:27:18.561475+0000","last_unstale":"2024-05-27T09:37:20.133428+0000","last_undegraded":"2024-05-27T09:37:20.133428+0000","last_fullsized":"2024-05-27T09:37:20.133428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:32:06.826783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053708779999999999,"stat_sum":{"num_bytes":11382190,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":20,"num_read_kb":4114,"num_write":347,"num_write_kb":19366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"50'377","reported_seq":2507,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.357466+0000","last_change":"2024-05-27T09:33:11.454566+0000","last_active":"2024-05-27T09:37:37.357466+0000","last_peered":"2024-05-27T09:37:37.357466+0000","last_clean":"2024-05-27T09:37:37.357466+0000","last_became_active":"2024-05-27T09:27:18.563103+0000","last_became_peered":"2024-05-27T09:27:18.563103+0000","last_unstale":"2024-05-27T09:37:37.357466+0000","last_undegraded":"2024-05-27T09:37:37.357466+0000","last_fullsized":"2024-05-27T09:37:37.357466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:25.776404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011066041,"stat_sum":{"num_bytes":30187091,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":28,"num_read_kb":16405,"num_write":369,"num_write_kb":50152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"50'389","reported_seq":2511,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.231487+0000","last_change":"2024-05-27T09:33:11.625969+0000","last_active":"2024-05-27T09:37:20.231487+0000","last_peered":"2024-05-27T09:37:20.231487+0000","last_clean":"2024-05-27T09:37:20.231487+0000","last_became_active":"2024-05-27T09:27:18.560244+0000","last_became_peered":"2024-05-27T09:27:18.560244+0000","last_unstale":"2024-05-27T09:37:20.231487+0000","last_undegraded":"2024-05-27T09:37:20.231487+0000","last_fullsized":"2024-05-27T09:37:20.231487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:14:56.979490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012131959999999999,"stat_sum":{"num_bytes":8541274,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":19,"num_read_kb":2631,"num_write":389,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"50'363","reported_seq":2493,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.377327+0000","last_change":"2024-05-27T09:33:11.447687+0000","last_active":"2024-05-27T09:37:37.377327+0000","last_peered":"2024-05-27T09:37:37.377327+0000","last_clean":"2024-05-27T09:37:37.377327+0000","last_became_active":"2024-05-27T09:27:18.567845+0000","last_became_peered":"2024-05-27T09:27:18.567845+0000","last_unstale":"2024-05-27T09:37:37.377327+0000","last_undegraded":"2024-05-27T09:37:37.377327+0000","last_fullsized":"2024-05-27T09:37:37.377327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:48:36.494196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082936180000000009,"stat_sum":{"num_bytes":1444930,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":352,"num_write_kb":9783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"50'336","reported_seq":2458,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.370918+0000","last_change":"2024-05-27T09:33:11.738447+0000","last_active":"2024-05-27T09:37:37.370918+0000","last_peered":"2024-05-27T09:37:37.370918+0000","last_clean":"2024-05-27T09:37:37.370918+0000","last_became_active":"2024-05-27T09:27:18.556156+0000","last_became_peered":"2024-05-27T09:27:18.556156+0000","last_unstale":"2024-05-27T09:37:37.370918+0000","last_undegraded":"2024-05-27T09:37:37.370918+0000","last_fullsized":"2024-05-27T09:37:37.370918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:25:11.600058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0094378269999999993,"stat_sum":{"num_bytes":19241078,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":19,"num_read_kb":8208,"num_write":329,"num_write_kb":23062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"50'73","reported_seq":156,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:21.449092+0000","last_change":"2024-05-27T09:27:17.548038+0000","last_active":"2024-05-27T09:37:21.449092+0000","last_peered":"2024-05-27T09:37:21.449092+0000","last_clean":"2024-05-27T09:37:21.449092+0000","last_became_active":"2024-05-27T09:27:17.547653+0000","last_became_peered":"2024-05-27T09:27:17.547653+0000","last_unstale":"2024-05-27T09:37:21.449092+0000","last_undegraded":"2024-05-27T09:37:21.449092+0000","last_fullsized":"2024-05-27T09:37:21.449092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195412,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":4100,"num_write":71,"num_write_kb":4220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"50'320","reported_seq":2441,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.327073+0000","last_change":"2024-05-27T09:33:11.733348+0000","last_active":"2024-05-27T09:37:20.327073+0000","last_peered":"2024-05-27T09:37:20.327073+0000","last_clean":"2024-05-27T09:37:20.327073+0000","last_became_active":"2024-05-27T09:27:18.542715+0000","last_became_peered":"2024-05-27T09:27:18.542715+0000","last_unstale":"2024-05-27T09:37:20.327073+0000","last_undegraded":"2024-05-27T09:37:20.327073+0000","last_fullsized":"2024-05-27T09:37:20.327073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:45:47.000935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004594323,"stat_sum":{"num_bytes":10080804,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":19,"num_read_kb":4113,"num_write":319,"num_write_kb":14102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"49'299","reported_seq":384,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.547628+0000","last_change":"2024-05-27T09:27:17.570014+0000","last_active":"2024-05-27T09:37:15.547628+0000","last_peered":"2024-05-27T09:37:15.547628+0000","last_clean":"2024-05-27T09:37:15.547628+0000","last_became_active":"2024-05-27T09:27:17.569673+0000","last_became_peered":"2024-05-27T09:27:17.569673+0000","last_unstale":"2024-05-27T09:37:15.547628+0000","last_undegraded":"2024-05-27T09:37:15.547628+0000","last_fullsized":"2024-05-27T09:37:15.547628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:26.210041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":22,"num_read_kb":12200,"num_write":300,"num_write_kb":12509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"49'46","reported_seq":129,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.952341+0000","last_change":"2024-05-27T09:27:17.565489+0000","last_active":"2024-05-27T09:37:15.952341+0000","last_peered":"2024-05-27T09:37:15.952341+0000","last_clean":"2024-05-27T09:37:15.952341+0000","last_became_active":"2024-05-27T09:27:17.565137+0000","last_became_peered":"2024-05-27T09:27:17.565137+0000","last_unstale":"2024-05-27T09:37:15.952341+0000","last_undegraded":"2024-05-27T09:37:15.952341+0000","last_fullsized":"2024-05-27T09:37:15.952341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":19,"num_read_kb":4114,"num_write":50,"num_write_kb":4167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"50'389","reported_seq":2515,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.164212+0000","last_change":"2024-05-27T09:33:11.733228+0000","last_active":"2024-05-27T09:37:20.164212+0000","last_peered":"2024-05-27T09:37:20.164212+0000","last_clean":"2024-05-27T09:37:20.164212+0000","last_became_active":"2024-05-27T09:27:18.564115+0000","last_became_peered":"2024-05-27T09:27:18.564115+0000","last_unstale":"2024-05-27T09:37:20.164212+0000","last_undegraded":"2024-05-27T09:37:20.164212+0000","last_fullsized":"2024-05-27T09:37:20.164212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:06:44.772830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066511199999999999,"stat_sum":{"num_bytes":10503109,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":23,"num_read_kb":4118,"num_write":389,"num_write_kb":18659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"50'294","reported_seq":371,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.547032+0000","last_change":"2024-05-27T09:27:17.561204+0000","last_active":"2024-05-27T09:37:15.547032+0000","last_peered":"2024-05-27T09:37:15.547032+0000","last_clean":"2024-05-27T09:37:15.547032+0000","last_became_active":"2024-05-27T09:27:17.560149+0000","last_became_peered":"2024-05-27T09:27:17.560149+0000","last_unstale":"2024-05-27T09:37:15.547032+0000","last_undegraded":"2024-05-27T09:37:15.547032+0000","last_fullsized":"2024-05-27T09:37:15.547032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.284270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":291,"num_write_kb":12506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"50'334","reported_seq":2456,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.237522+0000","last_change":"2024-05-27T09:33:11.734046+0000","last_active":"2024-05-27T09:37:20.237522+0000","last_peered":"2024-05-27T09:37:20.237522+0000","last_clean":"2024-05-27T09:37:20.237522+0000","last_became_active":"2024-05-27T09:27:18.541946+0000","last_became_peered":"2024-05-27T09:27:18.541946+0000","last_unstale":"2024-05-27T09:37:20.237522+0000","last_undegraded":"2024-05-27T09:37:20.237522+0000","last_fullsized":"2024-05-27T09:37:20.237522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:30.087688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030229530000000001,"stat_sum":{"num_bytes":19296821,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":20,"num_read_kb":12305,"num_write":330,"num_write_kb":31302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"50'9","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.530539+0000","last_change":"2024-05-27T09:27:17.561714+0000","last_active":"2024-05-27T09:37:20.530539+0000","last_peered":"2024-05-27T09:37:20.530539+0000","last_clean":"2024-05-27T09:37:20.530539+0000","last_became_active":"2024-05-27T09:27:17.561355+0000","last_became_peered":"2024-05-27T09:27:17.561355+0000","last_unstale":"2024-05-27T09:37:20.530539+0000","last_undegraded":"2024-05-27T09:37:20.530539+0000","last_fullsized":"2024-05-27T09:37:20.530539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:07:02.481451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":8,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"50'354","reported_seq":2478,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.394166+0000","last_change":"2024-05-27T09:33:11.454088+0000","last_active":"2024-05-27T09:37:37.394166+0000","last_peered":"2024-05-27T09:37:37.394166+0000","last_clean":"2024-05-27T09:37:37.394166+0000","last_became_active":"2024-05-27T09:27:18.587921+0000","last_became_peered":"2024-05-27T09:27:18.587921+0000","last_unstale":"2024-05-27T09:37:37.394166+0000","last_undegraded":"2024-05-27T09:37:37.394166+0000","last_fullsized":"2024-05-27T09:37:37.394166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:17:20.248600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0078895319999999994,"stat_sum":{"num_bytes":10502055,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":21,"num_read_kb":4116,"num_write":345,"num_write_kb":10394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"49'65","reported_seq":144,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.540718+0000","last_change":"2024-05-27T09:27:17.565854+0000","last_active":"2024-05-27T09:37:15.540718+0000","last_peered":"2024-05-27T09:37:15.540718+0000","last_clean":"2024-05-27T09:37:15.540718+0000","last_became_active":"2024-05-27T09:27:17.565488+0000","last_became_peered":"2024-05-27T09:27:17.565488+0000","last_unstale":"2024-05-27T09:37:15.540718+0000","last_undegraded":"2024-05-27T09:37:15.540718+0000","last_fullsized":"2024-05-27T09:37:15.540718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:53.711508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":4102,"num_write":67,"num_write_kb":4232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"50'329","reported_seq":2455,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.223387+0000","last_change":"2024-05-27T09:33:11.733328+0000","last_active":"2024-05-27T09:37:20.223387+0000","last_peered":"2024-05-27T09:37:20.223387+0000","last_clean":"2024-05-27T09:37:20.223387+0000","last_became_active":"2024-05-27T09:27:18.585243+0000","last_became_peered":"2024-05-27T09:27:18.585243+0000","last_unstale":"2024-05-27T09:37:20.223387+0000","last_undegraded":"2024-05-27T09:37:20.223387+0000","last_fullsized":"2024-05-27T09:37:20.223387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:18:44.814432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067133569999999997,"stat_sum":{"num_bytes":18063712,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":23,"num_read_kb":8211,"num_write":328,"num_write_kb":30099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"50'204","reported_seq":284,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.529879+0000","last_change":"2024-05-27T09:27:17.548651+0000","last_active":"2024-05-27T09:37:20.529879+0000","last_peered":"2024-05-27T09:37:20.529879+0000","last_clean":"2024-05-27T09:37:20.529879+0000","last_became_active":"2024-05-27T09:27:17.548307+0000","last_became_peered":"2024-05-27T09:27:17.548307+0000","last_unstale":"2024-05-27T09:37:20.529879+0000","last_undegraded":"2024-05-27T09:37:20.529879+0000","last_fullsized":"2024-05-27T09:37:20.529879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:48.905422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":206,"num_write_kb":4295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"50'341","reported_seq":2460,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.387494+0000","last_change":"2024-05-27T09:33:11.455764+0000","last_active":"2024-05-27T09:37:37.387494+0000","last_peered":"2024-05-27T09:37:37.387494+0000","last_clean":"2024-05-27T09:37:37.387494+0000","last_became_active":"2024-05-27T09:27:18.581742+0000","last_became_peered":"2024-05-27T09:27:18.581742+0000","last_unstale":"2024-05-27T09:37:37.387494+0000","last_undegraded":"2024-05-27T09:37:37.387494+0000","last_fullsized":"2024-05-27T09:37:37.387494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:14:45.868801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089613100000000001,"stat_sum":{"num_bytes":20607307,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":17,"num_read_kb":10914,"num_write":328,"num_write_kb":24390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"50'245","reported_seq":328,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.536925+0000","last_change":"2024-05-27T09:27:17.556419+0000","last_active":"2024-05-27T09:37:15.536925+0000","last_peered":"2024-05-27T09:37:15.536925+0000","last_clean":"2024-05-27T09:37:15.536925+0000","last_became_active":"2024-05-27T09:27:17.556036+0000","last_became_peered":"2024-05-27T09:27:17.556036+0000","last_unstale":"2024-05-27T09:37:15.536925+0000","last_undegraded":"2024-05-27T09:37:15.536925+0000","last_fullsized":"2024-05-27T09:37:15.536925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:48:20.718085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":13,"num_read_kb":8203,"num_write":243,"num_write_kb":12459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"50'335","reported_seq":2457,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.383012+0000","last_change":"2024-05-27T09:33:11.449451+0000","last_active":"2024-05-27T09:37:37.383012+0000","last_peered":"2024-05-27T09:37:37.383012+0000","last_clean":"2024-05-27T09:37:37.383012+0000","last_became_active":"2024-05-27T09:27:18.565500+0000","last_became_peered":"2024-05-27T09:27:18.565500+0000","last_unstale":"2024-05-27T09:37:37.383012+0000","last_undegraded":"2024-05-27T09:37:37.383012+0000","last_fullsized":"2024-05-27T09:37:37.383012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:25:55.844796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011618516000000001,"stat_sum":{"num_bytes":10741446,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":19,"num_read_kb":4114,"num_write":329,"num_write_kb":14761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"49'17","reported_seq":105,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.861134+0000","last_change":"2024-05-27T09:27:17.572168+0000","last_active":"2024-05-27T09:37:15.861134+0000","last_peered":"2024-05-27T09:37:15.861134+0000","last_clean":"2024-05-27T09:37:15.861134+0000","last_became_active":"2024-05-27T09:27:17.571709+0000","last_became_peered":"2024-05-27T09:27:17.571709+0000","last_unstale":"2024-05-27T09:37:15.861134+0000","last_undegraded":"2024-05-27T09:37:15.861134+0000","last_fullsized":"2024-05-27T09:37:15.861134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":30,"num_read_kb":8224,"num_write":18,"num_write_kb":8221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"50'334","reported_seq":2463,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.312575+0000","last_change":"2024-05-27T09:33:11.728047+0000","last_active":"2024-05-27T09:37:20.312575+0000","last_peered":"2024-05-27T09:37:20.312575+0000","last_clean":"2024-05-27T09:37:20.312575+0000","last_became_active":"2024-05-27T09:27:18.564589+0000","last_became_peered":"2024-05-27T09:27:18.564589+0000","last_unstale":"2024-05-27T09:37:20.312575+0000","last_undegraded":"2024-05-27T09:37:20.312575+0000","last_fullsized":"2024-05-27T09:37:20.312575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:09:11.832557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028467779999999999,"stat_sum":{"num_bytes":14347466,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":26,"num_read_kb":16403,"num_write":332,"num_write_kb":30566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"50'8","reported_seq":85,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.547741+0000","last_change":"2024-05-27T09:27:17.561344+0000","last_active":"2024-05-27T09:37:15.547741+0000","last_peered":"2024-05-27T09:37:15.547741+0000","last_clean":"2024-05-27T09:37:15.547741+0000","last_became_active":"2024-05-27T09:27:17.554860+0000","last_became_peered":"2024-05-27T09:27:17.554860+0000","last_unstale":"2024-05-27T09:37:15.547741+0000","last_undegraded":"2024-05-27T09:37:15.547741+0000","last_fullsized":"2024-05-27T09:37:15.547741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:16.880101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"50'359","reported_seq":2484,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.391039+0000","last_change":"2024-05-27T09:33:11.456274+0000","last_active":"2024-05-27T09:37:37.391039+0000","last_peered":"2024-05-27T09:37:37.391039+0000","last_clean":"2024-05-27T09:37:37.391039+0000","last_became_active":"2024-05-27T09:27:18.572024+0000","last_became_peered":"2024-05-27T09:27:18.572024+0000","last_unstale":"2024-05-27T09:37:37.391039+0000","last_undegraded":"2024-05-27T09:37:37.391039+0000","last_fullsized":"2024-05-27T09:37:37.391039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:40:00.595445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010292084,"stat_sum":{"num_bytes":12019831,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":346,"num_write_kb":11826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"50'12","reported_seq":97,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.541647+0000","last_change":"2024-05-27T09:27:17.575859+0000","last_active":"2024-05-27T09:37:15.541647+0000","last_peered":"2024-05-27T09:37:15.541647+0000","last_clean":"2024-05-27T09:37:15.541647+0000","last_became_active":"2024-05-27T09:27:17.574695+0000","last_became_peered":"2024-05-27T09:27:17.574695+0000","last_unstale":"2024-05-27T09:37:15.541647+0000","last_undegraded":"2024-05-27T09:37:15.541647+0000","last_fullsized":"2024-05-27T09:37:15.541647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:24:19.379819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1331,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":9,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"50'340","reported_seq":2462,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.354340+0000","last_change":"2024-05-27T09:33:11.736362+0000","last_active":"2024-05-27T09:37:37.354340+0000","last_peered":"2024-05-27T09:37:37.354340+0000","last_clean":"2024-05-27T09:37:37.354340+0000","last_became_active":"2024-05-27T09:27:18.554932+0000","last_became_peered":"2024-05-27T09:27:18.554932+0000","last_unstale":"2024-05-27T09:37:37.354340+0000","last_undegraded":"2024-05-27T09:37:37.354340+0000","last_fullsized":"2024-05-27T09:37:37.354340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:32:26.268173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0083893030000000007,"stat_sum":{"num_bytes":24206949,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":20,"num_read_kb":12303,"num_write":329,"num_write_kb":23805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"49'68","reported_seq":161,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.631995+0000","last_change":"2024-05-27T09:27:17.547895+0000","last_active":"2024-05-27T09:37:15.631995+0000","last_peered":"2024-05-27T09:37:15.631995+0000","last_clean":"2024-05-27T09:37:15.631995+0000","last_became_active":"2024-05-27T09:27:17.547523+0000","last_became_peered":"2024-05-27T09:27:17.547523+0000","last_unstale":"2024-05-27T09:37:15.631995+0000","last_undegraded":"2024-05-27T09:37:15.631995+0000","last_fullsized":"2024-05-27T09:37:15.631995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":66,"num_write_kb":8220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"50'366","reported_seq":2491,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.385150+0000","last_change":"2024-05-27T09:33:11.457430+0000","last_active":"2024-05-27T09:37:37.385150+0000","last_peered":"2024-05-27T09:37:37.385150+0000","last_clean":"2024-05-27T09:37:37.385150+0000","last_became_active":"2024-05-27T09:27:18.582360+0000","last_became_peered":"2024-05-27T09:27:18.582360+0000","last_unstale":"2024-05-27T09:37:37.385150+0000","last_undegraded":"2024-05-27T09:37:37.385150+0000","last_fullsized":"2024-05-27T09:37:37.385150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:40:26.596775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0087858729999999996,"stat_sum":{"num_bytes":23260375,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":23,"num_read_kb":13874,"num_write":361,"num_write_kb":35144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"49'42","reported_seq":123,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.543499+0000","last_change":"2024-05-27T09:27:17.571481+0000","last_active":"2024-05-27T09:37:15.543499+0000","last_peered":"2024-05-27T09:37:15.543499+0000","last_clean":"2024-05-27T09:37:15.543499+0000","last_became_active":"2024-05-27T09:27:17.571146+0000","last_became_peered":"2024-05-27T09:27:17.571146+0000","last_unstale":"2024-05-27T09:37:15.543499+0000","last_undegraded":"2024-05-27T09:37:15.543499+0000","last_fullsized":"2024-05-27T09:37:15.543499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:18.038221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":8,"num_read_kb":4103,"num_write":60,"num_write_kb":4132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"50'302","reported_seq":2417,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.390053+0000","last_change":"2024-05-27T09:33:11.454615+0000","last_active":"2024-05-27T09:37:37.390053+0000","last_peered":"2024-05-27T09:37:37.390053+0000","last_clean":"2024-05-27T09:37:37.390053+0000","last_became_active":"2024-05-27T09:27:18.579440+0000","last_became_peered":"2024-05-27T09:27:18.579440+0000","last_unstale":"2024-05-27T09:37:37.390053+0000","last_undegraded":"2024-05-27T09:37:37.390053+0000","last_fullsized":"2024-05-27T09:37:37.390053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:06:43.673978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079260630000000006,"stat_sum":{"num_bytes":7949059,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":12,"num_read_kb":4346,"num_write":296,"num_write_kb":16109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"49'27","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.632660+0000","last_change":"2024-05-27T09:27:17.551295+0000","last_active":"2024-05-27T09:37:15.632660+0000","last_peered":"2024-05-27T09:37:15.632660+0000","last_clean":"2024-05-27T09:37:15.632660+0000","last_became_active":"2024-05-27T09:27:17.550861+0000","last_became_peered":"2024-05-27T09:27:17.550861+0000","last_unstale":"2024-05-27T09:37:15.632660+0000","last_undegraded":"2024-05-27T09:37:15.632660+0000","last_fullsized":"2024-05-27T09:37:15.632660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:13.955191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":4099,"num_write":31,"num_write_kb":4134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"50'337","reported_seq":2460,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.376487+0000","last_change":"2024-05-27T09:33:11.453500+0000","last_active":"2024-05-27T09:37:37.376487+0000","last_peered":"2024-05-27T09:37:37.376487+0000","last_clean":"2024-05-27T09:37:37.376487+0000","last_became_active":"2024-05-27T09:27:18.543683+0000","last_became_peered":"2024-05-27T09:27:18.543683+0000","last_unstale":"2024-05-27T09:37:37.376487+0000","last_undegraded":"2024-05-27T09:37:37.376487+0000","last_fullsized":"2024-05-27T09:37:37.376487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:29:05.179158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012478668,"stat_sum":{"num_bytes":14963144,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":20,"num_read_kb":4115,"num_write":326,"num_write_kb":22968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"49'93","reported_seq":171,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.952185+0000","last_change":"2024-05-27T09:27:17.559255+0000","last_active":"2024-05-27T09:37:15.952185+0000","last_peered":"2024-05-27T09:37:15.952185+0000","last_clean":"2024-05-27T09:37:15.952185+0000","last_became_active":"2024-05-27T09:27:17.558973+0000","last_became_peered":"2024-05-27T09:27:17.558973+0000","last_unstale":"2024-05-27T09:37:15.952185+0000","last_undegraded":"2024-05-27T09:37:15.952185+0000","last_fullsized":"2024-05-27T09:37:15.952185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":4097,"num_write":96,"num_write_kb":4263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"50'366","reported_seq":2483,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.371889+0000","last_change":"2024-05-27T09:33:11.452657+0000","last_active":"2024-05-27T09:37:37.371889+0000","last_peered":"2024-05-27T09:37:37.371889+0000","last_clean":"2024-05-27T09:37:37.371889+0000","last_became_active":"2024-05-27T09:27:18.557076+0000","last_became_peered":"2024-05-27T09:27:18.557076+0000","last_unstale":"2024-05-27T09:37:37.371889+0000","last_undegraded":"2024-05-27T09:37:37.371889+0000","last_fullsized":"2024-05-27T09:37:37.371889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:26:39.506639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014584746000000001,"stat_sum":{"num_bytes":15385587,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":358,"num_write_kb":15182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"49'49","reported_seq":134,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.953808+0000","last_change":"2024-05-27T09:27:17.573700+0000","last_active":"2024-05-27T09:37:15.953808+0000","last_peered":"2024-05-27T09:37:15.953808+0000","last_clean":"2024-05-27T09:37:15.953808+0000","last_became_active":"2024-05-27T09:27:17.573484+0000","last_became_peered":"2024-05-27T09:27:17.573484+0000","last_unstale":"2024-05-27T09:37:15.953808+0000","last_undegraded":"2024-05-27T09:37:15.953808+0000","last_fullsized":"2024-05-27T09:37:15.953808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:48.905422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":51,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"50'336","reported_seq":2461,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.377629+0000","last_change":"2024-05-27T09:33:11.456688+0000","last_active":"2024-05-27T09:37:37.377629+0000","last_peered":"2024-05-27T09:37:37.377629+0000","last_clean":"2024-05-27T09:37:37.377629+0000","last_became_active":"2024-05-27T09:27:18.574959+0000","last_became_peered":"2024-05-27T09:27:18.574959+0000","last_unstale":"2024-05-27T09:37:37.377629+0000","last_undegraded":"2024-05-27T09:37:37.377629+0000","last_fullsized":"2024-05-27T09:37:37.377629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:53.188459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0093604480000000004,"stat_sum":{"num_bytes":20164393,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":23,"num_read_kb":12305,"num_write":329,"num_write_kb":28020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":173,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.533665+0000","last_change":"2024-05-27T09:27:12.693423+0000","last_active":"2024-05-27T09:37:15.533665+0000","last_peered":"2024-05-27T09:37:15.533665+0000","last_clean":"2024-05-27T09:37:15.533665+0000","last_became_active":"2024-05-27T09:27:12.690301+0000","last_became_peered":"2024-05-27T09:27:12.690301+0000","last_unstale":"2024-05-27T09:37:15.533665+0000","last_undegraded":"2024-05-27T09:37:15.533665+0000","last_fullsized":"2024-05-27T09:37:15.533665+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:11.463226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:11.463226+0000","last_clean_scrub_stamp":"2024-05-27T09:27:11.463226+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:01.810071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'12","reported_seq":93,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.547944+0000","last_change":"2024-05-27T09:27:17.567670+0000","last_active":"2024-05-27T09:37:15.547944+0000","last_peered":"2024-05-27T09:37:15.547944+0000","last_clean":"2024-05-27T09:37:15.547944+0000","last_became_active":"2024-05-27T09:27:17.566292+0000","last_became_peered":"2024-05-27T09:27:17.566292+0000","last_unstale":"2024-05-27T09:37:15.547944+0000","last_undegraded":"2024-05-27T09:37:15.547944+0000","last_fullsized":"2024-05-27T09:37:15.547944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:18.038221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194338,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":14,"num_write_kb":4206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"50'333","reported_seq":2448,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.242277+0000","last_change":"2024-05-27T09:33:11.733467+0000","last_active":"2024-05-27T09:37:20.242277+0000","last_peered":"2024-05-27T09:37:20.242277+0000","last_clean":"2024-05-27T09:37:20.242277+0000","last_became_active":"2024-05-27T09:27:18.570782+0000","last_became_peered":"2024-05-27T09:27:18.570782+0000","last_unstale":"2024-05-27T09:37:20.242277+0000","last_undegraded":"2024-05-27T09:37:20.242277+0000","last_fullsized":"2024-05-27T09:37:20.242277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:22.877083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044685039999999999,"stat_sum":{"num_bytes":10376859,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":333,"num_write_kb":18496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"50'344","reported_seq":428,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:17.550853+0000","last_change":"2024-05-27T09:27:17.563259+0000","last_active":"2024-05-27T09:37:17.550853+0000","last_peered":"2024-05-27T09:37:17.550853+0000","last_clean":"2024-05-27T09:37:17.550853+0000","last_became_active":"2024-05-27T09:27:17.563010+0000","last_became_peered":"2024-05-27T09:27:17.563010+0000","last_unstale":"2024-05-27T09:37:17.550853+0000","last_undegraded":"2024-05-27T09:37:17.550853+0000","last_fullsized":"2024-05-27T09:37:17.550853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:53.711508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":8210,"num_write":339,"num_write_kb":12482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"50'318","reported_seq":2431,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.233244+0000","last_change":"2024-05-27T09:33:11.730552+0000","last_active":"2024-05-27T09:37:20.233244+0000","last_peered":"2024-05-27T09:37:20.233244+0000","last_clean":"2024-05-27T09:37:20.233244+0000","last_became_active":"2024-05-27T09:27:18.574090+0000","last_became_peered":"2024-05-27T09:27:18.574090+0000","last_unstale":"2024-05-27T09:37:20.233244+0000","last_undegraded":"2024-05-27T09:37:20.233244+0000","last_fullsized":"2024-05-27T09:37:20.233244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:29:43.530819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002904064,"stat_sum":{"num_bytes":5516261,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":316,"num_write_kb":17833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"50'122","reported_seq":316,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.860934+0000","last_change":"2024-05-27T09:27:17.551310+0000","last_active":"2024-05-27T09:37:15.860934+0000","last_peered":"2024-05-27T09:37:15.860934+0000","last_clean":"2024-05-27T09:37:15.860934+0000","last_became_active":"2024-05-27T09:27:17.550982+0000","last_became_peered":"2024-05-27T09:27:17.550982+0000","last_unstale":"2024-05-27T09:37:15.860934+0000","last_undegraded":"2024-05-27T09:37:15.860934+0000","last_fullsized":"2024-05-27T09:37:15.860934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":534380,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":130,"num_read_kb":615,"num_write":120,"num_write_kb":620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"50'340","reported_seq":2465,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.169125+0000","last_change":"2024-05-27T09:33:11.732933+0000","last_active":"2024-05-27T09:37:20.169125+0000","last_peered":"2024-05-27T09:37:20.169125+0000","last_clean":"2024-05-27T09:37:20.169125+0000","last_became_active":"2024-05-27T09:27:18.559017+0000","last_became_peered":"2024-05-27T09:27:18.559017+0000","last_unstale":"2024-05-27T09:37:20.169125+0000","last_undegraded":"2024-05-27T09:37:20.169125+0000","last_fullsized":"2024-05-27T09:37:20.169125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:06:28.204044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065168789999999997,"stat_sum":{"num_bytes":26414737,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":21,"num_read_kb":8209,"num_write":340,"num_write_kb":34165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"49'48","reported_seq":125,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.547864+0000","last_change":"2024-05-27T09:27:17.563947+0000","last_active":"2024-05-27T09:37:15.547864+0000","last_peered":"2024-05-27T09:37:15.547864+0000","last_clean":"2024-05-27T09:37:15.547864+0000","last_became_active":"2024-05-27T09:27:17.563640+0000","last_became_peered":"2024-05-27T09:27:17.563640+0000","last_unstale":"2024-05-27T09:37:15.547864+0000","last_undegraded":"2024-05-27T09:37:15.547864+0000","last_fullsized":"2024-05-27T09:37:15.547864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:02.747708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":43,"num_write_kb":8223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"50'359","reported_seq":2479,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.388774+0000","last_change":"2024-05-27T09:33:11.442514+0000","last_active":"2024-05-27T09:37:37.388774+0000","last_peered":"2024-05-27T09:37:37.388774+0000","last_clean":"2024-05-27T09:37:37.388774+0000","last_became_active":"2024-05-27T09:27:18.574710+0000","last_became_peered":"2024-05-27T09:27:18.574710+0000","last_unstale":"2024-05-27T09:37:37.388774+0000","last_undegraded":"2024-05-27T09:37:37.388774+0000","last_fullsized":"2024-05-27T09:37:37.388774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:50:58.577908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075561990000000004,"stat_sum":{"num_bytes":9767075,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":17,"num_read_kb":8206,"num_write":352,"num_write_kb":9721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"49'81","reported_seq":157,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.952072+0000","last_change":"2024-05-27T09:27:17.550749+0000","last_active":"2024-05-27T09:37:15.952072+0000","last_peered":"2024-05-27T09:37:15.952072+0000","last_clean":"2024-05-27T09:37:15.952072+0000","last_became_active":"2024-05-27T09:27:17.550378+0000","last_became_peered":"2024-05-27T09:27:17.550378+0000","last_unstale":"2024-05-27T09:37:15.952072+0000","last_undegraded":"2024-05-27T09:37:15.952072+0000","last_fullsized":"2024-05-27T09:37:15.952072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:20.167629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83,"num_write_kb":4165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"50'360","reported_seq":2483,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.368574+0000","last_change":"2024-05-27T09:33:11.451931+0000","last_active":"2024-05-27T09:37:37.368574+0000","last_peered":"2024-05-27T09:37:37.368574+0000","last_clean":"2024-05-27T09:37:37.368574+0000","last_became_active":"2024-05-27T09:27:18.566326+0000","last_became_peered":"2024-05-27T09:27:18.566326+0000","last_unstale":"2024-05-27T09:37:37.368574+0000","last_undegraded":"2024-05-27T09:37:37.368574+0000","last_fullsized":"2024-05-27T09:37:37.368574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:50.848181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075094710000000002,"stat_sum":{"num_bytes":14788118,"num_objects":187,"num_object_clones":0,"num_object_copies":374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":187,"num_whiteouts":0,"num_read":20,"num_read_kb":8209,"num_write":356,"num_write_kb":35101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'352","reported_seq":430,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.548573+0000","last_change":"2024-05-27T09:27:17.557912+0000","last_active":"2024-05-27T09:37:15.548573+0000","last_peered":"2024-05-27T09:37:15.548573+0000","last_clean":"2024-05-27T09:37:15.548573+0000","last_became_active":"2024-05-27T09:27:17.557216+0000","last_became_peered":"2024-05-27T09:27:17.557216+0000","last_unstale":"2024-05-27T09:37:15.548573+0000","last_undegraded":"2024-05-27T09:37:15.548573+0000","last_fullsized":"2024-05-27T09:37:15.548573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:21:22.020724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":4098,"num_write":353,"num_write_kb":8433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"50'338","reported_seq":2454,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.161215+0000","last_change":"2024-05-27T09:33:11.728508+0000","last_active":"2024-05-27T09:37:20.161215+0000","last_peered":"2024-05-27T09:37:20.161215+0000","last_clean":"2024-05-27T09:37:20.161215+0000","last_became_active":"2024-05-27T09:27:18.562565+0000","last_became_peered":"2024-05-27T09:27:18.562565+0000","last_unstale":"2024-05-27T09:37:20.161215+0000","last_undegraded":"2024-05-27T09:37:20.161215+0000","last_fullsized":"2024-05-27T09:37:20.161215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:45:47.000935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030336680000000002,"stat_sum":{"num_bytes":10461277,"num_objects":176,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":176,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":338,"num_write_kb":10392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"50'197","reported_seq":279,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:28.642623+0000","last_change":"2024-05-27T09:27:17.571911+0000","last_active":"2024-05-27T09:37:28.642623+0000","last_peered":"2024-05-27T09:37:28.642623+0000","last_clean":"2024-05-27T09:37:28.642623+0000","last_became_active":"2024-05-27T09:27:17.571551+0000","last_became_peered":"2024-05-27T09:27:17.571551+0000","last_unstale":"2024-05-27T09:37:28.642623+0000","last_undegraded":"2024-05-27T09:37:28.642623+0000","last_fullsized":"2024-05-27T09:37:28.642623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:20.167629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":453,"num_write_kb":2056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"50'324","reported_seq":2442,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.167207+0000","last_change":"2024-05-27T09:33:11.734048+0000","last_active":"2024-05-27T09:37:20.167207+0000","last_peered":"2024-05-27T09:37:20.167207+0000","last_clean":"2024-05-27T09:37:20.167207+0000","last_became_active":"2024-05-27T09:27:18.558463+0000","last_became_peered":"2024-05-27T09:27:18.558463+0000","last_unstale":"2024-05-27T09:37:20.167207+0000","last_undegraded":"2024-05-27T09:37:20.167207+0000","last_fullsized":"2024-05-27T09:37:20.167207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:29:43.530819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041648589999999999,"stat_sum":{"num_bytes":14892598,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":326,"num_write_kb":22861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"49'171","reported_seq":249,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.549131+0000","last_change":"2024-05-27T09:27:17.567973+0000","last_active":"2024-05-27T09:37:15.549131+0000","last_peered":"2024-05-27T09:37:15.549131+0000","last_clean":"2024-05-27T09:37:15.549131+0000","last_became_active":"2024-05-27T09:27:17.567650+0000","last_became_peered":"2024-05-27T09:27:17.567650+0000","last_unstale":"2024-05-27T09:37:15.549131+0000","last_undegraded":"2024-05-27T09:37:15.549131+0000","last_fullsized":"2024-05-27T09:37:15.549131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:25:52.924185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":176,"num_write_kb":4290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"50'340","reported_seq":2469,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.169202+0000","last_change":"2024-05-27T09:33:11.733662+0000","last_active":"2024-05-27T09:37:20.169202+0000","last_peered":"2024-05-27T09:37:20.169202+0000","last_clean":"2024-05-27T09:37:20.169202+0000","last_became_active":"2024-05-27T09:27:18.554462+0000","last_became_peered":"2024-05-27T09:27:18.554462+0000","last_unstale":"2024-05-27T09:37:20.169202+0000","last_undegraded":"2024-05-27T09:37:20.169202+0000","last_fullsized":"2024-05-27T09:37:20.169202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:21:17.734561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042444429999999997,"stat_sum":{"num_bytes":6471160,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":27,"num_read_kb":4122,"num_write":339,"num_write_kb":6465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"49'126","reported_seq":205,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.539172+0000","last_change":"2024-05-27T09:27:17.570300+0000","last_active":"2024-05-27T09:37:15.539172+0000","last_peered":"2024-05-27T09:37:15.539172+0000","last_clean":"2024-05-27T09:37:15.539172+0000","last_became_active":"2024-05-27T09:27:17.569685+0000","last_became_peered":"2024-05-27T09:27:17.569685+0000","last_unstale":"2024-05-27T09:37:15.539172+0000","last_undegraded":"2024-05-27T09:37:15.539172+0000","last_fullsized":"2024-05-27T09:37:15.539172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:26.210041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":4102,"num_write":127,"num_write_kb":4196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"50'355","reported_seq":2484,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.352377+0000","last_change":"2024-05-27T09:33:11.450475+0000","last_active":"2024-05-27T09:37:37.352377+0000","last_peered":"2024-05-27T09:37:37.352377+0000","last_clean":"2024-05-27T09:37:37.352377+0000","last_became_active":"2024-05-27T09:27:18.562455+0000","last_became_peered":"2024-05-27T09:27:18.562455+0000","last_unstale":"2024-05-27T09:37:37.352377+0000","last_undegraded":"2024-05-27T09:37:37.352377+0000","last_fullsized":"2024-05-27T09:37:37.352377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:31.423566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082016410000000008,"stat_sum":{"num_bytes":31948882,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":27,"num_read_kb":17799,"num_write":353,"num_write_kb":35479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"49'171","reported_seq":250,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.632383+0000","last_change":"2024-05-27T09:27:17.563172+0000","last_active":"2024-05-27T09:37:15.632383+0000","last_peered":"2024-05-27T09:37:15.632383+0000","last_clean":"2024-05-27T09:37:15.632383+0000","last_became_active":"2024-05-27T09:27:17.562941+0000","last_became_peered":"2024-05-27T09:27:17.562941+0000","last_unstale":"2024-05-27T09:37:15.632383+0000","last_undegraded":"2024-05-27T09:37:15.632383+0000","last_fullsized":"2024-05-27T09:37:15.632383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:51.742078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":171,"num_write_kb":4214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"50'340","reported_seq":2461,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.117188+0000","last_change":"2024-05-27T09:33:11.731891+0000","last_active":"2024-05-27T09:37:20.117188+0000","last_peered":"2024-05-27T09:37:20.117188+0000","last_clean":"2024-05-27T09:37:20.117188+0000","last_became_active":"2024-05-27T09:27:18.555280+0000","last_became_peered":"2024-05-27T09:27:18.555280+0000","last_unstale":"2024-05-27T09:37:20.117188+0000","last_undegraded":"2024-05-27T09:37:20.117188+0000","last_fullsized":"2024-05-27T09:37:20.117188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:31.147919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041574209999999997,"stat_sum":{"num_bytes":13440874,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":19,"num_read_kb":8208,"num_write":339,"num_write_kb":17394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"50'252","reported_seq":331,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:36.353976+0000","last_change":"2024-05-27T09:27:17.548266+0000","last_active":"2024-05-27T09:37:36.353976+0000","last_peered":"2024-05-27T09:37:36.353976+0000","last_clean":"2024-05-27T09:37:36.353976+0000","last_became_active":"2024-05-27T09:27:17.547938+0000","last_became_peered":"2024-05-27T09:27:17.547938+0000","last_unstale":"2024-05-27T09:37:36.353976+0000","last_undegraded":"2024-05-27T09:37:36.353976+0000","last_fullsized":"2024-05-27T09:37:36.353976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7112766,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":4100,"num_write":280,"num_write_kb":7097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"50'334","reported_seq":2452,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.163309+0000","last_change":"2024-05-27T09:33:11.625744+0000","last_active":"2024-05-27T09:37:20.163309+0000","last_peered":"2024-05-27T09:37:20.163309+0000","last_clean":"2024-05-27T09:37:20.163309+0000","last_became_active":"2024-05-27T09:27:18.563227+0000","last_became_peered":"2024-05-27T09:27:18.563227+0000","last_unstale":"2024-05-27T09:37:20.163309+0000","last_undegraded":"2024-05-27T09:37:20.163309+0000","last_fullsized":"2024-05-27T09:37:20.163309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:30:49.326058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010490250000000001,"stat_sum":{"num_bytes":10717337,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":15,"num_read_kb":2569,"num_write":334,"num_write_kb":14740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"50'342","reported_seq":422,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.542171+0000","last_change":"2024-05-27T09:27:17.562200+0000","last_active":"2024-05-27T09:37:20.542171+0000","last_peered":"2024-05-27T09:37:20.542171+0000","last_clean":"2024-05-27T09:37:20.542171+0000","last_became_active":"2024-05-27T09:27:17.561857+0000","last_became_peered":"2024-05-27T09:27:17.561857+0000","last_unstale":"2024-05-27T09:37:20.542171+0000","last_undegraded":"2024-05-27T09:37:20.542171+0000","last_fullsized":"2024-05-27T09:37:20.542171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:05.209528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":10,"num_read_kb":8200,"num_write":340,"num_write_kb":12481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"50'360","reported_seq":2475,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.372550+0000","last_change":"2024-05-27T09:33:11.455285+0000","last_active":"2024-05-27T09:37:37.372550+0000","last_peered":"2024-05-27T09:37:37.372550+0000","last_clean":"2024-05-27T09:37:37.372550+0000","last_became_active":"2024-05-27T09:27:18.572339+0000","last_became_peered":"2024-05-27T09:27:18.572339+0000","last_unstale":"2024-05-27T09:37:37.372550+0000","last_undegraded":"2024-05-27T09:37:37.372550+0000","last_fullsized":"2024-05-27T09:37:37.372550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:11:51.773780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011410574,"stat_sum":{"num_bytes":1288358,"num_objects":185,"num_object_clones":0,"num_object_copies":370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":185,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":355,"num_write_kb":9639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"50'330","reported_seq":2455,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.388768+0000","last_change":"2024-05-27T09:33:11.456154+0000","last_active":"2024-05-27T09:37:37.388768+0000","last_peered":"2024-05-27T09:37:37.388768+0000","last_clean":"2024-05-27T09:37:37.388768+0000","last_became_active":"2024-05-27T09:27:18.572307+0000","last_became_peered":"2024-05-27T09:27:18.572307+0000","last_unstale":"2024-05-27T09:37:37.388768+0000","last_undegraded":"2024-05-27T09:37:37.388768+0000","last_fullsized":"2024-05-27T09:37:37.388768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:38:38.928939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089418529999999996,"stat_sum":{"num_bytes":7551421,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":315,"num_write_kb":11635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"49'32","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.542484+0000","last_change":"2024-05-27T09:27:17.570634+0000","last_active":"2024-05-27T09:37:15.542484+0000","last_peered":"2024-05-27T09:37:15.542484+0000","last_clean":"2024-05-27T09:37:15.542484+0000","last_became_active":"2024-05-27T09:27:17.569866+0000","last_became_peered":"2024-05-27T09:27:17.569866+0000","last_unstale":"2024-05-27T09:37:15.542484+0000","last_undegraded":"2024-05-27T09:37:15.542484+0000","last_fullsized":"2024-05-27T09:37:15.542484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:21:38.268535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4099,"num_write":35,"num_write_kb":4131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"50'359","reported_seq":2476,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.380935+0000","last_change":"2024-05-27T09:33:11.737809+0000","last_active":"2024-05-27T09:37:37.380935+0000","last_peered":"2024-05-27T09:37:37.380935+0000","last_clean":"2024-05-27T09:37:37.380935+0000","last_became_active":"2024-05-27T09:27:18.549035+0000","last_became_peered":"2024-05-27T09:27:18.549035+0000","last_unstale":"2024-05-27T09:37:37.380935+0000","last_undegraded":"2024-05-27T09:37:37.380935+0000","last_fullsized":"2024-05-27T09:37:37.380935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:14:41.023261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0090041500000000007,"stat_sum":{"num_bytes":20118472,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":14,"num_read_kb":13,"num_write":350,"num_write_kb":27987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"49'143","reported_seq":226,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.543458+0000","last_change":"2024-05-27T09:27:17.568143+0000","last_active":"2024-05-27T09:37:15.543458+0000","last_peered":"2024-05-27T09:37:15.543458+0000","last_clean":"2024-05-27T09:37:15.543458+0000","last_became_active":"2024-05-27T09:27:17.567235+0000","last_became_peered":"2024-05-27T09:27:17.567235+0000","last_unstale":"2024-05-27T09:37:15.543458+0000","last_undegraded":"2024-05-27T09:37:15.543458+0000","last_fullsized":"2024-05-27T09:37:15.543458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.284270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":15,"num_read_kb":12312,"num_write":136,"num_write_kb":12381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"50'313","reported_seq":2437,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.234565+0000","last_change":"2024-05-27T09:33:11.732530+0000","last_active":"2024-05-27T09:37:20.234565+0000","last_peered":"2024-05-27T09:37:20.234565+0000","last_clean":"2024-05-27T09:37:20.234565+0000","last_became_active":"2024-05-27T09:27:18.562919+0000","last_became_peered":"2024-05-27T09:27:18.562919+0000","last_unstale":"2024-05-27T09:37:20.234565+0000","last_undegraded":"2024-05-27T09:37:20.234565+0000","last_fullsized":"2024-05-27T09:37:20.234565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:09:22.339045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0063913879999999996,"stat_sum":{"num_bytes":22598856,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":21,"num_read_kb":8211,"num_write":313,"num_write_kb":22222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"49'87","reported_seq":165,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.953251+0000","last_change":"2024-05-27T09:27:17.551193+0000","last_active":"2024-05-27T09:37:15.953251+0000","last_peered":"2024-05-27T09:37:15.953251+0000","last_clean":"2024-05-27T09:37:15.953251+0000","last_became_active":"2024-05-27T09:27:17.550849+0000","last_became_peered":"2024-05-27T09:27:17.550849+0000","last_unstale":"2024-05-27T09:37:15.953251+0000","last_undegraded":"2024-05-27T09:37:15.953251+0000","last_fullsized":"2024-05-27T09:37:15.953251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:51.742078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":81,"num_write_kb":4153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"50'330","reported_seq":2448,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.256184+0000","last_change":"2024-05-27T09:33:11.732054+0000","last_active":"2024-05-27T09:37:20.256184+0000","last_peered":"2024-05-27T09:37:20.256184+0000","last_clean":"2024-05-27T09:37:20.256184+0000","last_became_active":"2024-05-27T09:27:18.579574+0000","last_became_peered":"2024-05-27T09:27:18.579574+0000","last_unstale":"2024-05-27T09:37:20.256184+0000","last_undegraded":"2024-05-27T09:37:20.256184+0000","last_fullsized":"2024-05-27T09:37:20.256184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:56:05.185246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062209780000000003,"stat_sum":{"num_bytes":14492175,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":14,"num_read_kb":4108,"num_write":330,"num_write_kb":14321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"49'82","reported_seq":172,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.632721+0000","last_change":"2024-05-27T09:27:17.549711+0000","last_active":"2024-05-27T09:37:15.632721+0000","last_peered":"2024-05-27T09:37:15.632721+0000","last_clean":"2024-05-27T09:37:15.632721+0000","last_became_active":"2024-05-27T09:27:17.549291+0000","last_became_peered":"2024-05-27T09:27:17.549291+0000","last_unstale":"2024-05-27T09:37:15.632721+0000","last_undegraded":"2024-05-27T09:37:15.632721+0000","last_fullsized":"2024-05-27T09:37:15.632721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:16:22.057803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":41,"num_read_kb":4136,"num_write":85,"num_write_kb":8333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":952881208,"num_objects":10697,"num_object_clones":0,"num_object_copies":21394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10697,"num_whiteouts":0,"num_read":1250,"num_read_kb":378565,"num_write":21433,"num_write_kb":1379166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1734066176,"data_stored":1904222320,"data_compressed":168711384,"data_compressed_allocated":186425344,"data_compressed_original":412123136,"omap_allocated":0,"internal_metadata":0},"log_size":21735,"ondisk_log_size":21735,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":188007639,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":494,"num_read_kb":123738,"num_write":4412,"num_write_kb":189065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":105,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":83959808,"data_stored":376015278,"data_compressed":49745852,"data_compressed_allocated":62447616,"data_compressed_original":354656256,"omap_allocated":0,"internal_metadata":0},"log_size":4113,"ondisk_log_size":4113,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65078,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509569,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":363184,"kb_used_data":309800,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94008656,"statfs":{"total":96636764160,"available":96264863744,"internally_reserved":0,"allocated":317235200,"data_stored":412121967,"data_compressed":41350796,"data_compressed_allocated":47357952,"data_compressed_original":150941696,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509569,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":278116,"kb_used_data":233052,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94093724,"statfs":{"total":96636764160,"available":96351973376,"internally_reserved":0,"allocated":238645248,"data_stored":280794177,"data_compressed":23230909,"data_compressed_allocated":26537984,"data_compressed_original":76705792,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509569,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246488,"kb_used_data":201424,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94125352,"statfs":{"total":96636764160,"available":96384360448,"internally_reserved":0,"allocated":206258176,"data_stored":257021246,"data_compressed":23229218,"data_compressed_allocated":26480640,"data_compressed_original":83582976,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509569,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356864,"kb_used_data":285432,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94014976,"statfs":{"total":96636764160,"available":96271335424,"internally_reserved":0,"allocated":292282368,"data_stored":379742031,"data_compressed":39531620,"data_compressed_allocated":45047808,"data_compressed_original":141885440,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542272,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":157168,"kb_used_data":130152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94214672,"statfs":{"total":96636764160,"available":96475824128,"internally_reserved":0,"allocated":133275648,"data_stored":168154774,"data_compressed":18202841,"data_compressed_allocated":20492288,"data_compressed_original":60579840,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542272,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":196044,"kb_used_data":146820,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94175796,"statfs":{"total":96636764160,"available":96436015104,"internally_reserved":0,"allocated":150343680,"data_stored":185739612,"data_compressed":18837119,"data_compressed_allocated":21549056,"data_compressed_original":63336448,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542272,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":358772,"kb_used_data":305388,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94013068,"statfs":{"total":96636764160,"available":96269381632,"internally_reserved":0,"allocated":312717312,"data_stored":372347367,"data_compressed":31432531,"data_compressed_allocated":35454976,"data_compressed_original":103374848,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542272,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":221244,"kb_used_data":167860,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94150596,"statfs":{"total":96636764160,"available":96410210304,"internally_reserved":0,"allocated":171888640,"data_stored":227475258,"data_compressed":22698376,"data_compressed_allocated":26025984,"data_compressed_original":87445504,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13996032,"data_stored":67110923,"data_compressed":8258411,"data_compressed_allocated":10555392,"data_compressed_original":63688704,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7446528,"data_stored":37748896,"data_compressed":4390710,"data_compressed_allocated":5623808,"data_compressed_original":35934208,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7921664,"data_stored":33555138,"data_compressed":4517094,"data_compressed_allocated":5791744,"data_compressed_original":31436800,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19890176,"data_stored":83888049,"data_compressed":11332765,"data_compressed_allocated":14295040,"data_compressed_original":78307328,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12206080,"data_stored":50333888,"data_compressed":7575831,"data_compressed_allocated":9129984,"data_compressed_original":47296512,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9977856,"data_stored":45397082,"data_compressed":5653863,"data_compressed_allocated":7127040,"data_compressed_original":42569728,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7090176,"data_stored":32280280,"data_compressed":4471564,"data_compressed_allocated":5599232,"data_compressed_original":30801920,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5431296,"data_stored":25701022,"data_compressed":3545614,"data_compressed_allocated":4325376,"data_compressed_original":24621056,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":277721088,"data_stored":312372852,"data_compressed":31273209,"data_compressed_allocated":34492416,"data_compressed_original":78196736,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":198180864,"data_stored":218423585,"data_compressed":18805969,"data_compressed_allocated":20811776,"data_compressed_original":47075328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":230158336,"data_stored":246980642,"data_compressed":18713815,"data_compressed_allocated":20746240,"data_compressed_original":45268992,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":296779776,"data_stored":327975662,"data_compressed":30018031,"data_compressed_allocated":33062912,"data_compressed_original":72634368,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":159117312,"data_stored":176882973,"data_compressed":15122545,"data_compressed_allocated":16896000,"data_compressed_original":40148992,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":302125056,"data_stored":326145918,"data_compressed":25750581,"data_compressed_allocated":28291072,"data_compressed_original":60268544,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":142704640,"data_stored":153245333,"data_compressed":14370007,"data_compressed_allocated":15958016,"data_compressed_original":32571392,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":127279104,"data_stored":142195355,"data_compressed":14657227,"data_compressed_allocated":16166912,"data_compressed_original":35958784,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T09:37:42.205 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T09:37:42.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.520+0000 7ff23ee00640 1 Processor -- start 2024-05-27T09:37:42.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.521+0000 7ff23ee00640 1 -- start start 2024-05-27T09:37:42.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.523+0000 7ff23ee00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff238164db0 0x7ff2380c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:42.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.523+0000 7ff23ee00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff238074860 0x7ff23815aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:42.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.523+0000 7ff23ee00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff23815b000 con 0x7ff238160b40 2024-05-27T09:37:42.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.523+0000 7ff23ee00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7ff2380cade0 con 0x7ff238074860 2024-05-27T09:37:42.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.523+0000 7ff23ee00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7ff2380c3d60 con 0x7ff238164db0 2024-05-27T09:37:42.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.523+0000 7ff22f400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff238074860 0x7ff23815aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:42.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.523+0000 7ff237e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff238164db0 0x7ff2380c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:42.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.523+0000 7ff22f400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7ff238074860 0x7ff23815aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39748/0 (socket says 172.21.15.114:39748) 2024-05-27T09:37:42.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.523+0000 7ff22f400640 1 -- 172.21.15.114:0/1069872223 learned_addr learned my addr 172.21.15.114:0/1069872223 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:42.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.524+0000 7ff237400640 1 -- 172.21.15.114:0/1069872223 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4139853384 0 0) 0x7ff23815b000 con 0x7ff238160b40 2024-05-27T09:37:42.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.524+0000 7ff22f400640 1 -- 172.21.15.114:0/1069872223 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff238164db0 msgr2=0x7ff2380c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:42.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.524+0000 7ff22f400640 1 --2- 172.21.15.114:0/1069872223 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff238164db0 0x7ff2380c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:42.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.524+0000 7ff22f400640 1 -- 172.21.15.114:0/1069872223 >> v1:172.21.15.163:6789/0 conn(0x7ff238160b40 legacy=0x7ff238073310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:42.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.524+0000 7ff22f400640 1 -- 172.21.15.114:0/1069872223 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2381615d0 con 0x7ff238074860 2024-05-27T09:37:42.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.525+0000 7ff22f400640 1 --2- 172.21.15.114:0/1069872223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff238074860 0x7ff23815aac0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff220009870 tx=0x7ff22002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf8a3643b7d1eed server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:42.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.525+0000 7ff237400640 1 -- 172.21.15.114:0/1069872223 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220037070 con 0x7ff238074860 2024-05-27T09:37:42.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.525+0000 7ff237400640 1 -- 172.21.15.114:0/1069872223 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff22002ae50 con 0x7ff238074860 2024-05-27T09:37:42.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.525+0000 7ff237400640 1 -- 172.21.15.114:0/1069872223 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220032860 con 0x7ff238074860 2024-05-27T09:37:42.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.526+0000 7ff23ee00640 1 -- 172.21.15.114:0/1069872223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff238074860 msgr2=0x7ff23815aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:42.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.526+0000 7ff23ee00640 1 --2- 172.21.15.114:0/1069872223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff238074860 0x7ff23815aac0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff220009870 tx=0x7ff22002a300 comp rx=0 tx=0).stop 2024-05-27T09:37:42.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.526+0000 7ff23ee00640 1 -- 172.21.15.114:0/1069872223 shutdown_connections 2024-05-27T09:37:42.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.526+0000 7ff23ee00640 1 --2- 172.21.15.114:0/1069872223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff238074860 0x7ff23815aac0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:42.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.526+0000 7ff23ee00640 1 --2- 172.21.15.114:0/1069872223 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff238164db0 0x7ff2380c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:42.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.526+0000 7ff23ee00640 1 -- 172.21.15.114:0/1069872223 >> 172.21.15.114:0/1069872223 conn(0x7ff23806f6e0 msgr2=0x7ff238070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:42.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.526+0000 7ff23ee00640 1 -- 172.21.15.114:0/1069872223 shutdown_connections 2024-05-27T09:37:42.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.526+0000 7ff23ee00640 1 -- 172.21.15.114:0/1069872223 wait complete. 2024-05-27T09:37:42.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.528+0000 7ff23ee00640 1 Processor -- start 2024-05-27T09:37:42.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.529+0000 7ff23ee00640 1 -- start start 2024-05-27T09:37:42.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.530+0000 7ff23ee00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff238074860 0x7ff2381f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:42.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.530+0000 7ff23ee00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff238160b40 0x7ff238157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:42.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.530+0000 7ff23ee00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff238164db0 0x7ff2380cb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:42.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.530+0000 7ff23ee00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff2381645b0 con 0x7ff238074860 2024-05-27T09:37:42.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.530+0000 7ff23ee00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff2381642d0 con 0x7ff238164db0 2024-05-27T09:37:42.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.530+0000 7ff23ee00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7ff238164890 con 0x7ff238160b40 2024-05-27T09:37:42.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.530+0000 7ff237e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff238074860 0x7ff2381f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:42.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.530+0000 7ff22f400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff238160b40 0x7ff238157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:42.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.530+0000 7ff237e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff238074860 0x7ff2381f6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39758/0 (socket says 172.21.15.114:39758) 2024-05-27T09:37:42.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.530+0000 7ff237e00640 1 -- 172.21.15.114:0/2028432000 learned_addr learned my addr 172.21.15.114:0/2028432000 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:42.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.530+0000 7ff23ce00640 1 --2- 172.21.15.114:0/2028432000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff238164db0 0x7ff2380cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:42.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.531+0000 7ff22f400640 1 -- 172.21.15.114:0/2028432000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff238164db0 msgr2=0x7ff2380cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:42.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.531+0000 7ff22f400640 1 --2- 172.21.15.114:0/2028432000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff238164db0 0x7ff2380cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:42.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.531+0000 7ff22f400640 1 -- 172.21.15.114:0/2028432000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff238074860 msgr2=0x7ff2381f6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:42.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.531+0000 7ff22f400640 1 --2- 172.21.15.114:0/2028432000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff238074860 0x7ff2381f6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:42.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.531+0000 7ff22f400640 1 -- 172.21.15.114:0/2028432000 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2381f6860 con 0x7ff238160b40 2024-05-27T09:37:42.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.531+0000 7ff237e00640 1 --2- 172.21.15.114:0/2028432000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff238074860 0x7ff2381f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:42.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.531+0000 7ff22f400640 1 --2- 172.21.15.114:0/2028432000 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff238160b40 0x7ff238157f20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff23806d830 tx=0x7ff2200046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:42.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.532+0000 7ff234c00640 1 -- 172.21.15.114:0/2028432000 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220037040 con 0x7ff238160b40 2024-05-27T09:37:42.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.532+0000 7ff234c00640 1 -- 172.21.15.114:0/2028432000 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff220004230 con 0x7ff238160b40 2024-05-27T09:37:42.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.532+0000 7ff234c00640 1 -- 172.21.15.114:0/2028432000 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220003610 con 0x7ff238160b40 2024-05-27T09:37:42.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.532+0000 7ff23ee00640 1 -- 172.21.15.114:0/2028432000 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2381f7260 con 0x7ff238160b40 2024-05-27T09:37:42.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.532+0000 7ff23ee00640 1 -- 172.21.15.114:0/2028432000 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2380cdb50 con 0x7ff238160b40 2024-05-27T09:37:42.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.533+0000 7ff23ee00640 1 -- 172.21.15.114:0/2028432000 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff23806a380 con 0x7ff238160b40 2024-05-27T09:37:42.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.534+0000 7ff234c00640 1 -- 172.21.15.114:0/2028432000 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff22003a070 con 0x7ff238160b40 2024-05-27T09:37:42.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.535+0000 7ff234c00640 1 --2- 172.21.15.114:0/2028432000 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff214076300 0x7ff214078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:42.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.535+0000 7ff234c00640 1 -- 172.21.15.114:0/2028432000 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7ff2200b7ce0 con 0x7ff238160b40 2024-05-27T09:37:42.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.535+0000 7ff237e00640 1 --2- 172.21.15.114:0/2028432000 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff214076300 0x7ff214078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:42.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.536+0000 7ff237e00640 1 --2- 172.21.15.114:0/2028432000 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff214076300 0x7ff214078810 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff2280094c0 tx=0x7ff22800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:42.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.540+0000 7ff234c00640 1 -- 172.21.15.114:0/2028432000 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff2381f6860 con 0x7ff238160b40 2024-05-27T09:37:42.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.752+0000 7ff23ee00640 1 -- 172.21.15.114:0/2028432000 --> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff238073e30 con 0x7ff214076300 2024-05-27T09:37:42.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.760+0000 7ff234c00640 1 -- 172.21.15.114:0/2028432000 <== mgr.4108 v2:172.21.15.163:6832/1122018204 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+256663 (secure 0 0 0) 0x7ff238073e30 con 0x7ff214076300 2024-05-27T09:37:42.765 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:37:42.766 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-05-27T09:37:42.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.766+0000 7ff23ee00640 1 -- 172.21.15.114:0/2028432000 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff214076300 msgr2=0x7ff214078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:42.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.766+0000 7ff23ee00640 1 --2- 172.21.15.114:0/2028432000 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff214076300 0x7ff214078810 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff2280094c0 tx=0x7ff22800b040 comp rx=0 tx=0).stop 2024-05-27T09:37:42.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.766+0000 7ff23ee00640 1 -- 172.21.15.114:0/2028432000 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff238160b40 msgr2=0x7ff238157f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:42.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.766+0000 7ff23ee00640 1 --2- 172.21.15.114:0/2028432000 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff238160b40 0x7ff238157f20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff23806d830 tx=0x7ff2200046c0 comp rx=0 tx=0).stop 2024-05-27T09:37:42.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.767+0000 7ff23ee00640 1 -- 172.21.15.114:0/2028432000 shutdown_connections 2024-05-27T09:37:42.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.767+0000 7ff23ee00640 1 --2- 172.21.15.114:0/2028432000 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff214076300 0x7ff214078810 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:42.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.767+0000 7ff23ee00640 1 --2- 172.21.15.114:0/2028432000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff238164db0 0x7ff2380cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:42.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.767+0000 7ff23ee00640 1 --2- 172.21.15.114:0/2028432000 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff238160b40 0x7ff238157f20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:42.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.767+0000 7ff23ee00640 1 --2- 172.21.15.114:0/2028432000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff238074860 0x7ff2381f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:42.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.767+0000 7ff23ee00640 1 -- 172.21.15.114:0/2028432000 >> 172.21.15.114:0/2028432000 conn(0x7ff23806f6e0 msgr2=0x7ff2380760b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:42.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.767+0000 7ff23ee00640 1 -- 172.21.15.114:0/2028432000 shutdown_connections 2024-05-27T09:37:42.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:42.767+0000 7ff23ee00640 1 -- 172.21.15.114:0/2028432000 wait complete. 2024-05-27T09:37:42.793 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-05-27T09:37:40.940503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1141479215,"num_objects":10877,"num_object_clones":0,"num_object_copies":21754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10877,"num_whiteouts":0,"num_read":1810,"num_read_kb":502358,"num_write":25946,"num_write_kb":1569859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":105,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25920,"ondisk_log_size":25920,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2177880,"kb_used_data":1779928,"kb_used_omap":12,"kb_used_meta":397875,"kb_avail":752796840,"statfs":{"total":773094113280,"available":770863964160,"internally_reserved":0,"allocated":1822646272,"data_stored":2283396432,"data_compressed":218513410,"data_compressed_allocated":248946688,"data_compressed_original":767852544,"omap_allocated":12750,"internal_metadata":407424562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1943541,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":166,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003502"},"pg_stats":[{"pgid":"3.3f","version":"50'328","reported_seq":2448,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.380193+0000","last_change":"2024-05-27T09:33:11.447418+0000","last_active":"2024-05-27T09:37:37.380193+0000","last_peered":"2024-05-27T09:37:37.380193+0000","last_clean":"2024-05-27T09:37:37.380193+0000","last_became_active":"2024-05-27T09:27:18.585169+0000","last_became_peered":"2024-05-27T09:27:18.585169+0000","last_unstale":"2024-05-27T09:37:37.380193+0000","last_undegraded":"2024-05-27T09:37:37.380193+0000","last_fullsized":"2024-05-27T09:37:37.380193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:30:18.926000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01102325,"stat_sum":{"num_bytes":23698106,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":17,"num_read_kb":4111,"num_write":320,"num_write_kb":27403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"50'313","reported_seq":2432,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.133411+0000","last_change":"2024-05-27T09:33:11.731804+0000","last_active":"2024-05-27T09:37:20.133411+0000","last_peered":"2024-05-27T09:37:20.133411+0000","last_clean":"2024-05-27T09:37:20.133411+0000","last_became_active":"2024-05-27T09:27:18.566269+0000","last_became_peered":"2024-05-27T09:27:18.566269+0000","last_unstale":"2024-05-27T09:37:20.133411+0000","last_undegraded":"2024-05-27T09:37:20.133411+0000","last_fullsized":"2024-05-27T09:37:20.133411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:30:51.472078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061074720000000001,"stat_sum":{"num_bytes":5460707,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":313,"num_write_kb":13693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"50'342","reported_seq":2469,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.128558+0000","last_change":"2024-05-27T09:33:11.731326+0000","last_active":"2024-05-27T09:37:20.128558+0000","last_peered":"2024-05-27T09:37:20.128558+0000","last_clean":"2024-05-27T09:37:20.128558+0000","last_became_active":"2024-05-27T09:27:18.560620+0000","last_became_peered":"2024-05-27T09:27:18.560620+0000","last_unstale":"2024-05-27T09:37:20.128558+0000","last_undegraded":"2024-05-27T09:37:20.128558+0000","last_fullsized":"2024-05-27T09:37:20.128558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:02:04.553273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005599354,"stat_sum":{"num_bytes":19266359,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":24,"num_read_kb":17715,"num_write":342,"num_write_kb":39437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"50'307","reported_seq":2421,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.378615+0000","last_change":"2024-05-27T09:33:11.450825+0000","last_active":"2024-05-27T09:37:37.378615+0000","last_peered":"2024-05-27T09:37:37.378615+0000","last_clean":"2024-05-27T09:37:37.378615+0000","last_became_active":"2024-05-27T09:27:18.569481+0000","last_became_peered":"2024-05-27T09:27:18.569481+0000","last_unstale":"2024-05-27T09:37:37.378615+0000","last_undegraded":"2024-05-27T09:37:37.378615+0000","last_fullsized":"2024-05-27T09:37:37.378615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:19:26.159843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007726882,"stat_sum":{"num_bytes":6546535,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":298,"num_write_kb":6559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"50'375","reported_seq":2496,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.373663+0000","last_change":"2024-05-27T09:33:11.440268+0000","last_active":"2024-05-27T09:37:37.373663+0000","last_peered":"2024-05-27T09:37:37.373663+0000","last_clean":"2024-05-27T09:37:37.373663+0000","last_became_active":"2024-05-27T09:27:18.567795+0000","last_became_peered":"2024-05-27T09:27:18.567795+0000","last_unstale":"2024-05-27T09:37:37.373663+0000","last_undegraded":"2024-05-27T09:37:37.373663+0000","last_fullsized":"2024-05-27T09:37:37.373663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:31.321140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049759740000000002,"stat_sum":{"num_bytes":10629514,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":371,"num_write_kb":12093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"50'350","reported_seq":2475,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.371402+0000","last_change":"2024-05-27T09:33:11.445483+0000","last_active":"2024-05-27T09:37:37.371402+0000","last_peered":"2024-05-27T09:37:37.371402+0000","last_clean":"2024-05-27T09:37:37.371402+0000","last_became_active":"2024-05-27T09:27:18.573576+0000","last_became_peered":"2024-05-27T09:27:18.573576+0000","last_unstale":"2024-05-27T09:37:37.371402+0000","last_undegraded":"2024-05-27T09:37:37.371402+0000","last_fullsized":"2024-05-27T09:37:37.371402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:45:04.738799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074740400000000004,"stat_sum":{"num_bytes":30579450,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":22,"num_read_kb":8211,"num_write":344,"num_write_kb":38238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"50'360","reported_seq":2481,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.370720+0000","last_change":"2024-05-27T09:33:11.448897+0000","last_active":"2024-05-27T09:37:37.370720+0000","last_peered":"2024-05-27T09:37:37.370720+0000","last_clean":"2024-05-27T09:37:37.370720+0000","last_became_active":"2024-05-27T09:27:18.542593+0000","last_became_peered":"2024-05-27T09:27:18.542593+0000","last_unstale":"2024-05-27T09:37:37.370720+0000","last_undegraded":"2024-05-27T09:37:37.370720+0000","last_fullsized":"2024-05-27T09:37:37.370720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:11:49.487143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014070156,"stat_sum":{"num_bytes":5299988,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":18,"num_read_kb":4113,"num_write":355,"num_write_kb":9461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"50'320","reported_seq":2436,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.234926+0000","last_change":"2024-05-27T09:33:11.732725+0000","last_active":"2024-05-27T09:37:20.234926+0000","last_peered":"2024-05-27T09:37:20.234926+0000","last_clean":"2024-05-27T09:37:20.234926+0000","last_became_active":"2024-05-27T09:27:18.561788+0000","last_became_peered":"2024-05-27T09:27:18.561788+0000","last_unstale":"2024-05-27T09:37:20.234926+0000","last_undegraded":"2024-05-27T09:37:20.234926+0000","last_fullsized":"2024-05-27T09:37:20.234926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:51:29.653992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065132080000000004,"stat_sum":{"num_bytes":27492830,"num_objects":191,"num_object_clones":0,"num_object_copies":382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":191,"num_whiteouts":0,"num_read":13,"num_read_kb":4107,"num_write":319,"num_write_kb":27008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"50'338","reported_seq":2463,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.369696+0000","last_change":"2024-05-27T09:33:11.739498+0000","last_active":"2024-05-27T09:37:37.369696+0000","last_peered":"2024-05-27T09:37:37.369696+0000","last_clean":"2024-05-27T09:37:37.369696+0000","last_became_active":"2024-05-27T09:27:18.582041+0000","last_became_peered":"2024-05-27T09:27:18.582041+0000","last_unstale":"2024-05-27T09:37:37.369696+0000","last_undegraded":"2024-05-27T09:37:37.369696+0000","last_fullsized":"2024-05-27T09:37:37.369696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:29:10.337097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0083382379999999996,"stat_sum":{"num_bytes":9733898,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":22,"num_read_kb":8210,"num_write":328,"num_write_kb":30110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"50'321","reported_seq":2441,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.392171+0000","last_change":"2024-05-27T09:33:11.446829+0000","last_active":"2024-05-27T09:37:37.392171+0000","last_peered":"2024-05-27T09:37:37.392171+0000","last_clean":"2024-05-27T09:37:37.392171+0000","last_became_active":"2024-05-27T09:27:18.574409+0000","last_became_peered":"2024-05-27T09:27:18.574409+0000","last_unstale":"2024-05-27T09:37:37.392171+0000","last_undegraded":"2024-05-27T09:37:37.392171+0000","last_fullsized":"2024-05-27T09:37:37.392171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:41:37.603718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010493662000000001,"stat_sum":{"num_bytes":13833965,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":17,"num_read_kb":4112,"num_write":306,"num_write_kb":13622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"50'319","reported_seq":2444,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.384747+0000","last_change":"2024-05-27T09:33:11.443445+0000","last_active":"2024-05-27T09:37:37.384747+0000","last_peered":"2024-05-27T09:37:37.384747+0000","last_clean":"2024-05-27T09:37:37.384747+0000","last_became_active":"2024-05-27T09:27:18.578679+0000","last_became_peered":"2024-05-27T09:27:18.578679+0000","last_unstale":"2024-05-27T09:37:37.384747+0000","last_undegraded":"2024-05-27T09:37:37.384747+0000","last_fullsized":"2024-05-27T09:37:37.384747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:33:46.683180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080365679999999991,"stat_sum":{"num_bytes":26002241,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":22,"num_read_kb":7043,"num_write":312,"num_write_kb":25555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"50'294","reported_seq":2402,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.218006+0000","last_change":"2024-05-27T09:33:11.625670+0000","last_active":"2024-05-27T09:37:20.218006+0000","last_peered":"2024-05-27T09:37:20.218006+0000","last_clean":"2024-05-27T09:37:20.218006+0000","last_became_active":"2024-05-27T09:28:06.207562+0000","last_became_peered":"2024-05-27T09:28:06.207562+0000","last_unstale":"2024-05-27T09:37:20.218006+0000","last_undegraded":"2024-05-27T09:37:20.218006+0000","last_fullsized":"2024-05-27T09:37:20.218006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:52:32.773329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001018333,"stat_sum":{"num_bytes":11019660,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":12,"num_read_kb":8201,"num_write":293,"num_write_kb":23198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"50'298","reported_seq":2422,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.265350+0000","last_change":"2024-05-27T09:33:11.728591+0000","last_active":"2024-05-27T09:37:20.265350+0000","last_peered":"2024-05-27T09:37:20.265350+0000","last_clean":"2024-05-27T09:37:20.265350+0000","last_became_active":"2024-05-27T09:27:18.561493+0000","last_became_peered":"2024-05-27T09:27:18.561493+0000","last_unstale":"2024-05-27T09:37:20.265350+0000","last_undegraded":"2024-05-27T09:37:20.265350+0000","last_fullsized":"2024-05-27T09:37:20.265350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:22.877083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024650470000000002,"stat_sum":{"num_bytes":14891482,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":21,"num_read_kb":4116,"num_write":298,"num_write_kb":31046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"50'335","reported_seq":2456,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.131958+0000","last_change":"2024-05-27T09:33:11.727658+0000","last_active":"2024-05-27T09:37:20.131958+0000","last_peered":"2024-05-27T09:37:20.131958+0000","last_clean":"2024-05-27T09:37:20.131958+0000","last_became_active":"2024-05-27T09:27:18.562843+0000","last_became_peered":"2024-05-27T09:27:18.562843+0000","last_unstale":"2024-05-27T09:37:20.131958+0000","last_undegraded":"2024-05-27T09:37:20.131958+0000","last_fullsized":"2024-05-27T09:37:20.131958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:31.147919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025802960000000002,"stat_sum":{"num_bytes":6434369,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":335,"num_write_kb":10518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"50'357","reported_seq":2482,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.244846+0000","last_change":"2024-05-27T09:33:11.729192+0000","last_active":"2024-05-27T09:37:20.244846+0000","last_peered":"2024-05-27T09:37:20.244846+0000","last_clean":"2024-05-27T09:37:20.244846+0000","last_became_active":"2024-05-27T09:27:18.566108+0000","last_became_peered":"2024-05-27T09:27:18.566108+0000","last_unstale":"2024-05-27T09:37:20.244846+0000","last_undegraded":"2024-05-27T09:37:20.244846+0000","last_fullsized":"2024-05-27T09:37:20.244846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:39:23.102944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041189030000000001,"stat_sum":{"num_bytes":20283045,"num_objects":201,"num_object_clones":0,"num_object_copies":402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":201,"num_whiteouts":0,"num_read":22,"num_read_kb":6627,"num_write":357,"num_write_kb":19995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"50'369","reported_seq":2493,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.386640+0000","last_change":"2024-05-27T09:33:11.457079+0000","last_active":"2024-05-27T09:37:37.386640+0000","last_peered":"2024-05-27T09:37:37.386640+0000","last_clean":"2024-05-27T09:37:37.386640+0000","last_became_active":"2024-05-27T09:27:18.581719+0000","last_became_peered":"2024-05-27T09:27:18.581719+0000","last_unstale":"2024-05-27T09:37:37.386640+0000","last_undegraded":"2024-05-27T09:37:37.386640+0000","last_fullsized":"2024-05-27T09:37:37.386640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:59:21.546098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.008579415,"stat_sum":{"num_bytes":24362280,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":22,"num_read_kb":5287,"num_write":359,"num_write_kb":31349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"50'366","reported_seq":2491,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.387571+0000","last_change":"2024-05-27T09:33:11.458057+0000","last_active":"2024-05-27T09:37:37.387571+0000","last_peered":"2024-05-27T09:37:37.387571+0000","last_clean":"2024-05-27T09:37:37.387571+0000","last_became_active":"2024-05-27T09:27:18.577193+0000","last_became_peered":"2024-05-27T09:27:18.577193+0000","last_unstale":"2024-05-27T09:37:37.387571+0000","last_undegraded":"2024-05-27T09:37:37.387571+0000","last_fullsized":"2024-05-27T09:37:37.387571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:41:12.903002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079143630000000006,"stat_sum":{"num_bytes":23132412,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":23,"num_read_kb":8093,"num_write":357,"num_write_kb":30967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"50'297","reported_seq":2428,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.284414+0000","last_change":"2024-05-27T09:33:11.626051+0000","last_active":"2024-05-27T09:37:20.284414+0000","last_peered":"2024-05-27T09:37:20.284414+0000","last_clean":"2024-05-27T09:37:20.284414+0000","last_became_active":"2024-05-27T09:27:18.570712+0000","last_became_peered":"2024-05-27T09:27:18.570712+0000","last_unstale":"2024-05-27T09:37:20.284414+0000","last_undegraded":"2024-05-27T09:37:20.284414+0000","last_fullsized":"2024-05-27T09:37:20.284414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:32:58.266174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767629999999999,"stat_sum":{"num_bytes":13973992,"num_objects":138,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":0,"num_read":28,"num_read_kb":8218,"num_write":296,"num_write_kb":21985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"50'323","reported_seq":2443,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.367564+0000","last_change":"2024-05-27T09:33:11.456909+0000","last_active":"2024-05-27T09:37:37.367564+0000","last_peered":"2024-05-27T09:37:37.367564+0000","last_clean":"2024-05-27T09:37:37.367564+0000","last_became_active":"2024-05-27T09:27:18.564643+0000","last_became_peered":"2024-05-27T09:27:18.564643+0000","last_unstale":"2024-05-27T09:37:37.367564+0000","last_undegraded":"2024-05-27T09:37:37.367564+0000","last_fullsized":"2024-05-27T09:37:37.367564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:14:07.864139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0091687629999999999,"stat_sum":{"num_bytes":2873482,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":18,"num_read_kb":4113,"num_write":315,"num_write_kb":11167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"50'345","reported_seq":2469,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.376407+0000","last_change":"2024-05-27T09:33:11.452205+0000","last_active":"2024-05-27T09:37:37.376407+0000","last_peered":"2024-05-27T09:37:37.376407+0000","last_clean":"2024-05-27T09:37:37.376407+0000","last_became_active":"2024-05-27T09:27:18.543777+0000","last_became_peered":"2024-05-27T09:27:18.543777+0000","last_unstale":"2024-05-27T09:37:37.376407+0000","last_undegraded":"2024-05-27T09:37:37.376407+0000","last_fullsized":"2024-05-27T09:37:37.376407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:11:45.868921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012534072,"stat_sum":{"num_bytes":9859903,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":21,"num_read_kb":4116,"num_write":337,"num_write_kb":17999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"50'326","reported_seq":2451,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.216955+0000","last_change":"2024-05-27T09:33:11.732683+0000","last_active":"2024-05-27T09:37:20.216955+0000","last_peered":"2024-05-27T09:37:20.216955+0000","last_clean":"2024-05-27T09:37:20.216955+0000","last_became_active":"2024-05-27T09:27:18.570948+0000","last_became_peered":"2024-05-27T09:27:18.570948+0000","last_unstale":"2024-05-27T09:37:20.216955+0000","last_undegraded":"2024-05-27T09:37:20.216955+0000","last_fullsized":"2024-05-27T09:37:20.216955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:03:03.895209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040840829999999996,"stat_sum":{"num_bytes":5923849,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":325,"num_write_kb":5958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"50'343","reported_seq":2466,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.383214+0000","last_change":"2024-05-27T09:33:11.737682+0000","last_active":"2024-05-27T09:37:37.383214+0000","last_peered":"2024-05-27T09:37:37.383214+0000","last_clean":"2024-05-27T09:37:37.383214+0000","last_became_active":"2024-05-27T09:27:18.543921+0000","last_became_peered":"2024-05-27T09:27:18.543921+0000","last_unstale":"2024-05-27T09:37:37.383214+0000","last_undegraded":"2024-05-27T09:37:37.383214+0000","last_fullsized":"2024-05-27T09:37:37.383214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:47:03.620703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0094259180000000001,"stat_sum":{"num_bytes":14450069,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":20,"num_read_kb":8209,"num_write":332,"num_write_kb":18385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"50'329","reported_seq":2463,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.233562+0000","last_change":"2024-05-27T09:33:11.731031+0000","last_active":"2024-05-27T09:37:20.233562+0000","last_peered":"2024-05-27T09:37:20.233562+0000","last_clean":"2024-05-27T09:37:20.233562+0000","last_became_active":"2024-05-27T09:27:18.575115+0000","last_became_peered":"2024-05-27T09:27:18.575115+0000","last_unstale":"2024-05-27T09:37:20.233562+0000","last_undegraded":"2024-05-27T09:37:20.233562+0000","last_fullsized":"2024-05-27T09:37:20.233562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:56:57.284871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054063330000000001,"stat_sum":{"num_bytes":18656408,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":31,"num_read_kb":19450,"num_write":329,"num_write_kb":38861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"50'353","reported_seq":2474,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.367488+0000","last_change":"2024-05-27T09:33:11.451714+0000","last_active":"2024-05-27T09:37:37.367488+0000","last_peered":"2024-05-27T09:37:37.367488+0000","last_clean":"2024-05-27T09:37:37.367488+0000","last_became_active":"2024-05-27T09:27:18.563991+0000","last_became_peered":"2024-05-27T09:27:18.563991+0000","last_unstale":"2024-05-27T09:37:37.367488+0000","last_undegraded":"2024-05-27T09:37:37.367488+0000","last_fullsized":"2024-05-27T09:37:37.367488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:41:37.963962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009841753,"stat_sum":{"num_bytes":13963475,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":19,"num_read_kb":8208,"num_write":347,"num_write_kb":22013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"50'330","reported_seq":2454,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.234416+0000","last_change":"2024-05-27T09:33:11.732272+0000","last_active":"2024-05-27T09:37:20.234416+0000","last_peered":"2024-05-27T09:37:20.234416+0000","last_clean":"2024-05-27T09:37:20.234416+0000","last_became_active":"2024-05-27T09:27:18.562803+0000","last_became_peered":"2024-05-27T09:27:18.562803+0000","last_unstale":"2024-05-27T09:37:20.234416+0000","last_undegraded":"2024-05-27T09:37:20.234416+0000","last_fullsized":"2024-05-27T09:37:20.234416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:09:56.174365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067443290000000003,"stat_sum":{"num_bytes":23071782,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":21,"num_read_kb":4115,"num_write":328,"num_write_kb":30886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"50'384","reported_seq":2507,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.393993+0000","last_change":"2024-05-27T09:33:11.453888+0000","last_active":"2024-05-27T09:37:37.393993+0000","last_peered":"2024-05-27T09:37:37.393993+0000","last_clean":"2024-05-27T09:37:37.393993+0000","last_became_active":"2024-05-27T09:27:18.590083+0000","last_became_peered":"2024-05-27T09:27:18.590083+0000","last_unstale":"2024-05-27T09:37:37.393993+0000","last_undegraded":"2024-05-27T09:37:37.393993+0000","last_fullsized":"2024-05-27T09:37:37.393993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:00.006136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.008181008,"stat_sum":{"num_bytes":14547405,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":20,"num_read_kb":4115,"num_write":369,"num_write_kb":14397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"50'326","reported_seq":2445,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.169040+0000","last_change":"2024-05-27T09:33:11.728099+0000","last_active":"2024-05-27T09:37:20.169040+0000","last_peered":"2024-05-27T09:37:20.169040+0000","last_clean":"2024-05-27T09:37:20.169040+0000","last_became_active":"2024-05-27T09:27:18.557953+0000","last_became_peered":"2024-05-27T09:27:18.557953+0000","last_unstale":"2024-05-27T09:37:20.169040+0000","last_undegraded":"2024-05-27T09:37:20.169040+0000","last_fullsized":"2024-05-27T09:37:20.169040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:03:03.895209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027117650000000001,"stat_sum":{"num_bytes":18627167,"num_objects":187,"num_object_clones":0,"num_object_copies":374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":187,"num_whiteouts":0,"num_read":16,"num_read_kb":4110,"num_write":325,"num_write_kb":26562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"50'303","reported_seq":2423,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.207979+0000","last_change":"2024-05-27T09:33:11.729430+0000","last_active":"2024-05-27T09:37:20.207979+0000","last_peered":"2024-05-27T09:37:20.207979+0000","last_clean":"2024-05-27T09:37:20.207979+0000","last_became_active":"2024-05-27T09:27:18.562839+0000","last_became_peered":"2024-05-27T09:27:18.562839+0000","last_unstale":"2024-05-27T09:37:20.207979+0000","last_undegraded":"2024-05-27T09:37:20.207979+0000","last_fullsized":"2024-05-27T09:37:20.207979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:21:17.734561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032647399999999999,"stat_sum":{"num_bytes":14533754,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":17,"num_read_kb":11479,"num_write":303,"num_write_kb":21724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"50'344","reported_seq":2467,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.133428+0000","last_change":"2024-05-27T09:33:11.730679+0000","last_active":"2024-05-27T09:37:20.133428+0000","last_peered":"2024-05-27T09:37:20.133428+0000","last_clean":"2024-05-27T09:37:20.133428+0000","last_became_active":"2024-05-27T09:27:18.561475+0000","last_became_peered":"2024-05-27T09:27:18.561475+0000","last_unstale":"2024-05-27T09:37:20.133428+0000","last_undegraded":"2024-05-27T09:37:20.133428+0000","last_fullsized":"2024-05-27T09:37:20.133428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:32:06.826783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053708779999999999,"stat_sum":{"num_bytes":11382190,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":20,"num_read_kb":4114,"num_write":347,"num_write_kb":19366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"50'377","reported_seq":2507,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.357466+0000","last_change":"2024-05-27T09:33:11.454566+0000","last_active":"2024-05-27T09:37:37.357466+0000","last_peered":"2024-05-27T09:37:37.357466+0000","last_clean":"2024-05-27T09:37:37.357466+0000","last_became_active":"2024-05-27T09:27:18.563103+0000","last_became_peered":"2024-05-27T09:27:18.563103+0000","last_unstale":"2024-05-27T09:37:37.357466+0000","last_undegraded":"2024-05-27T09:37:37.357466+0000","last_fullsized":"2024-05-27T09:37:37.357466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:25.776404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011066041,"stat_sum":{"num_bytes":30187091,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":28,"num_read_kb":16405,"num_write":369,"num_write_kb":50152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"50'389","reported_seq":2511,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.231487+0000","last_change":"2024-05-27T09:33:11.625969+0000","last_active":"2024-05-27T09:37:20.231487+0000","last_peered":"2024-05-27T09:37:20.231487+0000","last_clean":"2024-05-27T09:37:20.231487+0000","last_became_active":"2024-05-27T09:27:18.560244+0000","last_became_peered":"2024-05-27T09:27:18.560244+0000","last_unstale":"2024-05-27T09:37:20.231487+0000","last_undegraded":"2024-05-27T09:37:20.231487+0000","last_fullsized":"2024-05-27T09:37:20.231487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:14:56.979490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012131959999999999,"stat_sum":{"num_bytes":8541274,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":19,"num_read_kb":2631,"num_write":389,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"50'363","reported_seq":2493,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.377327+0000","last_change":"2024-05-27T09:33:11.447687+0000","last_active":"2024-05-27T09:37:37.377327+0000","last_peered":"2024-05-27T09:37:37.377327+0000","last_clean":"2024-05-27T09:37:37.377327+0000","last_became_active":"2024-05-27T09:27:18.567845+0000","last_became_peered":"2024-05-27T09:27:18.567845+0000","last_unstale":"2024-05-27T09:37:37.377327+0000","last_undegraded":"2024-05-27T09:37:37.377327+0000","last_fullsized":"2024-05-27T09:37:37.377327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:48:36.494196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082936180000000009,"stat_sum":{"num_bytes":1444930,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":352,"num_write_kb":9783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"50'336","reported_seq":2458,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.370918+0000","last_change":"2024-05-27T09:33:11.738447+0000","last_active":"2024-05-27T09:37:37.370918+0000","last_peered":"2024-05-27T09:37:37.370918+0000","last_clean":"2024-05-27T09:37:37.370918+0000","last_became_active":"2024-05-27T09:27:18.556156+0000","last_became_peered":"2024-05-27T09:27:18.556156+0000","last_unstale":"2024-05-27T09:37:37.370918+0000","last_undegraded":"2024-05-27T09:37:37.370918+0000","last_fullsized":"2024-05-27T09:37:37.370918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:25:11.600058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0094378269999999993,"stat_sum":{"num_bytes":19241078,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":19,"num_read_kb":8208,"num_write":329,"num_write_kb":23062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"50'73","reported_seq":156,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:21.449092+0000","last_change":"2024-05-27T09:27:17.548038+0000","last_active":"2024-05-27T09:37:21.449092+0000","last_peered":"2024-05-27T09:37:21.449092+0000","last_clean":"2024-05-27T09:37:21.449092+0000","last_became_active":"2024-05-27T09:27:17.547653+0000","last_became_peered":"2024-05-27T09:27:17.547653+0000","last_unstale":"2024-05-27T09:37:21.449092+0000","last_undegraded":"2024-05-27T09:37:21.449092+0000","last_fullsized":"2024-05-27T09:37:21.449092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195412,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":4100,"num_write":71,"num_write_kb":4220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"50'320","reported_seq":2441,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.327073+0000","last_change":"2024-05-27T09:33:11.733348+0000","last_active":"2024-05-27T09:37:20.327073+0000","last_peered":"2024-05-27T09:37:20.327073+0000","last_clean":"2024-05-27T09:37:20.327073+0000","last_became_active":"2024-05-27T09:27:18.542715+0000","last_became_peered":"2024-05-27T09:27:18.542715+0000","last_unstale":"2024-05-27T09:37:20.327073+0000","last_undegraded":"2024-05-27T09:37:20.327073+0000","last_fullsized":"2024-05-27T09:37:20.327073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:45:47.000935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004594323,"stat_sum":{"num_bytes":10080804,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":19,"num_read_kb":4113,"num_write":319,"num_write_kb":14102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"49'299","reported_seq":384,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.547628+0000","last_change":"2024-05-27T09:27:17.570014+0000","last_active":"2024-05-27T09:37:15.547628+0000","last_peered":"2024-05-27T09:37:15.547628+0000","last_clean":"2024-05-27T09:37:15.547628+0000","last_became_active":"2024-05-27T09:27:17.569673+0000","last_became_peered":"2024-05-27T09:27:17.569673+0000","last_unstale":"2024-05-27T09:37:15.547628+0000","last_undegraded":"2024-05-27T09:37:15.547628+0000","last_fullsized":"2024-05-27T09:37:15.547628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:26.210041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":22,"num_read_kb":12200,"num_write":300,"num_write_kb":12509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"49'46","reported_seq":129,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.952341+0000","last_change":"2024-05-27T09:27:17.565489+0000","last_active":"2024-05-27T09:37:15.952341+0000","last_peered":"2024-05-27T09:37:15.952341+0000","last_clean":"2024-05-27T09:37:15.952341+0000","last_became_active":"2024-05-27T09:27:17.565137+0000","last_became_peered":"2024-05-27T09:27:17.565137+0000","last_unstale":"2024-05-27T09:37:15.952341+0000","last_undegraded":"2024-05-27T09:37:15.952341+0000","last_fullsized":"2024-05-27T09:37:15.952341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":19,"num_read_kb":4114,"num_write":50,"num_write_kb":4167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"50'389","reported_seq":2515,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.164212+0000","last_change":"2024-05-27T09:33:11.733228+0000","last_active":"2024-05-27T09:37:20.164212+0000","last_peered":"2024-05-27T09:37:20.164212+0000","last_clean":"2024-05-27T09:37:20.164212+0000","last_became_active":"2024-05-27T09:27:18.564115+0000","last_became_peered":"2024-05-27T09:27:18.564115+0000","last_unstale":"2024-05-27T09:37:20.164212+0000","last_undegraded":"2024-05-27T09:37:20.164212+0000","last_fullsized":"2024-05-27T09:37:20.164212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:06:44.772830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066511199999999999,"stat_sum":{"num_bytes":10503109,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":23,"num_read_kb":4118,"num_write":389,"num_write_kb":18659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"50'294","reported_seq":371,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.547032+0000","last_change":"2024-05-27T09:27:17.561204+0000","last_active":"2024-05-27T09:37:15.547032+0000","last_peered":"2024-05-27T09:37:15.547032+0000","last_clean":"2024-05-27T09:37:15.547032+0000","last_became_active":"2024-05-27T09:27:17.560149+0000","last_became_peered":"2024-05-27T09:27:17.560149+0000","last_unstale":"2024-05-27T09:37:15.547032+0000","last_undegraded":"2024-05-27T09:37:15.547032+0000","last_fullsized":"2024-05-27T09:37:15.547032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.284270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":291,"num_write_kb":12506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"50'334","reported_seq":2456,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.237522+0000","last_change":"2024-05-27T09:33:11.734046+0000","last_active":"2024-05-27T09:37:20.237522+0000","last_peered":"2024-05-27T09:37:20.237522+0000","last_clean":"2024-05-27T09:37:20.237522+0000","last_became_active":"2024-05-27T09:27:18.541946+0000","last_became_peered":"2024-05-27T09:27:18.541946+0000","last_unstale":"2024-05-27T09:37:20.237522+0000","last_undegraded":"2024-05-27T09:37:20.237522+0000","last_fullsized":"2024-05-27T09:37:20.237522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:30.087688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030229530000000001,"stat_sum":{"num_bytes":19296821,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":20,"num_read_kb":12305,"num_write":330,"num_write_kb":31302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"50'9","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.530539+0000","last_change":"2024-05-27T09:27:17.561714+0000","last_active":"2024-05-27T09:37:20.530539+0000","last_peered":"2024-05-27T09:37:20.530539+0000","last_clean":"2024-05-27T09:37:20.530539+0000","last_became_active":"2024-05-27T09:27:17.561355+0000","last_became_peered":"2024-05-27T09:27:17.561355+0000","last_unstale":"2024-05-27T09:37:20.530539+0000","last_undegraded":"2024-05-27T09:37:20.530539+0000","last_fullsized":"2024-05-27T09:37:20.530539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:07:02.481451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":8,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"50'354","reported_seq":2478,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.394166+0000","last_change":"2024-05-27T09:33:11.454088+0000","last_active":"2024-05-27T09:37:37.394166+0000","last_peered":"2024-05-27T09:37:37.394166+0000","last_clean":"2024-05-27T09:37:37.394166+0000","last_became_active":"2024-05-27T09:27:18.587921+0000","last_became_peered":"2024-05-27T09:27:18.587921+0000","last_unstale":"2024-05-27T09:37:37.394166+0000","last_undegraded":"2024-05-27T09:37:37.394166+0000","last_fullsized":"2024-05-27T09:37:37.394166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:17:20.248600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0078895319999999994,"stat_sum":{"num_bytes":10502055,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":21,"num_read_kb":4116,"num_write":345,"num_write_kb":10394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"49'65","reported_seq":144,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.540718+0000","last_change":"2024-05-27T09:27:17.565854+0000","last_active":"2024-05-27T09:37:15.540718+0000","last_peered":"2024-05-27T09:37:15.540718+0000","last_clean":"2024-05-27T09:37:15.540718+0000","last_became_active":"2024-05-27T09:27:17.565488+0000","last_became_peered":"2024-05-27T09:27:17.565488+0000","last_unstale":"2024-05-27T09:37:15.540718+0000","last_undegraded":"2024-05-27T09:37:15.540718+0000","last_fullsized":"2024-05-27T09:37:15.540718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:53.711508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":4102,"num_write":67,"num_write_kb":4232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"50'329","reported_seq":2455,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.223387+0000","last_change":"2024-05-27T09:33:11.733328+0000","last_active":"2024-05-27T09:37:20.223387+0000","last_peered":"2024-05-27T09:37:20.223387+0000","last_clean":"2024-05-27T09:37:20.223387+0000","last_became_active":"2024-05-27T09:27:18.585243+0000","last_became_peered":"2024-05-27T09:27:18.585243+0000","last_unstale":"2024-05-27T09:37:20.223387+0000","last_undegraded":"2024-05-27T09:37:20.223387+0000","last_fullsized":"2024-05-27T09:37:20.223387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:18:44.814432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067133569999999997,"stat_sum":{"num_bytes":18063712,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":23,"num_read_kb":8211,"num_write":328,"num_write_kb":30099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"50'204","reported_seq":284,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.529879+0000","last_change":"2024-05-27T09:27:17.548651+0000","last_active":"2024-05-27T09:37:20.529879+0000","last_peered":"2024-05-27T09:37:20.529879+0000","last_clean":"2024-05-27T09:37:20.529879+0000","last_became_active":"2024-05-27T09:27:17.548307+0000","last_became_peered":"2024-05-27T09:27:17.548307+0000","last_unstale":"2024-05-27T09:37:20.529879+0000","last_undegraded":"2024-05-27T09:37:20.529879+0000","last_fullsized":"2024-05-27T09:37:20.529879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:48.905422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":206,"num_write_kb":4295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"50'341","reported_seq":2460,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.387494+0000","last_change":"2024-05-27T09:33:11.455764+0000","last_active":"2024-05-27T09:37:37.387494+0000","last_peered":"2024-05-27T09:37:37.387494+0000","last_clean":"2024-05-27T09:37:37.387494+0000","last_became_active":"2024-05-27T09:27:18.581742+0000","last_became_peered":"2024-05-27T09:27:18.581742+0000","last_unstale":"2024-05-27T09:37:37.387494+0000","last_undegraded":"2024-05-27T09:37:37.387494+0000","last_fullsized":"2024-05-27T09:37:37.387494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:14:45.868801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089613100000000001,"stat_sum":{"num_bytes":20607307,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":17,"num_read_kb":10914,"num_write":328,"num_write_kb":24390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"50'245","reported_seq":328,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.536925+0000","last_change":"2024-05-27T09:27:17.556419+0000","last_active":"2024-05-27T09:37:15.536925+0000","last_peered":"2024-05-27T09:37:15.536925+0000","last_clean":"2024-05-27T09:37:15.536925+0000","last_became_active":"2024-05-27T09:27:17.556036+0000","last_became_peered":"2024-05-27T09:27:17.556036+0000","last_unstale":"2024-05-27T09:37:15.536925+0000","last_undegraded":"2024-05-27T09:37:15.536925+0000","last_fullsized":"2024-05-27T09:37:15.536925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:48:20.718085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":13,"num_read_kb":8203,"num_write":243,"num_write_kb":12459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"50'335","reported_seq":2457,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.383012+0000","last_change":"2024-05-27T09:33:11.449451+0000","last_active":"2024-05-27T09:37:37.383012+0000","last_peered":"2024-05-27T09:37:37.383012+0000","last_clean":"2024-05-27T09:37:37.383012+0000","last_became_active":"2024-05-27T09:27:18.565500+0000","last_became_peered":"2024-05-27T09:27:18.565500+0000","last_unstale":"2024-05-27T09:37:37.383012+0000","last_undegraded":"2024-05-27T09:37:37.383012+0000","last_fullsized":"2024-05-27T09:37:37.383012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:25:55.844796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011618516000000001,"stat_sum":{"num_bytes":10741446,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":19,"num_read_kb":4114,"num_write":329,"num_write_kb":14761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"49'17","reported_seq":105,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.861134+0000","last_change":"2024-05-27T09:27:17.572168+0000","last_active":"2024-05-27T09:37:15.861134+0000","last_peered":"2024-05-27T09:37:15.861134+0000","last_clean":"2024-05-27T09:37:15.861134+0000","last_became_active":"2024-05-27T09:27:17.571709+0000","last_became_peered":"2024-05-27T09:27:17.571709+0000","last_unstale":"2024-05-27T09:37:15.861134+0000","last_undegraded":"2024-05-27T09:37:15.861134+0000","last_fullsized":"2024-05-27T09:37:15.861134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":30,"num_read_kb":8224,"num_write":18,"num_write_kb":8221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"50'334","reported_seq":2463,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.312575+0000","last_change":"2024-05-27T09:33:11.728047+0000","last_active":"2024-05-27T09:37:20.312575+0000","last_peered":"2024-05-27T09:37:20.312575+0000","last_clean":"2024-05-27T09:37:20.312575+0000","last_became_active":"2024-05-27T09:27:18.564589+0000","last_became_peered":"2024-05-27T09:27:18.564589+0000","last_unstale":"2024-05-27T09:37:20.312575+0000","last_undegraded":"2024-05-27T09:37:20.312575+0000","last_fullsized":"2024-05-27T09:37:20.312575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:09:11.832557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028467779999999999,"stat_sum":{"num_bytes":14347466,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":26,"num_read_kb":16403,"num_write":332,"num_write_kb":30566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"50'8","reported_seq":85,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.547741+0000","last_change":"2024-05-27T09:27:17.561344+0000","last_active":"2024-05-27T09:37:15.547741+0000","last_peered":"2024-05-27T09:37:15.547741+0000","last_clean":"2024-05-27T09:37:15.547741+0000","last_became_active":"2024-05-27T09:27:17.554860+0000","last_became_peered":"2024-05-27T09:27:17.554860+0000","last_unstale":"2024-05-27T09:37:15.547741+0000","last_undegraded":"2024-05-27T09:37:15.547741+0000","last_fullsized":"2024-05-27T09:37:15.547741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:16.880101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"50'359","reported_seq":2484,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.391039+0000","last_change":"2024-05-27T09:33:11.456274+0000","last_active":"2024-05-27T09:37:37.391039+0000","last_peered":"2024-05-27T09:37:37.391039+0000","last_clean":"2024-05-27T09:37:37.391039+0000","last_became_active":"2024-05-27T09:27:18.572024+0000","last_became_peered":"2024-05-27T09:27:18.572024+0000","last_unstale":"2024-05-27T09:37:37.391039+0000","last_undegraded":"2024-05-27T09:37:37.391039+0000","last_fullsized":"2024-05-27T09:37:37.391039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:40:00.595445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010292084,"stat_sum":{"num_bytes":12019831,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":346,"num_write_kb":11826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"50'12","reported_seq":97,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.541647+0000","last_change":"2024-05-27T09:27:17.575859+0000","last_active":"2024-05-27T09:37:15.541647+0000","last_peered":"2024-05-27T09:37:15.541647+0000","last_clean":"2024-05-27T09:37:15.541647+0000","last_became_active":"2024-05-27T09:27:17.574695+0000","last_became_peered":"2024-05-27T09:27:17.574695+0000","last_unstale":"2024-05-27T09:37:15.541647+0000","last_undegraded":"2024-05-27T09:37:15.541647+0000","last_fullsized":"2024-05-27T09:37:15.541647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:24:19.379819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1331,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":9,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"50'340","reported_seq":2462,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.354340+0000","last_change":"2024-05-27T09:33:11.736362+0000","last_active":"2024-05-27T09:37:37.354340+0000","last_peered":"2024-05-27T09:37:37.354340+0000","last_clean":"2024-05-27T09:37:37.354340+0000","last_became_active":"2024-05-27T09:27:18.554932+0000","last_became_peered":"2024-05-27T09:27:18.554932+0000","last_unstale":"2024-05-27T09:37:37.354340+0000","last_undegraded":"2024-05-27T09:37:37.354340+0000","last_fullsized":"2024-05-27T09:37:37.354340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:32:26.268173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0083893030000000007,"stat_sum":{"num_bytes":24206949,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":20,"num_read_kb":12303,"num_write":329,"num_write_kb":23805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"49'68","reported_seq":161,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.631995+0000","last_change":"2024-05-27T09:27:17.547895+0000","last_active":"2024-05-27T09:37:15.631995+0000","last_peered":"2024-05-27T09:37:15.631995+0000","last_clean":"2024-05-27T09:37:15.631995+0000","last_became_active":"2024-05-27T09:27:17.547523+0000","last_became_peered":"2024-05-27T09:27:17.547523+0000","last_unstale":"2024-05-27T09:37:15.631995+0000","last_undegraded":"2024-05-27T09:37:15.631995+0000","last_fullsized":"2024-05-27T09:37:15.631995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":66,"num_write_kb":8220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"50'366","reported_seq":2491,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.385150+0000","last_change":"2024-05-27T09:33:11.457430+0000","last_active":"2024-05-27T09:37:37.385150+0000","last_peered":"2024-05-27T09:37:37.385150+0000","last_clean":"2024-05-27T09:37:37.385150+0000","last_became_active":"2024-05-27T09:27:18.582360+0000","last_became_peered":"2024-05-27T09:27:18.582360+0000","last_unstale":"2024-05-27T09:37:37.385150+0000","last_undegraded":"2024-05-27T09:37:37.385150+0000","last_fullsized":"2024-05-27T09:37:37.385150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:40:26.596775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0087858729999999996,"stat_sum":{"num_bytes":23260375,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":23,"num_read_kb":13874,"num_write":361,"num_write_kb":35144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"49'42","reported_seq":123,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.543499+0000","last_change":"2024-05-27T09:27:17.571481+0000","last_active":"2024-05-27T09:37:15.543499+0000","last_peered":"2024-05-27T09:37:15.543499+0000","last_clean":"2024-05-27T09:37:15.543499+0000","last_became_active":"2024-05-27T09:27:17.571146+0000","last_became_peered":"2024-05-27T09:27:17.571146+0000","last_unstale":"2024-05-27T09:37:15.543499+0000","last_undegraded":"2024-05-27T09:37:15.543499+0000","last_fullsized":"2024-05-27T09:37:15.543499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:18.038221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":8,"num_read_kb":4103,"num_write":60,"num_write_kb":4132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"50'302","reported_seq":2417,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.390053+0000","last_change":"2024-05-27T09:33:11.454615+0000","last_active":"2024-05-27T09:37:37.390053+0000","last_peered":"2024-05-27T09:37:37.390053+0000","last_clean":"2024-05-27T09:37:37.390053+0000","last_became_active":"2024-05-27T09:27:18.579440+0000","last_became_peered":"2024-05-27T09:27:18.579440+0000","last_unstale":"2024-05-27T09:37:37.390053+0000","last_undegraded":"2024-05-27T09:37:37.390053+0000","last_fullsized":"2024-05-27T09:37:37.390053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:06:43.673978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079260630000000006,"stat_sum":{"num_bytes":7949059,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":12,"num_read_kb":4346,"num_write":296,"num_write_kb":16109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"49'27","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.632660+0000","last_change":"2024-05-27T09:27:17.551295+0000","last_active":"2024-05-27T09:37:15.632660+0000","last_peered":"2024-05-27T09:37:15.632660+0000","last_clean":"2024-05-27T09:37:15.632660+0000","last_became_active":"2024-05-27T09:27:17.550861+0000","last_became_peered":"2024-05-27T09:27:17.550861+0000","last_unstale":"2024-05-27T09:37:15.632660+0000","last_undegraded":"2024-05-27T09:37:15.632660+0000","last_fullsized":"2024-05-27T09:37:15.632660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:13.955191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":4099,"num_write":31,"num_write_kb":4134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"50'337","reported_seq":2460,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.376487+0000","last_change":"2024-05-27T09:33:11.453500+0000","last_active":"2024-05-27T09:37:37.376487+0000","last_peered":"2024-05-27T09:37:37.376487+0000","last_clean":"2024-05-27T09:37:37.376487+0000","last_became_active":"2024-05-27T09:27:18.543683+0000","last_became_peered":"2024-05-27T09:27:18.543683+0000","last_unstale":"2024-05-27T09:37:37.376487+0000","last_undegraded":"2024-05-27T09:37:37.376487+0000","last_fullsized":"2024-05-27T09:37:37.376487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:29:05.179158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012478668,"stat_sum":{"num_bytes":14963144,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":20,"num_read_kb":4115,"num_write":326,"num_write_kb":22968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"49'93","reported_seq":171,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.952185+0000","last_change":"2024-05-27T09:27:17.559255+0000","last_active":"2024-05-27T09:37:15.952185+0000","last_peered":"2024-05-27T09:37:15.952185+0000","last_clean":"2024-05-27T09:37:15.952185+0000","last_became_active":"2024-05-27T09:27:17.558973+0000","last_became_peered":"2024-05-27T09:27:17.558973+0000","last_unstale":"2024-05-27T09:37:15.952185+0000","last_undegraded":"2024-05-27T09:37:15.952185+0000","last_fullsized":"2024-05-27T09:37:15.952185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":4097,"num_write":96,"num_write_kb":4263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"50'366","reported_seq":2483,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.371889+0000","last_change":"2024-05-27T09:33:11.452657+0000","last_active":"2024-05-27T09:37:37.371889+0000","last_peered":"2024-05-27T09:37:37.371889+0000","last_clean":"2024-05-27T09:37:37.371889+0000","last_became_active":"2024-05-27T09:27:18.557076+0000","last_became_peered":"2024-05-27T09:27:18.557076+0000","last_unstale":"2024-05-27T09:37:37.371889+0000","last_undegraded":"2024-05-27T09:37:37.371889+0000","last_fullsized":"2024-05-27T09:37:37.371889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:26:39.506639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014584746000000001,"stat_sum":{"num_bytes":15385587,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":358,"num_write_kb":15182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"49'49","reported_seq":134,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.953808+0000","last_change":"2024-05-27T09:27:17.573700+0000","last_active":"2024-05-27T09:37:15.953808+0000","last_peered":"2024-05-27T09:37:15.953808+0000","last_clean":"2024-05-27T09:37:15.953808+0000","last_became_active":"2024-05-27T09:27:17.573484+0000","last_became_peered":"2024-05-27T09:27:17.573484+0000","last_unstale":"2024-05-27T09:37:15.953808+0000","last_undegraded":"2024-05-27T09:37:15.953808+0000","last_fullsized":"2024-05-27T09:37:15.953808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:48.905422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":51,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"50'336","reported_seq":2461,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.377629+0000","last_change":"2024-05-27T09:33:11.456688+0000","last_active":"2024-05-27T09:37:37.377629+0000","last_peered":"2024-05-27T09:37:37.377629+0000","last_clean":"2024-05-27T09:37:37.377629+0000","last_became_active":"2024-05-27T09:27:18.574959+0000","last_became_peered":"2024-05-27T09:27:18.574959+0000","last_unstale":"2024-05-27T09:37:37.377629+0000","last_undegraded":"2024-05-27T09:37:37.377629+0000","last_fullsized":"2024-05-27T09:37:37.377629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:53.188459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0093604480000000004,"stat_sum":{"num_bytes":20164393,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":23,"num_read_kb":12305,"num_write":329,"num_write_kb":28020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":173,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.533665+0000","last_change":"2024-05-27T09:27:12.693423+0000","last_active":"2024-05-27T09:37:15.533665+0000","last_peered":"2024-05-27T09:37:15.533665+0000","last_clean":"2024-05-27T09:37:15.533665+0000","last_became_active":"2024-05-27T09:27:12.690301+0000","last_became_peered":"2024-05-27T09:27:12.690301+0000","last_unstale":"2024-05-27T09:37:15.533665+0000","last_undegraded":"2024-05-27T09:37:15.533665+0000","last_fullsized":"2024-05-27T09:37:15.533665+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:11.463226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:11.463226+0000","last_clean_scrub_stamp":"2024-05-27T09:27:11.463226+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:01.810071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'12","reported_seq":93,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.547944+0000","last_change":"2024-05-27T09:27:17.567670+0000","last_active":"2024-05-27T09:37:15.547944+0000","last_peered":"2024-05-27T09:37:15.547944+0000","last_clean":"2024-05-27T09:37:15.547944+0000","last_became_active":"2024-05-27T09:27:17.566292+0000","last_became_peered":"2024-05-27T09:27:17.566292+0000","last_unstale":"2024-05-27T09:37:15.547944+0000","last_undegraded":"2024-05-27T09:37:15.547944+0000","last_fullsized":"2024-05-27T09:37:15.547944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:18.038221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194338,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":14,"num_write_kb":4206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"50'333","reported_seq":2448,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.242277+0000","last_change":"2024-05-27T09:33:11.733467+0000","last_active":"2024-05-27T09:37:20.242277+0000","last_peered":"2024-05-27T09:37:20.242277+0000","last_clean":"2024-05-27T09:37:20.242277+0000","last_became_active":"2024-05-27T09:27:18.570782+0000","last_became_peered":"2024-05-27T09:27:18.570782+0000","last_unstale":"2024-05-27T09:37:20.242277+0000","last_undegraded":"2024-05-27T09:37:20.242277+0000","last_fullsized":"2024-05-27T09:37:20.242277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:22.877083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044685039999999999,"stat_sum":{"num_bytes":10376859,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":333,"num_write_kb":18496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"50'344","reported_seq":428,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:17.550853+0000","last_change":"2024-05-27T09:27:17.563259+0000","last_active":"2024-05-27T09:37:17.550853+0000","last_peered":"2024-05-27T09:37:17.550853+0000","last_clean":"2024-05-27T09:37:17.550853+0000","last_became_active":"2024-05-27T09:27:17.563010+0000","last_became_peered":"2024-05-27T09:27:17.563010+0000","last_unstale":"2024-05-27T09:37:17.550853+0000","last_undegraded":"2024-05-27T09:37:17.550853+0000","last_fullsized":"2024-05-27T09:37:17.550853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:53.711508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":8210,"num_write":339,"num_write_kb":12482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"50'318","reported_seq":2431,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.233244+0000","last_change":"2024-05-27T09:33:11.730552+0000","last_active":"2024-05-27T09:37:20.233244+0000","last_peered":"2024-05-27T09:37:20.233244+0000","last_clean":"2024-05-27T09:37:20.233244+0000","last_became_active":"2024-05-27T09:27:18.574090+0000","last_became_peered":"2024-05-27T09:27:18.574090+0000","last_unstale":"2024-05-27T09:37:20.233244+0000","last_undegraded":"2024-05-27T09:37:20.233244+0000","last_fullsized":"2024-05-27T09:37:20.233244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:29:43.530819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002904064,"stat_sum":{"num_bytes":5516261,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":316,"num_write_kb":17833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"50'122","reported_seq":316,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.860934+0000","last_change":"2024-05-27T09:27:17.551310+0000","last_active":"2024-05-27T09:37:15.860934+0000","last_peered":"2024-05-27T09:37:15.860934+0000","last_clean":"2024-05-27T09:37:15.860934+0000","last_became_active":"2024-05-27T09:27:17.550982+0000","last_became_peered":"2024-05-27T09:27:17.550982+0000","last_unstale":"2024-05-27T09:37:15.860934+0000","last_undegraded":"2024-05-27T09:37:15.860934+0000","last_fullsized":"2024-05-27T09:37:15.860934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":534380,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":130,"num_read_kb":615,"num_write":120,"num_write_kb":620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"50'340","reported_seq":2465,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.169125+0000","last_change":"2024-05-27T09:33:11.732933+0000","last_active":"2024-05-27T09:37:20.169125+0000","last_peered":"2024-05-27T09:37:20.169125+0000","last_clean":"2024-05-27T09:37:20.169125+0000","last_became_active":"2024-05-27T09:27:18.559017+0000","last_became_peered":"2024-05-27T09:27:18.559017+0000","last_unstale":"2024-05-27T09:37:20.169125+0000","last_undegraded":"2024-05-27T09:37:20.169125+0000","last_fullsized":"2024-05-27T09:37:20.169125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:06:28.204044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065168789999999997,"stat_sum":{"num_bytes":26414737,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":21,"num_read_kb":8209,"num_write":340,"num_write_kb":34165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"49'48","reported_seq":125,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.547864+0000","last_change":"2024-05-27T09:27:17.563947+0000","last_active":"2024-05-27T09:37:15.547864+0000","last_peered":"2024-05-27T09:37:15.547864+0000","last_clean":"2024-05-27T09:37:15.547864+0000","last_became_active":"2024-05-27T09:27:17.563640+0000","last_became_peered":"2024-05-27T09:27:17.563640+0000","last_unstale":"2024-05-27T09:37:15.547864+0000","last_undegraded":"2024-05-27T09:37:15.547864+0000","last_fullsized":"2024-05-27T09:37:15.547864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:02.747708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":43,"num_write_kb":8223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"50'359","reported_seq":2479,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.388774+0000","last_change":"2024-05-27T09:33:11.442514+0000","last_active":"2024-05-27T09:37:37.388774+0000","last_peered":"2024-05-27T09:37:37.388774+0000","last_clean":"2024-05-27T09:37:37.388774+0000","last_became_active":"2024-05-27T09:27:18.574710+0000","last_became_peered":"2024-05-27T09:27:18.574710+0000","last_unstale":"2024-05-27T09:37:37.388774+0000","last_undegraded":"2024-05-27T09:37:37.388774+0000","last_fullsized":"2024-05-27T09:37:37.388774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:50:58.577908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075561990000000004,"stat_sum":{"num_bytes":9767075,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":17,"num_read_kb":8206,"num_write":352,"num_write_kb":9721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"49'81","reported_seq":157,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.952072+0000","last_change":"2024-05-27T09:27:17.550749+0000","last_active":"2024-05-27T09:37:15.952072+0000","last_peered":"2024-05-27T09:37:15.952072+0000","last_clean":"2024-05-27T09:37:15.952072+0000","last_became_active":"2024-05-27T09:27:17.550378+0000","last_became_peered":"2024-05-27T09:27:17.550378+0000","last_unstale":"2024-05-27T09:37:15.952072+0000","last_undegraded":"2024-05-27T09:37:15.952072+0000","last_fullsized":"2024-05-27T09:37:15.952072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:20.167629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83,"num_write_kb":4165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"50'360","reported_seq":2483,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.368574+0000","last_change":"2024-05-27T09:33:11.451931+0000","last_active":"2024-05-27T09:37:37.368574+0000","last_peered":"2024-05-27T09:37:37.368574+0000","last_clean":"2024-05-27T09:37:37.368574+0000","last_became_active":"2024-05-27T09:27:18.566326+0000","last_became_peered":"2024-05-27T09:27:18.566326+0000","last_unstale":"2024-05-27T09:37:37.368574+0000","last_undegraded":"2024-05-27T09:37:37.368574+0000","last_fullsized":"2024-05-27T09:37:37.368574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:50.848181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075094710000000002,"stat_sum":{"num_bytes":14788118,"num_objects":187,"num_object_clones":0,"num_object_copies":374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":187,"num_whiteouts":0,"num_read":20,"num_read_kb":8209,"num_write":356,"num_write_kb":35101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'352","reported_seq":430,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.548573+0000","last_change":"2024-05-27T09:27:17.557912+0000","last_active":"2024-05-27T09:37:15.548573+0000","last_peered":"2024-05-27T09:37:15.548573+0000","last_clean":"2024-05-27T09:37:15.548573+0000","last_became_active":"2024-05-27T09:27:17.557216+0000","last_became_peered":"2024-05-27T09:27:17.557216+0000","last_unstale":"2024-05-27T09:37:15.548573+0000","last_undegraded":"2024-05-27T09:37:15.548573+0000","last_fullsized":"2024-05-27T09:37:15.548573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:21:22.020724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":4098,"num_write":353,"num_write_kb":8433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"50'338","reported_seq":2454,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.161215+0000","last_change":"2024-05-27T09:33:11.728508+0000","last_active":"2024-05-27T09:37:20.161215+0000","last_peered":"2024-05-27T09:37:20.161215+0000","last_clean":"2024-05-27T09:37:20.161215+0000","last_became_active":"2024-05-27T09:27:18.562565+0000","last_became_peered":"2024-05-27T09:27:18.562565+0000","last_unstale":"2024-05-27T09:37:20.161215+0000","last_undegraded":"2024-05-27T09:37:20.161215+0000","last_fullsized":"2024-05-27T09:37:20.161215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:45:47.000935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030336680000000002,"stat_sum":{"num_bytes":10461277,"num_objects":176,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":176,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":338,"num_write_kb":10392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"50'197","reported_seq":279,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:28.642623+0000","last_change":"2024-05-27T09:27:17.571911+0000","last_active":"2024-05-27T09:37:28.642623+0000","last_peered":"2024-05-27T09:37:28.642623+0000","last_clean":"2024-05-27T09:37:28.642623+0000","last_became_active":"2024-05-27T09:27:17.571551+0000","last_became_peered":"2024-05-27T09:27:17.571551+0000","last_unstale":"2024-05-27T09:37:28.642623+0000","last_undegraded":"2024-05-27T09:37:28.642623+0000","last_fullsized":"2024-05-27T09:37:28.642623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:20.167629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":453,"num_write_kb":2056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"50'324","reported_seq":2442,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.167207+0000","last_change":"2024-05-27T09:33:11.734048+0000","last_active":"2024-05-27T09:37:20.167207+0000","last_peered":"2024-05-27T09:37:20.167207+0000","last_clean":"2024-05-27T09:37:20.167207+0000","last_became_active":"2024-05-27T09:27:18.558463+0000","last_became_peered":"2024-05-27T09:27:18.558463+0000","last_unstale":"2024-05-27T09:37:20.167207+0000","last_undegraded":"2024-05-27T09:37:20.167207+0000","last_fullsized":"2024-05-27T09:37:20.167207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:29:43.530819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041648589999999999,"stat_sum":{"num_bytes":14892598,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":326,"num_write_kb":22861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"49'171","reported_seq":249,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.549131+0000","last_change":"2024-05-27T09:27:17.567973+0000","last_active":"2024-05-27T09:37:15.549131+0000","last_peered":"2024-05-27T09:37:15.549131+0000","last_clean":"2024-05-27T09:37:15.549131+0000","last_became_active":"2024-05-27T09:27:17.567650+0000","last_became_peered":"2024-05-27T09:27:17.567650+0000","last_unstale":"2024-05-27T09:37:15.549131+0000","last_undegraded":"2024-05-27T09:37:15.549131+0000","last_fullsized":"2024-05-27T09:37:15.549131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:25:52.924185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":176,"num_write_kb":4290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"50'340","reported_seq":2469,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.169202+0000","last_change":"2024-05-27T09:33:11.733662+0000","last_active":"2024-05-27T09:37:20.169202+0000","last_peered":"2024-05-27T09:37:20.169202+0000","last_clean":"2024-05-27T09:37:20.169202+0000","last_became_active":"2024-05-27T09:27:18.554462+0000","last_became_peered":"2024-05-27T09:27:18.554462+0000","last_unstale":"2024-05-27T09:37:20.169202+0000","last_undegraded":"2024-05-27T09:37:20.169202+0000","last_fullsized":"2024-05-27T09:37:20.169202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:21:17.734561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042444429999999997,"stat_sum":{"num_bytes":6471160,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":27,"num_read_kb":4122,"num_write":339,"num_write_kb":6465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"49'126","reported_seq":205,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.539172+0000","last_change":"2024-05-27T09:27:17.570300+0000","last_active":"2024-05-27T09:37:15.539172+0000","last_peered":"2024-05-27T09:37:15.539172+0000","last_clean":"2024-05-27T09:37:15.539172+0000","last_became_active":"2024-05-27T09:27:17.569685+0000","last_became_peered":"2024-05-27T09:27:17.569685+0000","last_unstale":"2024-05-27T09:37:15.539172+0000","last_undegraded":"2024-05-27T09:37:15.539172+0000","last_fullsized":"2024-05-27T09:37:15.539172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:26.210041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":4102,"num_write":127,"num_write_kb":4196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"50'355","reported_seq":2484,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.352377+0000","last_change":"2024-05-27T09:33:11.450475+0000","last_active":"2024-05-27T09:37:37.352377+0000","last_peered":"2024-05-27T09:37:37.352377+0000","last_clean":"2024-05-27T09:37:37.352377+0000","last_became_active":"2024-05-27T09:27:18.562455+0000","last_became_peered":"2024-05-27T09:27:18.562455+0000","last_unstale":"2024-05-27T09:37:37.352377+0000","last_undegraded":"2024-05-27T09:37:37.352377+0000","last_fullsized":"2024-05-27T09:37:37.352377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:31.423566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082016410000000008,"stat_sum":{"num_bytes":31948882,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":27,"num_read_kb":17799,"num_write":353,"num_write_kb":35479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"49'171","reported_seq":250,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.632383+0000","last_change":"2024-05-27T09:27:17.563172+0000","last_active":"2024-05-27T09:37:15.632383+0000","last_peered":"2024-05-27T09:37:15.632383+0000","last_clean":"2024-05-27T09:37:15.632383+0000","last_became_active":"2024-05-27T09:27:17.562941+0000","last_became_peered":"2024-05-27T09:27:17.562941+0000","last_unstale":"2024-05-27T09:37:15.632383+0000","last_undegraded":"2024-05-27T09:37:15.632383+0000","last_fullsized":"2024-05-27T09:37:15.632383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:51.742078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":171,"num_write_kb":4214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"50'340","reported_seq":2461,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.117188+0000","last_change":"2024-05-27T09:33:11.731891+0000","last_active":"2024-05-27T09:37:20.117188+0000","last_peered":"2024-05-27T09:37:20.117188+0000","last_clean":"2024-05-27T09:37:20.117188+0000","last_became_active":"2024-05-27T09:27:18.555280+0000","last_became_peered":"2024-05-27T09:27:18.555280+0000","last_unstale":"2024-05-27T09:37:20.117188+0000","last_undegraded":"2024-05-27T09:37:20.117188+0000","last_fullsized":"2024-05-27T09:37:20.117188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:31.147919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041574209999999997,"stat_sum":{"num_bytes":13440874,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":19,"num_read_kb":8208,"num_write":339,"num_write_kb":17394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"50'252","reported_seq":331,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:36.353976+0000","last_change":"2024-05-27T09:27:17.548266+0000","last_active":"2024-05-27T09:37:36.353976+0000","last_peered":"2024-05-27T09:37:36.353976+0000","last_clean":"2024-05-27T09:37:36.353976+0000","last_became_active":"2024-05-27T09:27:17.547938+0000","last_became_peered":"2024-05-27T09:27:17.547938+0000","last_unstale":"2024-05-27T09:37:36.353976+0000","last_undegraded":"2024-05-27T09:37:36.353976+0000","last_fullsized":"2024-05-27T09:37:36.353976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7112766,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":4100,"num_write":280,"num_write_kb":7097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"50'334","reported_seq":2452,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.163309+0000","last_change":"2024-05-27T09:33:11.625744+0000","last_active":"2024-05-27T09:37:20.163309+0000","last_peered":"2024-05-27T09:37:20.163309+0000","last_clean":"2024-05-27T09:37:20.163309+0000","last_became_active":"2024-05-27T09:27:18.563227+0000","last_became_peered":"2024-05-27T09:27:18.563227+0000","last_unstale":"2024-05-27T09:37:20.163309+0000","last_undegraded":"2024-05-27T09:37:20.163309+0000","last_fullsized":"2024-05-27T09:37:20.163309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:30:49.326058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010490250000000001,"stat_sum":{"num_bytes":10717337,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":15,"num_read_kb":2569,"num_write":334,"num_write_kb":14740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"50'342","reported_seq":422,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.542171+0000","last_change":"2024-05-27T09:27:17.562200+0000","last_active":"2024-05-27T09:37:20.542171+0000","last_peered":"2024-05-27T09:37:20.542171+0000","last_clean":"2024-05-27T09:37:20.542171+0000","last_became_active":"2024-05-27T09:27:17.561857+0000","last_became_peered":"2024-05-27T09:27:17.561857+0000","last_unstale":"2024-05-27T09:37:20.542171+0000","last_undegraded":"2024-05-27T09:37:20.542171+0000","last_fullsized":"2024-05-27T09:37:20.542171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:05.209528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":10,"num_read_kb":8200,"num_write":340,"num_write_kb":12481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"50'360","reported_seq":2475,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.372550+0000","last_change":"2024-05-27T09:33:11.455285+0000","last_active":"2024-05-27T09:37:37.372550+0000","last_peered":"2024-05-27T09:37:37.372550+0000","last_clean":"2024-05-27T09:37:37.372550+0000","last_became_active":"2024-05-27T09:27:18.572339+0000","last_became_peered":"2024-05-27T09:27:18.572339+0000","last_unstale":"2024-05-27T09:37:37.372550+0000","last_undegraded":"2024-05-27T09:37:37.372550+0000","last_fullsized":"2024-05-27T09:37:37.372550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:11:51.773780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011410574,"stat_sum":{"num_bytes":1288358,"num_objects":185,"num_object_clones":0,"num_object_copies":370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":185,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":355,"num_write_kb":9639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"50'330","reported_seq":2455,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.388768+0000","last_change":"2024-05-27T09:33:11.456154+0000","last_active":"2024-05-27T09:37:37.388768+0000","last_peered":"2024-05-27T09:37:37.388768+0000","last_clean":"2024-05-27T09:37:37.388768+0000","last_became_active":"2024-05-27T09:27:18.572307+0000","last_became_peered":"2024-05-27T09:27:18.572307+0000","last_unstale":"2024-05-27T09:37:37.388768+0000","last_undegraded":"2024-05-27T09:37:37.388768+0000","last_fullsized":"2024-05-27T09:37:37.388768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:38:38.928939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089418529999999996,"stat_sum":{"num_bytes":7551421,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":315,"num_write_kb":11635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"49'32","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.542484+0000","last_change":"2024-05-27T09:27:17.570634+0000","last_active":"2024-05-27T09:37:15.542484+0000","last_peered":"2024-05-27T09:37:15.542484+0000","last_clean":"2024-05-27T09:37:15.542484+0000","last_became_active":"2024-05-27T09:27:17.569866+0000","last_became_peered":"2024-05-27T09:27:17.569866+0000","last_unstale":"2024-05-27T09:37:15.542484+0000","last_undegraded":"2024-05-27T09:37:15.542484+0000","last_fullsized":"2024-05-27T09:37:15.542484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:21:38.268535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4099,"num_write":35,"num_write_kb":4131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"50'359","reported_seq":2476,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.380935+0000","last_change":"2024-05-27T09:33:11.737809+0000","last_active":"2024-05-27T09:37:37.380935+0000","last_peered":"2024-05-27T09:37:37.380935+0000","last_clean":"2024-05-27T09:37:37.380935+0000","last_became_active":"2024-05-27T09:27:18.549035+0000","last_became_peered":"2024-05-27T09:27:18.549035+0000","last_unstale":"2024-05-27T09:37:37.380935+0000","last_undegraded":"2024-05-27T09:37:37.380935+0000","last_fullsized":"2024-05-27T09:37:37.380935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:14:41.023261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0090041500000000007,"stat_sum":{"num_bytes":20118472,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":14,"num_read_kb":13,"num_write":350,"num_write_kb":27987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"49'143","reported_seq":226,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.543458+0000","last_change":"2024-05-27T09:27:17.568143+0000","last_active":"2024-05-27T09:37:15.543458+0000","last_peered":"2024-05-27T09:37:15.543458+0000","last_clean":"2024-05-27T09:37:15.543458+0000","last_became_active":"2024-05-27T09:27:17.567235+0000","last_became_peered":"2024-05-27T09:27:17.567235+0000","last_unstale":"2024-05-27T09:37:15.543458+0000","last_undegraded":"2024-05-27T09:37:15.543458+0000","last_fullsized":"2024-05-27T09:37:15.543458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.284270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":15,"num_read_kb":12312,"num_write":136,"num_write_kb":12381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"50'313","reported_seq":2437,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.234565+0000","last_change":"2024-05-27T09:33:11.732530+0000","last_active":"2024-05-27T09:37:20.234565+0000","last_peered":"2024-05-27T09:37:20.234565+0000","last_clean":"2024-05-27T09:37:20.234565+0000","last_became_active":"2024-05-27T09:27:18.562919+0000","last_became_peered":"2024-05-27T09:27:18.562919+0000","last_unstale":"2024-05-27T09:37:20.234565+0000","last_undegraded":"2024-05-27T09:37:20.234565+0000","last_fullsized":"2024-05-27T09:37:20.234565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:09:22.339045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0063913879999999996,"stat_sum":{"num_bytes":22598856,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":21,"num_read_kb":8211,"num_write":313,"num_write_kb":22222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"49'87","reported_seq":165,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.953251+0000","last_change":"2024-05-27T09:27:17.551193+0000","last_active":"2024-05-27T09:37:15.953251+0000","last_peered":"2024-05-27T09:37:15.953251+0000","last_clean":"2024-05-27T09:37:15.953251+0000","last_became_active":"2024-05-27T09:27:17.550849+0000","last_became_peered":"2024-05-27T09:27:17.550849+0000","last_unstale":"2024-05-27T09:37:15.953251+0000","last_undegraded":"2024-05-27T09:37:15.953251+0000","last_fullsized":"2024-05-27T09:37:15.953251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:51.742078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":81,"num_write_kb":4153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"50'330","reported_seq":2448,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.256184+0000","last_change":"2024-05-27T09:33:11.732054+0000","last_active":"2024-05-27T09:37:20.256184+0000","last_peered":"2024-05-27T09:37:20.256184+0000","last_clean":"2024-05-27T09:37:20.256184+0000","last_became_active":"2024-05-27T09:27:18.579574+0000","last_became_peered":"2024-05-27T09:27:18.579574+0000","last_unstale":"2024-05-27T09:37:20.256184+0000","last_undegraded":"2024-05-27T09:37:20.256184+0000","last_fullsized":"2024-05-27T09:37:20.256184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:56:05.185246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062209780000000003,"stat_sum":{"num_bytes":14492175,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":14,"num_read_kb":4108,"num_write":330,"num_write_kb":14321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"49'82","reported_seq":172,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.632721+0000","last_change":"2024-05-27T09:27:17.549711+0000","last_active":"2024-05-27T09:37:15.632721+0000","last_peered":"2024-05-27T09:37:15.632721+0000","last_clean":"2024-05-27T09:37:15.632721+0000","last_became_active":"2024-05-27T09:27:17.549291+0000","last_became_peered":"2024-05-27T09:27:17.549291+0000","last_unstale":"2024-05-27T09:37:15.632721+0000","last_undegraded":"2024-05-27T09:37:15.632721+0000","last_fullsized":"2024-05-27T09:37:15.632721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:16:22.057803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":41,"num_read_kb":4136,"num_write":85,"num_write_kb":8333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":952881208,"num_objects":10697,"num_object_clones":0,"num_object_copies":21394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10697,"num_whiteouts":0,"num_read":1250,"num_read_kb":378565,"num_write":21433,"num_write_kb":1379166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1734066176,"data_stored":1904222320,"data_compressed":168711384,"data_compressed_allocated":186425344,"data_compressed_original":412123136,"omap_allocated":0,"internal_metadata":0},"log_size":21735,"ondisk_log_size":21735,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":188007639,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":494,"num_read_kb":123738,"num_write":4412,"num_write_kb":189065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":105,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":83959808,"data_stored":376015278,"data_compressed":49745852,"data_compressed_allocated":62447616,"data_compressed_original":354656256,"omap_allocated":0,"internal_metadata":0},"log_size":4113,"ondisk_log_size":4113,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65078,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509569,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":363184,"kb_used_data":309800,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94008656,"statfs":{"total":96636764160,"available":96264863744,"internally_reserved":0,"allocated":317235200,"data_stored":412121967,"data_compressed":41350796,"data_compressed_allocated":47357952,"data_compressed_original":150941696,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509569,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":278116,"kb_used_data":233052,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94093724,"statfs":{"total":96636764160,"available":96351973376,"internally_reserved":0,"allocated":238645248,"data_stored":280794177,"data_compressed":23230909,"data_compressed_allocated":26537984,"data_compressed_original":76705792,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509569,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246488,"kb_used_data":201424,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94125352,"statfs":{"total":96636764160,"available":96384360448,"internally_reserved":0,"allocated":206258176,"data_stored":257021246,"data_compressed":23229218,"data_compressed_allocated":26480640,"data_compressed_original":83582976,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509569,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356864,"kb_used_data":285432,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94014976,"statfs":{"total":96636764160,"available":96271335424,"internally_reserved":0,"allocated":292282368,"data_stored":379742031,"data_compressed":39531620,"data_compressed_allocated":45047808,"data_compressed_original":141885440,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542272,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":157168,"kb_used_data":130152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94214672,"statfs":{"total":96636764160,"available":96475824128,"internally_reserved":0,"allocated":133275648,"data_stored":168154774,"data_compressed":18202841,"data_compressed_allocated":20492288,"data_compressed_original":60579840,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542272,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":196044,"kb_used_data":146820,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94175796,"statfs":{"total":96636764160,"available":96436015104,"internally_reserved":0,"allocated":150343680,"data_stored":185739612,"data_compressed":18837119,"data_compressed_allocated":21549056,"data_compressed_original":63336448,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542272,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":358772,"kb_used_data":305388,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94013068,"statfs":{"total":96636764160,"available":96269381632,"internally_reserved":0,"allocated":312717312,"data_stored":372347367,"data_compressed":31432531,"data_compressed_allocated":35454976,"data_compressed_original":103374848,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542272,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":221244,"kb_used_data":167860,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94150596,"statfs":{"total":96636764160,"available":96410210304,"internally_reserved":0,"allocated":171888640,"data_stored":227475258,"data_compressed":22698376,"data_compressed_allocated":26025984,"data_compressed_original":87445504,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13996032,"data_stored":67110923,"data_compressed":8258411,"data_compressed_allocated":10555392,"data_compressed_original":63688704,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7446528,"data_stored":37748896,"data_compressed":4390710,"data_compressed_allocated":5623808,"data_compressed_original":35934208,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7921664,"data_stored":33555138,"data_compressed":4517094,"data_compressed_allocated":5791744,"data_compressed_original":31436800,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19890176,"data_stored":83888049,"data_compressed":11332765,"data_compressed_allocated":14295040,"data_compressed_original":78307328,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12206080,"data_stored":50333888,"data_compressed":7575831,"data_compressed_allocated":9129984,"data_compressed_original":47296512,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9977856,"data_stored":45397082,"data_compressed":5653863,"data_compressed_allocated":7127040,"data_compressed_original":42569728,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7090176,"data_stored":32280280,"data_compressed":4471564,"data_compressed_allocated":5599232,"data_compressed_original":30801920,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5431296,"data_stored":25701022,"data_compressed":3545614,"data_compressed_allocated":4325376,"data_compressed_original":24621056,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":277721088,"data_stored":312372852,"data_compressed":31273209,"data_compressed_allocated":34492416,"data_compressed_original":78196736,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":198180864,"data_stored":218423585,"data_compressed":18805969,"data_compressed_allocated":20811776,"data_compressed_original":47075328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":230158336,"data_stored":246980642,"data_compressed":18713815,"data_compressed_allocated":20746240,"data_compressed_original":45268992,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":296779776,"data_stored":327975662,"data_compressed":30018031,"data_compressed_allocated":33062912,"data_compressed_original":72634368,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":159117312,"data_stored":176882973,"data_compressed":15122545,"data_compressed_allocated":16896000,"data_compressed_original":40148992,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":302125056,"data_stored":326145918,"data_compressed":25750581,"data_compressed_allocated":28291072,"data_compressed_original":60268544,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":142704640,"data_stored":153245333,"data_compressed":14370007,"data_compressed_allocated":15958016,"data_compressed_original":32571392,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":127279104,"data_stored":142195355,"data_compressed":14657227,"data_compressed_allocated":16166912,"data_compressed_original":35958784,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T09:37:42.804 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-27T09:37:42.804 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T09:37:43.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.119+0000 7fd233200640 1 Processor -- start 2024-05-27T09:37:43.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.119+0000 7fd233200640 1 -- start start 2024-05-27T09:37:43.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.121+0000 7fd233200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd22c0741f0 0x7fd22c0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:43.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.121+0000 7fd233200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd22c074840 0x7fd22c15ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:43.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.121+0000 7fd233200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd22c06fe50 con 0x7fd22c1547d0 2024-05-27T09:37:43.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.121+0000 7fd233200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fd22c0703d0 con 0x7fd22c074840 2024-05-27T09:37:43.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.121+0000 7fd233200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fd22c15b260 con 0x7fd22c0741f0 2024-05-27T09:37:43.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.122+0000 7fd231e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd22c0741f0 0x7fd22c0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:43.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.122+0000 7fd231e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd22c0741f0 0x7fd22c0c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:39326/0 (socket says 172.21.15.114:39326) 2024-05-27T09:37:43.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.122+0000 7fd231e00640 1 -- 172.21.15.114:0/1434702343 learned_addr learned my addr 172.21.15.114:0/1434702343 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:43.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.122+0000 7fd232800640 1 --2- 172.21.15.114:0/1434702343 >> v2:172.21.15.114:3300/0 conn(0x7fd22c074840 0x7fd22c15ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:43.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.122+0000 7fd231e00640 1 -- 172.21.15.114:0/1434702343 >> v2:172.21.15.114:3300/0 conn(0x7fd22c074840 msgr2=0x7fd22c15ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:43.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.122+0000 7fd231e00640 1 --2- 172.21.15.114:0/1434702343 >> v2:172.21.15.114:3300/0 conn(0x7fd22c074840 0x7fd22c15ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:43.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.122+0000 7fd230a00640 1 -- 172.21.15.114:0/1434702343 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2652956617 0 0) 0x7fd22c06fe50 con 0x7fd22c1547d0 2024-05-27T09:37:43.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.122+0000 7fd231e00640 1 -- 172.21.15.114:0/1434702343 >> v1:172.21.15.163:6789/0 conn(0x7fd22c1547d0 legacy=0x7fd22c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:43.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.122+0000 7fd231e00640 1 -- 172.21.15.114:0/1434702343 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22c0700f0 con 0x7fd22c0741f0 2024-05-27T09:37:43.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.122+0000 7fd232800640 1 --2- 172.21.15.114:0/1434702343 >> v2:172.21.15.114:3300/0 conn(0x7fd22c074840 0x7fd22c15ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:37:43.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.123+0000 7fd231e00640 1 --2- 172.21.15.114:0/1434702343 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd22c0741f0 0x7fd22c0c63a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd220009870 tx=0x7fd22002a310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=568ad1be8d767bef server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:43.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.124+0000 7fd230a00640 1 -- 172.21.15.114:0/1434702343 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220037070 con 0x7fd22c0741f0 2024-05-27T09:37:43.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.124+0000 7fd230a00640 1 -- 172.21.15.114:0/1434702343 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd220032640 con 0x7fd22c0741f0 2024-05-27T09:37:43.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.124+0000 7fd230a00640 1 -- 172.21.15.114:0/1434702343 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22002ae60 con 0x7fd22c0741f0 2024-05-27T09:37:43.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.124+0000 7fd233200640 1 -- 172.21.15.114:0/1434702343 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd22c0741f0 msgr2=0x7fd22c0c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:43.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.124+0000 7fd233200640 1 --2- 172.21.15.114:0/1434702343 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd22c0741f0 0x7fd22c0c63a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd220009870 tx=0x7fd22002a310 comp rx=0 tx=0).stop 2024-05-27T09:37:43.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.124+0000 7fd233200640 1 -- 172.21.15.114:0/1434702343 shutdown_connections 2024-05-27T09:37:43.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.124+0000 7fd233200640 1 --2- 172.21.15.114:0/1434702343 >> v2:172.21.15.114:3300/0 conn(0x7fd22c074840 0x7fd22c15ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:43.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.124+0000 7fd233200640 1 --2- 172.21.15.114:0/1434702343 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd22c0741f0 0x7fd22c0c63a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:43.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.124+0000 7fd233200640 1 -- 172.21.15.114:0/1434702343 >> 172.21.15.114:0/1434702343 conn(0x7fd22c0ca320 msgr2=0x7fd22c0ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:43.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.125+0000 7fd233200640 1 -- 172.21.15.114:0/1434702343 shutdown_connections 2024-05-27T09:37:43.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.125+0000 7fd233200640 1 -- 172.21.15.114:0/1434702343 wait complete. 2024-05-27T09:37:43.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.127+0000 7fd233200640 1 Processor -- start 2024-05-27T09:37:43.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.127+0000 7fd233200640 1 -- start start 2024-05-27T09:37:43.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.129+0000 7fd233200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd22c0741f0 0x7fd22c0cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:43.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.129+0000 7fd233200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd22c074840 0x7fd22c1590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:43.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.129+0000 7fd233200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd22c1547d0 0x7fd22c0c38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:43.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.129+0000 7fd233200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c073790 con 0x7fd22c1547d0 2024-05-27T09:37:43.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.129+0000 7fd233200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c0734b0 con 0x7fd22c074840 2024-05-27T09:37:43.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.129+0000 7fd233200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fd22c06f500 con 0x7fd22c0741f0 2024-05-27T09:37:43.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.129+0000 7fd231e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd22c0741f0 0x7fd22c0cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:43.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.129+0000 7fd231400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd22c074840 0x7fd22c1590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:43.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.129+0000 7fd231e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd22c0741f0 0x7fd22c0cbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:39330/0 (socket says 172.21.15.114:39330) 2024-05-27T09:37:43.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.129+0000 7fd231e00640 1 -- 172.21.15.114:0/1342764468 learned_addr learned my addr 172.21.15.114:0/1342764468 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:43.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.129+0000 7fd232800640 1 --2- 172.21.15.114:0/1342764468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd22c1547d0 0x7fd22c0c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:43.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.130+0000 7fd231e00640 1 -- 172.21.15.114:0/1342764468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd22c074840 msgr2=0x7fd22c1590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:43.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.130+0000 7fd231e00640 1 --2- 172.21.15.114:0/1342764468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd22c074840 0x7fd22c1590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:43.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.130+0000 7fd231e00640 1 -- 172.21.15.114:0/1342764468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd22c1547d0 msgr2=0x7fd22c0c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:43.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.130+0000 7fd231e00640 1 --2- 172.21.15.114:0/1342764468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd22c1547d0 0x7fd22c0c38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:43.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.130+0000 7fd231e00640 1 -- 172.21.15.114:0/1342764468 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22c159620 con 0x7fd22c0741f0 2024-05-27T09:37:43.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.130+0000 7fd231e00640 1 --2- 172.21.15.114:0/1342764468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd22c0741f0 0x7fd22c0cbe80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd220000c00 tx=0x7fd2200046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:43.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.130+0000 7fd226000640 1 -- 172.21.15.114:0/1342764468 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220041070 con 0x7fd22c0741f0 2024-05-27T09:37:43.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.130+0000 7fd226000640 1 -- 172.21.15.114:0/1342764468 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd22003c890 con 0x7fd22c0741f0 2024-05-27T09:37:43.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.130+0000 7fd233200640 1 -- 172.21.15.114:0/1342764468 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd22c0cc4a0 con 0x7fd22c0741f0 2024-05-27T09:37:43.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.130+0000 7fd226000640 1 -- 172.21.15.114:0/1342764468 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220037040 con 0x7fd22c0741f0 2024-05-27T09:37:43.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.130+0000 7fd233200640 1 -- 172.21.15.114:0/1342764468 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd22c158600 con 0x7fd22c0741f0 2024-05-27T09:37:43.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.132+0000 7fd233200640 1 -- 172.21.15.114:0/1342764468 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd22c06a380 con 0x7fd22c0741f0 2024-05-27T09:37:43.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.133+0000 7fd226000640 1 -- 172.21.15.114:0/1342764468 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd220044070 con 0x7fd22c0741f0 2024-05-27T09:37:43.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.134+0000 7fd226000640 1 --2- 172.21.15.114:0/1342764468 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd2100762d0 0x7fd2100787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:43.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.134+0000 7fd226000640 1 -- 172.21.15.114:0/1342764468 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7fd2200b80d0 con 0x7fd22c0741f0 2024-05-27T09:37:43.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.134+0000 7fd231400640 1 --2- 172.21.15.114:0/1342764468 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd2100762d0 0x7fd2100787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:43.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.135+0000 7fd231400640 1 --2- 172.21.15.114:0/1342764468 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd2100762d0 0x7fd2100787e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd22c15a270 tx=0x7fd21c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:43.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.139+0000 7fd226000640 1 -- 172.21.15.114:0/1342764468 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd22c06a380 con 0x7fd22c0741f0 2024-05-27T09:37:43.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.345+0000 7fd233200640 1 -- 172.21.15.114:0/1342764468 --> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd22c157cb0 con 0x7fd2100762d0 2024-05-27T09:37:43.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.353+0000 7fd226000640 1 -- 172.21.15.114:0/1342764468 <== mgr.4108 v2:172.21.15.163:6832/1122018204 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+256647 (secure 0 0 0) 0x7fd22c157cb0 con 0x7fd2100762d0 2024-05-27T09:37:43.357 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:37:43.358 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-05-27T09:37:43.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.358+0000 7fd233200640 1 -- 172.21.15.114:0/1342764468 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd2100762d0 msgr2=0x7fd2100787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:43.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.358+0000 7fd233200640 1 --2- 172.21.15.114:0/1342764468 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd2100762d0 0x7fd2100787e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd22c15a270 tx=0x7fd21c009040 comp rx=0 tx=0).stop 2024-05-27T09:37:43.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.358+0000 7fd233200640 1 -- 172.21.15.114:0/1342764468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd22c0741f0 msgr2=0x7fd22c0cbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:43.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.358+0000 7fd233200640 1 --2- 172.21.15.114:0/1342764468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd22c0741f0 0x7fd22c0cbe80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd220000c00 tx=0x7fd2200046c0 comp rx=0 tx=0).stop 2024-05-27T09:37:43.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.358+0000 7fd233200640 1 -- 172.21.15.114:0/1342764468 shutdown_connections 2024-05-27T09:37:43.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.358+0000 7fd233200640 1 --2- 172.21.15.114:0/1342764468 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd2100762d0 0x7fd2100787e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:43.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.358+0000 7fd233200640 1 --2- 172.21.15.114:0/1342764468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd22c1547d0 0x7fd22c0c38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:43.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.358+0000 7fd233200640 1 --2- 172.21.15.114:0/1342764468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd22c074840 0x7fd22c1590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:43.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.358+0000 7fd233200640 1 --2- 172.21.15.114:0/1342764468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd22c0741f0 0x7fd22c0cbe80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:43.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.358+0000 7fd233200640 1 -- 172.21.15.114:0/1342764468 >> 172.21.15.114:0/1342764468 conn(0x7fd22c0ca320 msgr2=0x7fd22c14eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:43.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.359+0000 7fd233200640 1 -- 172.21.15.114:0/1342764468 shutdown_connections 2024-05-27T09:37:43.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.359+0000 7fd233200640 1 -- 172.21.15.114:0/1342764468 wait complete. 2024-05-27T09:37:43.385 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-05-27T09:37:42.941010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1141479637,"num_objects":10877,"num_object_clones":0,"num_object_copies":21754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10877,"num_whiteouts":0,"num_read":1873,"num_read_kb":502425,"num_write":25954,"num_write_kb":1569863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":105,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26152,"ondisk_log_size":26152,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2200120,"kb_used_data":1779960,"kb_used_omap":12,"kb_used_meta":420083,"kb_avail":752774600,"statfs":{"total":773094113280,"available":770841190400,"internally_reserved":0,"allocated":1822679040,"data_stored":2283486072,"data_compressed":218522314,"data_compressed_allocated":248963072,"data_compressed_original":767926272,"omap_allocated":12750,"internal_metadata":430165554},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1943963,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":67,"num_write":174,"num_write_kb":1988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003384"},"pg_stats":[{"pgid":"3.3f","version":"50'328","reported_seq":2448,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.380193+0000","last_change":"2024-05-27T09:33:11.447418+0000","last_active":"2024-05-27T09:37:37.380193+0000","last_peered":"2024-05-27T09:37:37.380193+0000","last_clean":"2024-05-27T09:37:37.380193+0000","last_became_active":"2024-05-27T09:27:18.585169+0000","last_became_peered":"2024-05-27T09:27:18.585169+0000","last_unstale":"2024-05-27T09:37:37.380193+0000","last_undegraded":"2024-05-27T09:37:37.380193+0000","last_fullsized":"2024-05-27T09:37:37.380193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:30:18.926000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01102325,"stat_sum":{"num_bytes":23698106,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":17,"num_read_kb":4111,"num_write":320,"num_write_kb":27403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"50'317","reported_seq":2436,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.351776+0000","last_change":"2024-05-27T09:33:11.731804+0000","last_active":"2024-05-27T09:37:37.351776+0000","last_peered":"2024-05-27T09:37:37.351776+0000","last_clean":"2024-05-27T09:37:37.351776+0000","last_became_active":"2024-05-27T09:27:18.566269+0000","last_became_peered":"2024-05-27T09:27:18.566269+0000","last_unstale":"2024-05-27T09:37:37.351776+0000","last_undegraded":"2024-05-27T09:37:37.351776+0000","last_fullsized":"2024-05-27T09:37:37.351776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:30:51.472078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061074720000000001,"stat_sum":{"num_bytes":5460707,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":313,"num_write_kb":13693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"50'349","reported_seq":2476,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.345788+0000","last_change":"2024-05-27T09:33:11.731326+0000","last_active":"2024-05-27T09:37:37.345788+0000","last_peered":"2024-05-27T09:37:37.345788+0000","last_clean":"2024-05-27T09:37:37.345788+0000","last_became_active":"2024-05-27T09:27:18.560620+0000","last_became_peered":"2024-05-27T09:27:18.560620+0000","last_unstale":"2024-05-27T09:37:37.345788+0000","last_undegraded":"2024-05-27T09:37:37.345788+0000","last_fullsized":"2024-05-27T09:37:37.345788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:02:04.553273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005599354,"stat_sum":{"num_bytes":19266359,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":24,"num_read_kb":17715,"num_write":342,"num_write_kb":39437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"50'307","reported_seq":2421,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.378615+0000","last_change":"2024-05-27T09:33:11.450825+0000","last_active":"2024-05-27T09:37:37.378615+0000","last_peered":"2024-05-27T09:37:37.378615+0000","last_clean":"2024-05-27T09:37:37.378615+0000","last_became_active":"2024-05-27T09:27:18.569481+0000","last_became_peered":"2024-05-27T09:27:18.569481+0000","last_unstale":"2024-05-27T09:37:37.378615+0000","last_undegraded":"2024-05-27T09:37:37.378615+0000","last_fullsized":"2024-05-27T09:37:37.378615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:19:26.159843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007726882,"stat_sum":{"num_bytes":6546535,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":298,"num_write_kb":6559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"50'375","reported_seq":2496,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.373663+0000","last_change":"2024-05-27T09:33:11.440268+0000","last_active":"2024-05-27T09:37:37.373663+0000","last_peered":"2024-05-27T09:37:37.373663+0000","last_clean":"2024-05-27T09:37:37.373663+0000","last_became_active":"2024-05-27T09:27:18.567795+0000","last_became_peered":"2024-05-27T09:27:18.567795+0000","last_unstale":"2024-05-27T09:37:37.373663+0000","last_undegraded":"2024-05-27T09:37:37.373663+0000","last_fullsized":"2024-05-27T09:37:37.373663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:31.321140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049759740000000002,"stat_sum":{"num_bytes":10629514,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":371,"num_write_kb":12093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"50'350","reported_seq":2475,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.371402+0000","last_change":"2024-05-27T09:33:11.445483+0000","last_active":"2024-05-27T09:37:37.371402+0000","last_peered":"2024-05-27T09:37:37.371402+0000","last_clean":"2024-05-27T09:37:37.371402+0000","last_became_active":"2024-05-27T09:27:18.573576+0000","last_became_peered":"2024-05-27T09:27:18.573576+0000","last_unstale":"2024-05-27T09:37:37.371402+0000","last_undegraded":"2024-05-27T09:37:37.371402+0000","last_fullsized":"2024-05-27T09:37:37.371402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:45:04.738799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074740400000000004,"stat_sum":{"num_bytes":30579450,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":22,"num_read_kb":8211,"num_write":344,"num_write_kb":38238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"50'360","reported_seq":2481,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.370720+0000","last_change":"2024-05-27T09:33:11.448897+0000","last_active":"2024-05-27T09:37:37.370720+0000","last_peered":"2024-05-27T09:37:37.370720+0000","last_clean":"2024-05-27T09:37:37.370720+0000","last_became_active":"2024-05-27T09:27:18.542593+0000","last_became_peered":"2024-05-27T09:27:18.542593+0000","last_unstale":"2024-05-27T09:37:37.370720+0000","last_undegraded":"2024-05-27T09:37:37.370720+0000","last_fullsized":"2024-05-27T09:37:37.370720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:11:49.487143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014070156,"stat_sum":{"num_bytes":5299988,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":18,"num_read_kb":4113,"num_write":355,"num_write_kb":9461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"50'329","reported_seq":2445,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.384663+0000","last_change":"2024-05-27T09:33:11.732725+0000","last_active":"2024-05-27T09:37:37.384663+0000","last_peered":"2024-05-27T09:37:37.384663+0000","last_clean":"2024-05-27T09:37:37.384663+0000","last_became_active":"2024-05-27T09:27:18.561788+0000","last_became_peered":"2024-05-27T09:27:18.561788+0000","last_unstale":"2024-05-27T09:37:37.384663+0000","last_undegraded":"2024-05-27T09:37:37.384663+0000","last_fullsized":"2024-05-27T09:37:37.384663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:51:29.653992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065132080000000004,"stat_sum":{"num_bytes":27492830,"num_objects":191,"num_object_clones":0,"num_object_copies":382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":191,"num_whiteouts":0,"num_read":13,"num_read_kb":4107,"num_write":319,"num_write_kb":27008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"50'338","reported_seq":2463,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.369696+0000","last_change":"2024-05-27T09:33:11.739498+0000","last_active":"2024-05-27T09:37:37.369696+0000","last_peered":"2024-05-27T09:37:37.369696+0000","last_clean":"2024-05-27T09:37:37.369696+0000","last_became_active":"2024-05-27T09:27:18.582041+0000","last_became_peered":"2024-05-27T09:27:18.582041+0000","last_unstale":"2024-05-27T09:37:37.369696+0000","last_undegraded":"2024-05-27T09:37:37.369696+0000","last_fullsized":"2024-05-27T09:37:37.369696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:29:10.337097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0083382379999999996,"stat_sum":{"num_bytes":9733898,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":22,"num_read_kb":8210,"num_write":328,"num_write_kb":30110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"50'321","reported_seq":2441,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.392171+0000","last_change":"2024-05-27T09:33:11.446829+0000","last_active":"2024-05-27T09:37:37.392171+0000","last_peered":"2024-05-27T09:37:37.392171+0000","last_clean":"2024-05-27T09:37:37.392171+0000","last_became_active":"2024-05-27T09:27:18.574409+0000","last_became_peered":"2024-05-27T09:27:18.574409+0000","last_unstale":"2024-05-27T09:37:37.392171+0000","last_undegraded":"2024-05-27T09:37:37.392171+0000","last_fullsized":"2024-05-27T09:37:37.392171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:41:37.603718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010493662000000001,"stat_sum":{"num_bytes":13833965,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":17,"num_read_kb":4112,"num_write":306,"num_write_kb":13622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"50'319","reported_seq":2444,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.384747+0000","last_change":"2024-05-27T09:33:11.443445+0000","last_active":"2024-05-27T09:37:37.384747+0000","last_peered":"2024-05-27T09:37:37.384747+0000","last_clean":"2024-05-27T09:37:37.384747+0000","last_became_active":"2024-05-27T09:27:18.578679+0000","last_became_peered":"2024-05-27T09:27:18.578679+0000","last_unstale":"2024-05-27T09:37:37.384747+0000","last_undegraded":"2024-05-27T09:37:37.384747+0000","last_fullsized":"2024-05-27T09:37:37.384747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:33:46.683180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080365679999999991,"stat_sum":{"num_bytes":26002241,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":22,"num_read_kb":7043,"num_write":312,"num_write_kb":25555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"50'301","reported_seq":2409,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.358299+0000","last_change":"2024-05-27T09:33:11.625670+0000","last_active":"2024-05-27T09:37:37.358299+0000","last_peered":"2024-05-27T09:37:37.358299+0000","last_clean":"2024-05-27T09:37:37.358299+0000","last_became_active":"2024-05-27T09:28:06.207562+0000","last_became_peered":"2024-05-27T09:28:06.207562+0000","last_unstale":"2024-05-27T09:37:37.358299+0000","last_undegraded":"2024-05-27T09:37:37.358299+0000","last_fullsized":"2024-05-27T09:37:37.358299+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:52:32.773329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001018333,"stat_sum":{"num_bytes":11019660,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":12,"num_read_kb":8201,"num_write":293,"num_write_kb":23198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"50'308","reported_seq":2432,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.385407+0000","last_change":"2024-05-27T09:33:11.728591+0000","last_active":"2024-05-27T09:37:37.385407+0000","last_peered":"2024-05-27T09:37:37.385407+0000","last_clean":"2024-05-27T09:37:37.385407+0000","last_became_active":"2024-05-27T09:27:18.561493+0000","last_became_peered":"2024-05-27T09:27:18.561493+0000","last_unstale":"2024-05-27T09:37:37.385407+0000","last_undegraded":"2024-05-27T09:37:37.385407+0000","last_fullsized":"2024-05-27T09:37:37.385407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:22.877083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024650470000000002,"stat_sum":{"num_bytes":14891482,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":21,"num_read_kb":4116,"num_write":298,"num_write_kb":31046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"50'342","reported_seq":2463,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.356879+0000","last_change":"2024-05-27T09:33:11.727658+0000","last_active":"2024-05-27T09:37:37.356879+0000","last_peered":"2024-05-27T09:37:37.356879+0000","last_clean":"2024-05-27T09:37:37.356879+0000","last_became_active":"2024-05-27T09:27:18.562843+0000","last_became_peered":"2024-05-27T09:27:18.562843+0000","last_unstale":"2024-05-27T09:37:37.356879+0000","last_undegraded":"2024-05-27T09:37:37.356879+0000","last_fullsized":"2024-05-27T09:37:37.356879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:31.147919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025802960000000002,"stat_sum":{"num_bytes":6434369,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":335,"num_write_kb":10518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"50'364","reported_seq":2489,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.384329+0000","last_change":"2024-05-27T09:33:11.729192+0000","last_active":"2024-05-27T09:37:37.384329+0000","last_peered":"2024-05-27T09:37:37.384329+0000","last_clean":"2024-05-27T09:37:37.384329+0000","last_became_active":"2024-05-27T09:27:18.566108+0000","last_became_peered":"2024-05-27T09:27:18.566108+0000","last_unstale":"2024-05-27T09:37:37.384329+0000","last_undegraded":"2024-05-27T09:37:37.384329+0000","last_fullsized":"2024-05-27T09:37:37.384329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:39:23.102944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041189030000000001,"stat_sum":{"num_bytes":20283045,"num_objects":201,"num_object_clones":0,"num_object_copies":402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":201,"num_whiteouts":0,"num_read":22,"num_read_kb":6627,"num_write":357,"num_write_kb":19995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"50'369","reported_seq":2493,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.386640+0000","last_change":"2024-05-27T09:33:11.457079+0000","last_active":"2024-05-27T09:37:37.386640+0000","last_peered":"2024-05-27T09:37:37.386640+0000","last_clean":"2024-05-27T09:37:37.386640+0000","last_became_active":"2024-05-27T09:27:18.581719+0000","last_became_peered":"2024-05-27T09:27:18.581719+0000","last_unstale":"2024-05-27T09:37:37.386640+0000","last_undegraded":"2024-05-27T09:37:37.386640+0000","last_fullsized":"2024-05-27T09:37:37.386640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:59:21.546098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.008579415,"stat_sum":{"num_bytes":24362280,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":22,"num_read_kb":5287,"num_write":359,"num_write_kb":31349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"50'366","reported_seq":2491,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.387571+0000","last_change":"2024-05-27T09:33:11.458057+0000","last_active":"2024-05-27T09:37:37.387571+0000","last_peered":"2024-05-27T09:37:37.387571+0000","last_clean":"2024-05-27T09:37:37.387571+0000","last_became_active":"2024-05-27T09:27:18.577193+0000","last_became_peered":"2024-05-27T09:27:18.577193+0000","last_unstale":"2024-05-27T09:37:37.387571+0000","last_undegraded":"2024-05-27T09:37:37.387571+0000","last_fullsized":"2024-05-27T09:37:37.387571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:41:12.903002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079143630000000006,"stat_sum":{"num_bytes":23132412,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":23,"num_read_kb":8093,"num_write":357,"num_write_kb":30967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"50'307","reported_seq":2438,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.372050+0000","last_change":"2024-05-27T09:33:11.626051+0000","last_active":"2024-05-27T09:37:37.372050+0000","last_peered":"2024-05-27T09:37:37.372050+0000","last_clean":"2024-05-27T09:37:37.372050+0000","last_became_active":"2024-05-27T09:27:18.570712+0000","last_became_peered":"2024-05-27T09:27:18.570712+0000","last_unstale":"2024-05-27T09:37:37.372050+0000","last_undegraded":"2024-05-27T09:37:37.372050+0000","last_fullsized":"2024-05-27T09:37:37.372050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:32:58.266174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767629999999999,"stat_sum":{"num_bytes":13973992,"num_objects":138,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":0,"num_read":28,"num_read_kb":8218,"num_write":296,"num_write_kb":21985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"50'323","reported_seq":2443,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.367564+0000","last_change":"2024-05-27T09:33:11.456909+0000","last_active":"2024-05-27T09:37:37.367564+0000","last_peered":"2024-05-27T09:37:37.367564+0000","last_clean":"2024-05-27T09:37:37.367564+0000","last_became_active":"2024-05-27T09:27:18.564643+0000","last_became_peered":"2024-05-27T09:27:18.564643+0000","last_unstale":"2024-05-27T09:37:37.367564+0000","last_undegraded":"2024-05-27T09:37:37.367564+0000","last_fullsized":"2024-05-27T09:37:37.367564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:14:07.864139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0091687629999999999,"stat_sum":{"num_bytes":2873482,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":18,"num_read_kb":4113,"num_write":315,"num_write_kb":11167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"50'345","reported_seq":2469,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.376407+0000","last_change":"2024-05-27T09:33:11.452205+0000","last_active":"2024-05-27T09:37:37.376407+0000","last_peered":"2024-05-27T09:37:37.376407+0000","last_clean":"2024-05-27T09:37:37.376407+0000","last_became_active":"2024-05-27T09:27:18.543777+0000","last_became_peered":"2024-05-27T09:27:18.543777+0000","last_unstale":"2024-05-27T09:37:37.376407+0000","last_undegraded":"2024-05-27T09:37:37.376407+0000","last_fullsized":"2024-05-27T09:37:37.376407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:11:45.868921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012534072,"stat_sum":{"num_bytes":9859903,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":21,"num_read_kb":4116,"num_write":337,"num_write_kb":17999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"50'337","reported_seq":2462,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.367546+0000","last_change":"2024-05-27T09:33:11.732683+0000","last_active":"2024-05-27T09:37:37.367546+0000","last_peered":"2024-05-27T09:37:37.367546+0000","last_clean":"2024-05-27T09:37:37.367546+0000","last_became_active":"2024-05-27T09:27:18.570948+0000","last_became_peered":"2024-05-27T09:27:18.570948+0000","last_unstale":"2024-05-27T09:37:37.367546+0000","last_undegraded":"2024-05-27T09:37:37.367546+0000","last_fullsized":"2024-05-27T09:37:37.367546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:03:03.895209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040840829999999996,"stat_sum":{"num_bytes":5923849,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":325,"num_write_kb":5958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"50'343","reported_seq":2466,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.383214+0000","last_change":"2024-05-27T09:33:11.737682+0000","last_active":"2024-05-27T09:37:37.383214+0000","last_peered":"2024-05-27T09:37:37.383214+0000","last_clean":"2024-05-27T09:37:37.383214+0000","last_became_active":"2024-05-27T09:27:18.543921+0000","last_became_peered":"2024-05-27T09:27:18.543921+0000","last_unstale":"2024-05-27T09:37:37.383214+0000","last_undegraded":"2024-05-27T09:37:37.383214+0000","last_fullsized":"2024-05-27T09:37:37.383214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:47:03.620703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0094259180000000001,"stat_sum":{"num_bytes":14450069,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":20,"num_read_kb":8209,"num_write":332,"num_write_kb":18385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"50'337","reported_seq":2471,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.387356+0000","last_change":"2024-05-27T09:33:11.731031+0000","last_active":"2024-05-27T09:37:37.387356+0000","last_peered":"2024-05-27T09:37:37.387356+0000","last_clean":"2024-05-27T09:37:37.387356+0000","last_became_active":"2024-05-27T09:27:18.575115+0000","last_became_peered":"2024-05-27T09:27:18.575115+0000","last_unstale":"2024-05-27T09:37:37.387356+0000","last_undegraded":"2024-05-27T09:37:37.387356+0000","last_fullsized":"2024-05-27T09:37:37.387356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:56:57.284871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054063330000000001,"stat_sum":{"num_bytes":18656408,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":31,"num_read_kb":19450,"num_write":329,"num_write_kb":38861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"50'353","reported_seq":2474,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.367488+0000","last_change":"2024-05-27T09:33:11.451714+0000","last_active":"2024-05-27T09:37:37.367488+0000","last_peered":"2024-05-27T09:37:37.367488+0000","last_clean":"2024-05-27T09:37:37.367488+0000","last_became_active":"2024-05-27T09:27:18.563991+0000","last_became_peered":"2024-05-27T09:27:18.563991+0000","last_unstale":"2024-05-27T09:37:37.367488+0000","last_undegraded":"2024-05-27T09:37:37.367488+0000","last_fullsized":"2024-05-27T09:37:37.367488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:41:37.963962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009841753,"stat_sum":{"num_bytes":13963475,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":19,"num_read_kb":8208,"num_write":347,"num_write_kb":22013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"50'332","reported_seq":2456,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.371538+0000","last_change":"2024-05-27T09:33:11.732272+0000","last_active":"2024-05-27T09:37:37.371538+0000","last_peered":"2024-05-27T09:37:37.371538+0000","last_clean":"2024-05-27T09:37:37.371538+0000","last_became_active":"2024-05-27T09:27:18.562803+0000","last_became_peered":"2024-05-27T09:27:18.562803+0000","last_unstale":"2024-05-27T09:37:37.371538+0000","last_undegraded":"2024-05-27T09:37:37.371538+0000","last_fullsized":"2024-05-27T09:37:37.371538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:09:56.174365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067443290000000003,"stat_sum":{"num_bytes":23071782,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":21,"num_read_kb":4115,"num_write":328,"num_write_kb":30886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"50'384","reported_seq":2507,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.393993+0000","last_change":"2024-05-27T09:33:11.453888+0000","last_active":"2024-05-27T09:37:37.393993+0000","last_peered":"2024-05-27T09:37:37.393993+0000","last_clean":"2024-05-27T09:37:37.393993+0000","last_became_active":"2024-05-27T09:27:18.590083+0000","last_became_peered":"2024-05-27T09:27:18.590083+0000","last_unstale":"2024-05-27T09:37:37.393993+0000","last_undegraded":"2024-05-27T09:37:37.393993+0000","last_fullsized":"2024-05-27T09:37:37.393993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:00.006136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.008181008,"stat_sum":{"num_bytes":14547405,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":20,"num_read_kb":4115,"num_write":369,"num_write_kb":14397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"50'332","reported_seq":2451,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.354888+0000","last_change":"2024-05-27T09:33:11.728099+0000","last_active":"2024-05-27T09:37:37.354888+0000","last_peered":"2024-05-27T09:37:37.354888+0000","last_clean":"2024-05-27T09:37:37.354888+0000","last_became_active":"2024-05-27T09:27:18.557953+0000","last_became_peered":"2024-05-27T09:27:18.557953+0000","last_unstale":"2024-05-27T09:37:37.354888+0000","last_undegraded":"2024-05-27T09:37:37.354888+0000","last_fullsized":"2024-05-27T09:37:37.354888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:03:03.895209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027117650000000001,"stat_sum":{"num_bytes":18627167,"num_objects":187,"num_object_clones":0,"num_object_copies":374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":187,"num_whiteouts":0,"num_read":16,"num_read_kb":4110,"num_write":325,"num_write_kb":26562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"50'307","reported_seq":2427,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.379634+0000","last_change":"2024-05-27T09:33:11.729430+0000","last_active":"2024-05-27T09:37:37.379634+0000","last_peered":"2024-05-27T09:37:37.379634+0000","last_clean":"2024-05-27T09:37:37.379634+0000","last_became_active":"2024-05-27T09:27:18.562839+0000","last_became_peered":"2024-05-27T09:27:18.562839+0000","last_unstale":"2024-05-27T09:37:37.379634+0000","last_undegraded":"2024-05-27T09:37:37.379634+0000","last_fullsized":"2024-05-27T09:37:37.379634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:21:17.734561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032647399999999999,"stat_sum":{"num_bytes":14533754,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":17,"num_read_kb":11479,"num_write":303,"num_write_kb":21724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"50'351","reported_seq":2474,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.344118+0000","last_change":"2024-05-27T09:33:11.730679+0000","last_active":"2024-05-27T09:37:37.344118+0000","last_peered":"2024-05-27T09:37:37.344118+0000","last_clean":"2024-05-27T09:37:37.344118+0000","last_became_active":"2024-05-27T09:27:18.561475+0000","last_became_peered":"2024-05-27T09:27:18.561475+0000","last_unstale":"2024-05-27T09:37:37.344118+0000","last_undegraded":"2024-05-27T09:37:37.344118+0000","last_fullsized":"2024-05-27T09:37:37.344118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:32:06.826783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053708779999999999,"stat_sum":{"num_bytes":11382190,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":20,"num_read_kb":4114,"num_write":347,"num_write_kb":19366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"50'377","reported_seq":2507,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.357466+0000","last_change":"2024-05-27T09:33:11.454566+0000","last_active":"2024-05-27T09:37:37.357466+0000","last_peered":"2024-05-27T09:37:37.357466+0000","last_clean":"2024-05-27T09:37:37.357466+0000","last_became_active":"2024-05-27T09:27:18.563103+0000","last_became_peered":"2024-05-27T09:27:18.563103+0000","last_unstale":"2024-05-27T09:37:37.357466+0000","last_undegraded":"2024-05-27T09:37:37.357466+0000","last_fullsized":"2024-05-27T09:37:37.357466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:25.776404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011066041,"stat_sum":{"num_bytes":30187091,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":28,"num_read_kb":16405,"num_write":369,"num_write_kb":50152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"50'401","reported_seq":2523,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.358464+0000","last_change":"2024-05-27T09:33:11.625969+0000","last_active":"2024-05-27T09:37:37.358464+0000","last_peered":"2024-05-27T09:37:37.358464+0000","last_clean":"2024-05-27T09:37:37.358464+0000","last_became_active":"2024-05-27T09:27:18.560244+0000","last_became_peered":"2024-05-27T09:27:18.560244+0000","last_unstale":"2024-05-27T09:37:37.358464+0000","last_undegraded":"2024-05-27T09:37:37.358464+0000","last_fullsized":"2024-05-27T09:37:37.358464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:14:56.979490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012131959999999999,"stat_sum":{"num_bytes":8541274,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":19,"num_read_kb":2631,"num_write":389,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"50'363","reported_seq":2493,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.377327+0000","last_change":"2024-05-27T09:33:11.447687+0000","last_active":"2024-05-27T09:37:37.377327+0000","last_peered":"2024-05-27T09:37:37.377327+0000","last_clean":"2024-05-27T09:37:37.377327+0000","last_became_active":"2024-05-27T09:27:18.567845+0000","last_became_peered":"2024-05-27T09:27:18.567845+0000","last_unstale":"2024-05-27T09:37:37.377327+0000","last_undegraded":"2024-05-27T09:37:37.377327+0000","last_fullsized":"2024-05-27T09:37:37.377327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:48:36.494196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082936180000000009,"stat_sum":{"num_bytes":1444930,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":352,"num_write_kb":9783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"50'336","reported_seq":2458,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.370918+0000","last_change":"2024-05-27T09:33:11.738447+0000","last_active":"2024-05-27T09:37:37.370918+0000","last_peered":"2024-05-27T09:37:37.370918+0000","last_clean":"2024-05-27T09:37:37.370918+0000","last_became_active":"2024-05-27T09:27:18.556156+0000","last_became_peered":"2024-05-27T09:27:18.556156+0000","last_unstale":"2024-05-27T09:37:37.370918+0000","last_undegraded":"2024-05-27T09:37:37.370918+0000","last_fullsized":"2024-05-27T09:37:37.370918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:25:11.600058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0094378269999999993,"stat_sum":{"num_bytes":19241078,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":19,"num_read_kb":8208,"num_write":329,"num_write_kb":23062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"50'74","reported_seq":157,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:36.346406+0000","last_change":"2024-05-27T09:27:17.548038+0000","last_active":"2024-05-27T09:37:36.346406+0000","last_peered":"2024-05-27T09:37:36.346406+0000","last_clean":"2024-05-27T09:37:36.346406+0000","last_became_active":"2024-05-27T09:27:17.547653+0000","last_became_peered":"2024-05-27T09:27:17.547653+0000","last_unstale":"2024-05-27T09:37:36.346406+0000","last_undegraded":"2024-05-27T09:37:36.346406+0000","last_fullsized":"2024-05-27T09:37:36.346406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195553,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":4100,"num_write":72,"num_write_kb":4221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"50'328","reported_seq":2449,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.384098+0000","last_change":"2024-05-27T09:33:11.733348+0000","last_active":"2024-05-27T09:37:37.384098+0000","last_peered":"2024-05-27T09:37:37.384098+0000","last_clean":"2024-05-27T09:37:37.384098+0000","last_became_active":"2024-05-27T09:27:18.542715+0000","last_became_peered":"2024-05-27T09:27:18.542715+0000","last_unstale":"2024-05-27T09:37:37.384098+0000","last_undegraded":"2024-05-27T09:37:37.384098+0000","last_fullsized":"2024-05-27T09:37:37.384098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:45:47.000935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004594323,"stat_sum":{"num_bytes":10080804,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":19,"num_read_kb":4113,"num_write":319,"num_write_kb":14102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"49'299","reported_seq":384,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.547628+0000","last_change":"2024-05-27T09:27:17.570014+0000","last_active":"2024-05-27T09:37:15.547628+0000","last_peered":"2024-05-27T09:37:15.547628+0000","last_clean":"2024-05-27T09:37:15.547628+0000","last_became_active":"2024-05-27T09:27:17.569673+0000","last_became_peered":"2024-05-27T09:27:17.569673+0000","last_unstale":"2024-05-27T09:37:15.547628+0000","last_undegraded":"2024-05-27T09:37:15.547628+0000","last_fullsized":"2024-05-27T09:37:15.547628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:26.210041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":22,"num_read_kb":12200,"num_write":300,"num_write_kb":12509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"49'46","reported_seq":129,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.952341+0000","last_change":"2024-05-27T09:27:17.565489+0000","last_active":"2024-05-27T09:37:15.952341+0000","last_peered":"2024-05-27T09:37:15.952341+0000","last_clean":"2024-05-27T09:37:15.952341+0000","last_became_active":"2024-05-27T09:27:17.565137+0000","last_became_peered":"2024-05-27T09:27:17.565137+0000","last_unstale":"2024-05-27T09:37:15.952341+0000","last_undegraded":"2024-05-27T09:37:15.952341+0000","last_fullsized":"2024-05-27T09:37:15.952341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":19,"num_read_kb":4114,"num_write":50,"num_write_kb":4167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"50'397","reported_seq":2523,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.353248+0000","last_change":"2024-05-27T09:33:11.733228+0000","last_active":"2024-05-27T09:37:37.353248+0000","last_peered":"2024-05-27T09:37:37.353248+0000","last_clean":"2024-05-27T09:37:37.353248+0000","last_became_active":"2024-05-27T09:27:18.564115+0000","last_became_peered":"2024-05-27T09:27:18.564115+0000","last_unstale":"2024-05-27T09:37:37.353248+0000","last_undegraded":"2024-05-27T09:37:37.353248+0000","last_fullsized":"2024-05-27T09:37:37.353248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:06:44.772830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066511199999999999,"stat_sum":{"num_bytes":10503109,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":23,"num_read_kb":4118,"num_write":389,"num_write_kb":18659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"50'294","reported_seq":371,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.547032+0000","last_change":"2024-05-27T09:27:17.561204+0000","last_active":"2024-05-27T09:37:15.547032+0000","last_peered":"2024-05-27T09:37:15.547032+0000","last_clean":"2024-05-27T09:37:15.547032+0000","last_became_active":"2024-05-27T09:27:17.560149+0000","last_became_peered":"2024-05-27T09:27:17.560149+0000","last_unstale":"2024-05-27T09:37:15.547032+0000","last_undegraded":"2024-05-27T09:37:15.547032+0000","last_fullsized":"2024-05-27T09:37:15.547032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.284270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":291,"num_write_kb":12506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"50'344","reported_seq":2466,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.356787+0000","last_change":"2024-05-27T09:33:11.734046+0000","last_active":"2024-05-27T09:37:37.356787+0000","last_peered":"2024-05-27T09:37:37.356787+0000","last_clean":"2024-05-27T09:37:37.356787+0000","last_became_active":"2024-05-27T09:27:18.541946+0000","last_became_peered":"2024-05-27T09:27:18.541946+0000","last_unstale":"2024-05-27T09:37:37.356787+0000","last_undegraded":"2024-05-27T09:37:37.356787+0000","last_fullsized":"2024-05-27T09:37:37.356787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:30.087688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030229530000000001,"stat_sum":{"num_bytes":19296821,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":20,"num_read_kb":12305,"num_write":330,"num_write_kb":31302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"50'9","reported_seq":91,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:41.372042+0000","last_change":"2024-05-27T09:27:17.561714+0000","last_active":"2024-05-27T09:37:41.372042+0000","last_peered":"2024-05-27T09:37:41.372042+0000","last_clean":"2024-05-27T09:37:41.372042+0000","last_became_active":"2024-05-27T09:27:17.561355+0000","last_became_peered":"2024-05-27T09:27:17.561355+0000","last_unstale":"2024-05-27T09:37:41.372042+0000","last_undegraded":"2024-05-27T09:37:41.372042+0000","last_fullsized":"2024-05-27T09:37:41.372042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:07:02.481451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":8,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"50'354","reported_seq":2478,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.394166+0000","last_change":"2024-05-27T09:33:11.454088+0000","last_active":"2024-05-27T09:37:37.394166+0000","last_peered":"2024-05-27T09:37:37.394166+0000","last_clean":"2024-05-27T09:37:37.394166+0000","last_became_active":"2024-05-27T09:27:18.587921+0000","last_became_peered":"2024-05-27T09:27:18.587921+0000","last_unstale":"2024-05-27T09:37:37.394166+0000","last_undegraded":"2024-05-27T09:37:37.394166+0000","last_fullsized":"2024-05-27T09:37:37.394166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:17:20.248600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0078895319999999994,"stat_sum":{"num_bytes":10502055,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":21,"num_read_kb":4116,"num_write":345,"num_write_kb":10394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"49'65","reported_seq":146,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:41.375549+0000","last_change":"2024-05-27T09:27:17.565854+0000","last_active":"2024-05-27T09:37:41.375549+0000","last_peered":"2024-05-27T09:37:41.375549+0000","last_clean":"2024-05-27T09:37:41.375549+0000","last_became_active":"2024-05-27T09:27:17.565488+0000","last_became_peered":"2024-05-27T09:27:17.565488+0000","last_unstale":"2024-05-27T09:37:41.375549+0000","last_undegraded":"2024-05-27T09:37:41.375549+0000","last_fullsized":"2024-05-27T09:37:41.375549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:53.711508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":4108,"num_write":67,"num_write_kb":4232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"50'338","reported_seq":2464,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.366109+0000","last_change":"2024-05-27T09:33:11.733328+0000","last_active":"2024-05-27T09:37:37.366109+0000","last_peered":"2024-05-27T09:37:37.366109+0000","last_clean":"2024-05-27T09:37:37.366109+0000","last_became_active":"2024-05-27T09:27:18.585243+0000","last_became_peered":"2024-05-27T09:27:18.585243+0000","last_unstale":"2024-05-27T09:37:37.366109+0000","last_undegraded":"2024-05-27T09:37:37.366109+0000","last_fullsized":"2024-05-27T09:37:37.366109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:18:44.814432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067133569999999997,"stat_sum":{"num_bytes":18063712,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":23,"num_read_kb":8211,"num_write":328,"num_write_kb":30099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"50'204","reported_seq":284,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.529879+0000","last_change":"2024-05-27T09:27:17.548651+0000","last_active":"2024-05-27T09:37:20.529879+0000","last_peered":"2024-05-27T09:37:20.529879+0000","last_clean":"2024-05-27T09:37:20.529879+0000","last_became_active":"2024-05-27T09:27:17.548307+0000","last_became_peered":"2024-05-27T09:27:17.548307+0000","last_unstale":"2024-05-27T09:37:20.529879+0000","last_undegraded":"2024-05-27T09:37:20.529879+0000","last_fullsized":"2024-05-27T09:37:20.529879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:48.905422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":206,"num_write_kb":4295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"50'341","reported_seq":2460,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.387494+0000","last_change":"2024-05-27T09:33:11.455764+0000","last_active":"2024-05-27T09:37:37.387494+0000","last_peered":"2024-05-27T09:37:37.387494+0000","last_clean":"2024-05-27T09:37:37.387494+0000","last_became_active":"2024-05-27T09:27:18.581742+0000","last_became_peered":"2024-05-27T09:27:18.581742+0000","last_unstale":"2024-05-27T09:37:37.387494+0000","last_undegraded":"2024-05-27T09:37:37.387494+0000","last_fullsized":"2024-05-27T09:37:37.387494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:14:45.868801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089613100000000001,"stat_sum":{"num_bytes":20607307,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":17,"num_read_kb":10914,"num_write":328,"num_write_kb":24390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"50'245","reported_seq":330,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:41.377977+0000","last_change":"2024-05-27T09:27:17.556419+0000","last_active":"2024-05-27T09:37:41.377977+0000","last_peered":"2024-05-27T09:37:41.377977+0000","last_clean":"2024-05-27T09:37:41.377977+0000","last_became_active":"2024-05-27T09:27:17.556036+0000","last_became_peered":"2024-05-27T09:27:17.556036+0000","last_unstale":"2024-05-27T09:37:41.377977+0000","last_undegraded":"2024-05-27T09:37:41.377977+0000","last_fullsized":"2024-05-27T09:37:41.377977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:48:20.718085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":19,"num_read_kb":8209,"num_write":243,"num_write_kb":12459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"50'335","reported_seq":2457,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.383012+0000","last_change":"2024-05-27T09:33:11.449451+0000","last_active":"2024-05-27T09:37:37.383012+0000","last_peered":"2024-05-27T09:37:37.383012+0000","last_clean":"2024-05-27T09:37:37.383012+0000","last_became_active":"2024-05-27T09:27:18.565500+0000","last_became_peered":"2024-05-27T09:27:18.565500+0000","last_unstale":"2024-05-27T09:37:37.383012+0000","last_undegraded":"2024-05-27T09:37:37.383012+0000","last_fullsized":"2024-05-27T09:37:37.383012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:25:55.844796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011618516000000001,"stat_sum":{"num_bytes":10741446,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":19,"num_read_kb":4114,"num_write":329,"num_write_kb":14761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"49'17","reported_seq":107,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:41.378694+0000","last_change":"2024-05-27T09:27:17.572168+0000","last_active":"2024-05-27T09:37:41.378694+0000","last_peered":"2024-05-27T09:37:41.378694+0000","last_clean":"2024-05-27T09:37:41.378694+0000","last_became_active":"2024-05-27T09:27:17.571709+0000","last_became_peered":"2024-05-27T09:27:17.571709+0000","last_unstale":"2024-05-27T09:37:41.378694+0000","last_undegraded":"2024-05-27T09:37:41.378694+0000","last_fullsized":"2024-05-27T09:37:41.378694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":8230,"num_write":18,"num_write_kb":8221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"50'343","reported_seq":2472,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.365363+0000","last_change":"2024-05-27T09:33:11.728047+0000","last_active":"2024-05-27T09:37:37.365363+0000","last_peered":"2024-05-27T09:37:37.365363+0000","last_clean":"2024-05-27T09:37:37.365363+0000","last_became_active":"2024-05-27T09:27:18.564589+0000","last_became_peered":"2024-05-27T09:27:18.564589+0000","last_unstale":"2024-05-27T09:37:37.365363+0000","last_undegraded":"2024-05-27T09:37:37.365363+0000","last_fullsized":"2024-05-27T09:37:37.365363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:09:11.832557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028467779999999999,"stat_sum":{"num_bytes":14347466,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":26,"num_read_kb":16403,"num_write":332,"num_write_kb":30566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"50'8","reported_seq":87,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:41.373266+0000","last_change":"2024-05-27T09:27:17.561344+0000","last_active":"2024-05-27T09:37:41.373266+0000","last_peered":"2024-05-27T09:37:41.373266+0000","last_clean":"2024-05-27T09:37:41.373266+0000","last_became_active":"2024-05-27T09:27:17.554860+0000","last_became_peered":"2024-05-27T09:27:17.554860+0000","last_unstale":"2024-05-27T09:37:41.373266+0000","last_undegraded":"2024-05-27T09:37:41.373266+0000","last_fullsized":"2024-05-27T09:37:41.373266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:16.880101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":9,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"50'359","reported_seq":2484,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.391039+0000","last_change":"2024-05-27T09:33:11.456274+0000","last_active":"2024-05-27T09:37:37.391039+0000","last_peered":"2024-05-27T09:37:37.391039+0000","last_clean":"2024-05-27T09:37:37.391039+0000","last_became_active":"2024-05-27T09:27:18.572024+0000","last_became_peered":"2024-05-27T09:27:18.572024+0000","last_unstale":"2024-05-27T09:37:37.391039+0000","last_undegraded":"2024-05-27T09:37:37.391039+0000","last_fullsized":"2024-05-27T09:37:37.391039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:40:00.595445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010292084,"stat_sum":{"num_bytes":12019831,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":346,"num_write_kb":11826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"50'12","reported_seq":97,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.541647+0000","last_change":"2024-05-27T09:27:17.575859+0000","last_active":"2024-05-27T09:37:15.541647+0000","last_peered":"2024-05-27T09:37:15.541647+0000","last_clean":"2024-05-27T09:37:15.541647+0000","last_became_active":"2024-05-27T09:27:17.574695+0000","last_became_peered":"2024-05-27T09:27:17.574695+0000","last_unstale":"2024-05-27T09:37:15.541647+0000","last_undegraded":"2024-05-27T09:37:15.541647+0000","last_fullsized":"2024-05-27T09:37:15.541647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:24:19.379819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1331,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":9,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"50'340","reported_seq":2462,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.354340+0000","last_change":"2024-05-27T09:33:11.736362+0000","last_active":"2024-05-27T09:37:37.354340+0000","last_peered":"2024-05-27T09:37:37.354340+0000","last_clean":"2024-05-27T09:37:37.354340+0000","last_became_active":"2024-05-27T09:27:18.554932+0000","last_became_peered":"2024-05-27T09:27:18.554932+0000","last_unstale":"2024-05-27T09:37:37.354340+0000","last_undegraded":"2024-05-27T09:37:37.354340+0000","last_fullsized":"2024-05-27T09:37:37.354340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:32:26.268173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0083893030000000007,"stat_sum":{"num_bytes":24206949,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":20,"num_read_kb":12303,"num_write":329,"num_write_kb":23805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"49'68","reported_seq":163,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:41.368008+0000","last_change":"2024-05-27T09:27:17.547895+0000","last_active":"2024-05-27T09:37:41.368008+0000","last_peered":"2024-05-27T09:37:41.368008+0000","last_clean":"2024-05-27T09:37:41.368008+0000","last_became_active":"2024-05-27T09:27:17.547523+0000","last_became_peered":"2024-05-27T09:27:17.547523+0000","last_unstale":"2024-05-27T09:37:41.368008+0000","last_undegraded":"2024-05-27T09:37:41.368008+0000","last_fullsized":"2024-05-27T09:37:41.368008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":66,"num_write_kb":8220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"50'366","reported_seq":2491,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.385150+0000","last_change":"2024-05-27T09:33:11.457430+0000","last_active":"2024-05-27T09:37:37.385150+0000","last_peered":"2024-05-27T09:37:37.385150+0000","last_clean":"2024-05-27T09:37:37.385150+0000","last_became_active":"2024-05-27T09:27:18.582360+0000","last_became_peered":"2024-05-27T09:27:18.582360+0000","last_unstale":"2024-05-27T09:37:37.385150+0000","last_undegraded":"2024-05-27T09:37:37.385150+0000","last_fullsized":"2024-05-27T09:37:37.385150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:40:26.596775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0087858729999999996,"stat_sum":{"num_bytes":23260375,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":23,"num_read_kb":13874,"num_write":361,"num_write_kb":35144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"49'42","reported_seq":123,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.543499+0000","last_change":"2024-05-27T09:27:17.571481+0000","last_active":"2024-05-27T09:37:15.543499+0000","last_peered":"2024-05-27T09:37:15.543499+0000","last_clean":"2024-05-27T09:37:15.543499+0000","last_became_active":"2024-05-27T09:27:17.571146+0000","last_became_peered":"2024-05-27T09:27:17.571146+0000","last_unstale":"2024-05-27T09:37:15.543499+0000","last_undegraded":"2024-05-27T09:37:15.543499+0000","last_fullsized":"2024-05-27T09:37:15.543499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:18.038221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":8,"num_read_kb":4103,"num_write":60,"num_write_kb":4132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"50'302","reported_seq":2417,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.390053+0000","last_change":"2024-05-27T09:33:11.454615+0000","last_active":"2024-05-27T09:37:37.390053+0000","last_peered":"2024-05-27T09:37:37.390053+0000","last_clean":"2024-05-27T09:37:37.390053+0000","last_became_active":"2024-05-27T09:27:18.579440+0000","last_became_peered":"2024-05-27T09:27:18.579440+0000","last_unstale":"2024-05-27T09:37:37.390053+0000","last_undegraded":"2024-05-27T09:37:37.390053+0000","last_fullsized":"2024-05-27T09:37:37.390053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:06:43.673978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079260630000000006,"stat_sum":{"num_bytes":7949059,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":12,"num_read_kb":4346,"num_write":296,"num_write_kb":16109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"49'27","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.632660+0000","last_change":"2024-05-27T09:27:17.551295+0000","last_active":"2024-05-27T09:37:15.632660+0000","last_peered":"2024-05-27T09:37:15.632660+0000","last_clean":"2024-05-27T09:37:15.632660+0000","last_became_active":"2024-05-27T09:27:17.550861+0000","last_became_peered":"2024-05-27T09:27:17.550861+0000","last_unstale":"2024-05-27T09:37:15.632660+0000","last_undegraded":"2024-05-27T09:37:15.632660+0000","last_fullsized":"2024-05-27T09:37:15.632660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:13.955191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":4099,"num_write":31,"num_write_kb":4134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"50'337","reported_seq":2460,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.376487+0000","last_change":"2024-05-27T09:33:11.453500+0000","last_active":"2024-05-27T09:37:37.376487+0000","last_peered":"2024-05-27T09:37:37.376487+0000","last_clean":"2024-05-27T09:37:37.376487+0000","last_became_active":"2024-05-27T09:27:18.543683+0000","last_became_peered":"2024-05-27T09:27:18.543683+0000","last_unstale":"2024-05-27T09:37:37.376487+0000","last_undegraded":"2024-05-27T09:37:37.376487+0000","last_fullsized":"2024-05-27T09:37:37.376487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:29:05.179158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012478668,"stat_sum":{"num_bytes":14963144,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":20,"num_read_kb":4115,"num_write":326,"num_write_kb":22968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"49'93","reported_seq":171,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.952185+0000","last_change":"2024-05-27T09:27:17.559255+0000","last_active":"2024-05-27T09:37:15.952185+0000","last_peered":"2024-05-27T09:37:15.952185+0000","last_clean":"2024-05-27T09:37:15.952185+0000","last_became_active":"2024-05-27T09:27:17.558973+0000","last_became_peered":"2024-05-27T09:27:17.558973+0000","last_unstale":"2024-05-27T09:37:15.952185+0000","last_undegraded":"2024-05-27T09:37:15.952185+0000","last_fullsized":"2024-05-27T09:37:15.952185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":4097,"num_write":96,"num_write_kb":4263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"50'366","reported_seq":2483,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.371889+0000","last_change":"2024-05-27T09:33:11.452657+0000","last_active":"2024-05-27T09:37:37.371889+0000","last_peered":"2024-05-27T09:37:37.371889+0000","last_clean":"2024-05-27T09:37:37.371889+0000","last_became_active":"2024-05-27T09:27:18.557076+0000","last_became_peered":"2024-05-27T09:27:18.557076+0000","last_unstale":"2024-05-27T09:37:37.371889+0000","last_undegraded":"2024-05-27T09:37:37.371889+0000","last_fullsized":"2024-05-27T09:37:37.371889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:26:39.506639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014584746000000001,"stat_sum":{"num_bytes":15385587,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":358,"num_write_kb":15182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"49'49","reported_seq":134,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.953808+0000","last_change":"2024-05-27T09:27:17.573700+0000","last_active":"2024-05-27T09:37:15.953808+0000","last_peered":"2024-05-27T09:37:15.953808+0000","last_clean":"2024-05-27T09:37:15.953808+0000","last_became_active":"2024-05-27T09:27:17.573484+0000","last_became_peered":"2024-05-27T09:27:17.573484+0000","last_unstale":"2024-05-27T09:37:15.953808+0000","last_undegraded":"2024-05-27T09:37:15.953808+0000","last_fullsized":"2024-05-27T09:37:15.953808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:48.905422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":51,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"50'336","reported_seq":2461,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.377629+0000","last_change":"2024-05-27T09:33:11.456688+0000","last_active":"2024-05-27T09:37:37.377629+0000","last_peered":"2024-05-27T09:37:37.377629+0000","last_clean":"2024-05-27T09:37:37.377629+0000","last_became_active":"2024-05-27T09:27:18.574959+0000","last_became_peered":"2024-05-27T09:27:18.574959+0000","last_unstale":"2024-05-27T09:37:37.377629+0000","last_undegraded":"2024-05-27T09:37:37.377629+0000","last_fullsized":"2024-05-27T09:37:37.377629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:53.188459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0093604480000000004,"stat_sum":{"num_bytes":20164393,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":23,"num_read_kb":12305,"num_write":329,"num_write_kb":28020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":173,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.533665+0000","last_change":"2024-05-27T09:27:12.693423+0000","last_active":"2024-05-27T09:37:15.533665+0000","last_peered":"2024-05-27T09:37:15.533665+0000","last_clean":"2024-05-27T09:37:15.533665+0000","last_became_active":"2024-05-27T09:27:12.690301+0000","last_became_peered":"2024-05-27T09:27:12.690301+0000","last_unstale":"2024-05-27T09:37:15.533665+0000","last_undegraded":"2024-05-27T09:37:15.533665+0000","last_fullsized":"2024-05-27T09:37:15.533665+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:11.463226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:11.463226+0000","last_clean_scrub_stamp":"2024-05-27T09:27:11.463226+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:51:01.810071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'12","reported_seq":95,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:41.365431+0000","last_change":"2024-05-27T09:27:17.567670+0000","last_active":"2024-05-27T09:37:41.365431+0000","last_peered":"2024-05-27T09:37:41.365431+0000","last_clean":"2024-05-27T09:37:41.365431+0000","last_became_active":"2024-05-27T09:27:17.566292+0000","last_became_peered":"2024-05-27T09:27:17.566292+0000","last_unstale":"2024-05-27T09:37:41.365431+0000","last_undegraded":"2024-05-27T09:37:41.365431+0000","last_fullsized":"2024-05-27T09:37:41.365431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:18.038221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194338,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":21,"num_write":14,"num_write_kb":4206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"50'341","reported_seq":2456,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.367701+0000","last_change":"2024-05-27T09:33:11.733467+0000","last_active":"2024-05-27T09:37:37.367701+0000","last_peered":"2024-05-27T09:37:37.367701+0000","last_clean":"2024-05-27T09:37:37.367701+0000","last_became_active":"2024-05-27T09:27:18.570782+0000","last_became_peered":"2024-05-27T09:27:18.570782+0000","last_unstale":"2024-05-27T09:37:37.367701+0000","last_undegraded":"2024-05-27T09:37:37.367701+0000","last_fullsized":"2024-05-27T09:37:37.367701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:22.877083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044685039999999999,"stat_sum":{"num_bytes":10376859,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":333,"num_write_kb":18496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"50'344","reported_seq":428,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:17.550853+0000","last_change":"2024-05-27T09:27:17.563259+0000","last_active":"2024-05-27T09:37:17.550853+0000","last_peered":"2024-05-27T09:37:17.550853+0000","last_clean":"2024-05-27T09:37:17.550853+0000","last_became_active":"2024-05-27T09:27:17.563010+0000","last_became_peered":"2024-05-27T09:27:17.563010+0000","last_unstale":"2024-05-27T09:37:17.550853+0000","last_undegraded":"2024-05-27T09:37:17.550853+0000","last_fullsized":"2024-05-27T09:37:17.550853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:53.711508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":8210,"num_write":339,"num_write_kb":12482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"50'323","reported_seq":2436,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.367571+0000","last_change":"2024-05-27T09:33:11.730552+0000","last_active":"2024-05-27T09:37:37.367571+0000","last_peered":"2024-05-27T09:37:37.367571+0000","last_clean":"2024-05-27T09:37:37.367571+0000","last_became_active":"2024-05-27T09:27:18.574090+0000","last_became_peered":"2024-05-27T09:27:18.574090+0000","last_unstale":"2024-05-27T09:37:37.367571+0000","last_undegraded":"2024-05-27T09:37:37.367571+0000","last_fullsized":"2024-05-27T09:37:37.367571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:29:43.530819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002904064,"stat_sum":{"num_bytes":5516261,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":316,"num_write_kb":17833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"50'123","reported_seq":317,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.544553+0000","last_change":"2024-05-27T09:27:17.551310+0000","last_active":"2024-05-27T09:37:20.544553+0000","last_peered":"2024-05-27T09:37:20.544553+0000","last_clean":"2024-05-27T09:37:20.544553+0000","last_became_active":"2024-05-27T09:27:17.550982+0000","last_became_peered":"2024-05-27T09:27:17.550982+0000","last_unstale":"2024-05-27T09:37:20.544553+0000","last_undegraded":"2024-05-27T09:37:20.544553+0000","last_fullsized":"2024-05-27T09:37:20.544553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":534380,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":130,"num_read_kb":615,"num_write":121,"num_write_kb":621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"50'348","reported_seq":2473,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.349125+0000","last_change":"2024-05-27T09:33:11.732933+0000","last_active":"2024-05-27T09:37:37.349125+0000","last_peered":"2024-05-27T09:37:37.349125+0000","last_clean":"2024-05-27T09:37:37.349125+0000","last_became_active":"2024-05-27T09:27:18.559017+0000","last_became_peered":"2024-05-27T09:27:18.559017+0000","last_unstale":"2024-05-27T09:37:37.349125+0000","last_undegraded":"2024-05-27T09:37:37.349125+0000","last_fullsized":"2024-05-27T09:37:37.349125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:06:28.204044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065168789999999997,"stat_sum":{"num_bytes":26414737,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":21,"num_read_kb":8209,"num_write":340,"num_write_kb":34165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"49'48","reported_seq":126,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.547944+0000","last_change":"2024-05-27T09:27:17.563947+0000","last_active":"2024-05-27T09:37:15.547944+0000","last_peered":"2024-05-27T09:37:15.547944+0000","last_clean":"2024-05-27T09:37:15.547944+0000","last_became_active":"2024-05-27T09:27:17.563640+0000","last_became_peered":"2024-05-27T09:27:17.563640+0000","last_unstale":"2024-05-27T09:37:15.547944+0000","last_undegraded":"2024-05-27T09:37:15.547944+0000","last_fullsized":"2024-05-27T09:37:15.547944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:02.747708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":43,"num_write_kb":8223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"50'359","reported_seq":2479,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.388774+0000","last_change":"2024-05-27T09:33:11.442514+0000","last_active":"2024-05-27T09:37:37.388774+0000","last_peered":"2024-05-27T09:37:37.388774+0000","last_clean":"2024-05-27T09:37:37.388774+0000","last_became_active":"2024-05-27T09:27:18.574710+0000","last_became_peered":"2024-05-27T09:27:18.574710+0000","last_unstale":"2024-05-27T09:37:37.388774+0000","last_undegraded":"2024-05-27T09:37:37.388774+0000","last_fullsized":"2024-05-27T09:37:37.388774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:50:58.577908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075561990000000004,"stat_sum":{"num_bytes":9767075,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":17,"num_read_kb":8206,"num_write":352,"num_write_kb":9721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"49'81","reported_seq":157,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.952072+0000","last_change":"2024-05-27T09:27:17.550749+0000","last_active":"2024-05-27T09:37:15.952072+0000","last_peered":"2024-05-27T09:37:15.952072+0000","last_clean":"2024-05-27T09:37:15.952072+0000","last_became_active":"2024-05-27T09:27:17.550378+0000","last_became_peered":"2024-05-27T09:27:17.550378+0000","last_unstale":"2024-05-27T09:37:15.952072+0000","last_undegraded":"2024-05-27T09:37:15.952072+0000","last_fullsized":"2024-05-27T09:37:15.952072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:20.167629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83,"num_write_kb":4165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"50'360","reported_seq":2483,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.368574+0000","last_change":"2024-05-27T09:33:11.451931+0000","last_active":"2024-05-27T09:37:37.368574+0000","last_peered":"2024-05-27T09:37:37.368574+0000","last_clean":"2024-05-27T09:37:37.368574+0000","last_became_active":"2024-05-27T09:27:18.566326+0000","last_became_peered":"2024-05-27T09:27:18.566326+0000","last_unstale":"2024-05-27T09:37:37.368574+0000","last_undegraded":"2024-05-27T09:37:37.368574+0000","last_fullsized":"2024-05-27T09:37:37.368574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:50.848181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075094710000000002,"stat_sum":{"num_bytes":14788118,"num_objects":187,"num_object_clones":0,"num_object_copies":374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":187,"num_whiteouts":0,"num_read":20,"num_read_kb":8209,"num_write":356,"num_write_kb":35101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'352","reported_seq":431,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.528059+0000","last_change":"2024-05-27T09:27:17.557912+0000","last_active":"2024-05-27T09:37:20.528059+0000","last_peered":"2024-05-27T09:37:20.528059+0000","last_clean":"2024-05-27T09:37:20.528059+0000","last_became_active":"2024-05-27T09:27:17.557216+0000","last_became_peered":"2024-05-27T09:27:17.557216+0000","last_unstale":"2024-05-27T09:37:20.528059+0000","last_undegraded":"2024-05-27T09:37:20.528059+0000","last_fullsized":"2024-05-27T09:37:20.528059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:21:22.020724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":4098,"num_write":353,"num_write_kb":8433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"50'347","reported_seq":2463,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.353953+0000","last_change":"2024-05-27T09:33:11.728508+0000","last_active":"2024-05-27T09:37:37.353953+0000","last_peered":"2024-05-27T09:37:37.353953+0000","last_clean":"2024-05-27T09:37:37.353953+0000","last_became_active":"2024-05-27T09:27:18.562565+0000","last_became_peered":"2024-05-27T09:27:18.562565+0000","last_unstale":"2024-05-27T09:37:37.353953+0000","last_undegraded":"2024-05-27T09:37:37.353953+0000","last_fullsized":"2024-05-27T09:37:37.353953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:45:47.000935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030336680000000002,"stat_sum":{"num_bytes":10461277,"num_objects":176,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":176,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":338,"num_write_kb":10392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"50'198","reported_seq":280,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:35.644068+0000","last_change":"2024-05-27T09:27:17.571911+0000","last_active":"2024-05-27T09:37:35.644068+0000","last_peered":"2024-05-27T09:37:35.644068+0000","last_clean":"2024-05-27T09:37:35.644068+0000","last_became_active":"2024-05-27T09:27:17.571551+0000","last_became_peered":"2024-05-27T09:27:17.571551+0000","last_unstale":"2024-05-27T09:37:35.644068+0000","last_undegraded":"2024-05-27T09:37:35.644068+0000","last_fullsized":"2024-05-27T09:37:35.644068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:20.167629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":455,"num_write_kb":2056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"50'329","reported_seq":2447,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.348511+0000","last_change":"2024-05-27T09:33:11.734048+0000","last_active":"2024-05-27T09:37:37.348511+0000","last_peered":"2024-05-27T09:37:37.348511+0000","last_clean":"2024-05-27T09:37:37.348511+0000","last_became_active":"2024-05-27T09:27:18.558463+0000","last_became_peered":"2024-05-27T09:27:18.558463+0000","last_unstale":"2024-05-27T09:37:37.348511+0000","last_undegraded":"2024-05-27T09:37:37.348511+0000","last_fullsized":"2024-05-27T09:37:37.348511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:29:43.530819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041648589999999999,"stat_sum":{"num_bytes":14892598,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":326,"num_write_kb":22861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"49'171","reported_seq":251,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:41.368959+0000","last_change":"2024-05-27T09:27:17.567973+0000","last_active":"2024-05-27T09:37:41.368959+0000","last_peered":"2024-05-27T09:37:41.368959+0000","last_clean":"2024-05-27T09:37:41.368959+0000","last_became_active":"2024-05-27T09:27:17.567650+0000","last_became_peered":"2024-05-27T09:27:17.567650+0000","last_unstale":"2024-05-27T09:37:41.368959+0000","last_undegraded":"2024-05-27T09:37:41.368959+0000","last_fullsized":"2024-05-27T09:37:41.368959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:25:52.924185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":176,"num_write_kb":4290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"50'347","reported_seq":2476,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.350628+0000","last_change":"2024-05-27T09:33:11.733662+0000","last_active":"2024-05-27T09:37:37.350628+0000","last_peered":"2024-05-27T09:37:37.350628+0000","last_clean":"2024-05-27T09:37:37.350628+0000","last_became_active":"2024-05-27T09:27:18.554462+0000","last_became_peered":"2024-05-27T09:27:18.554462+0000","last_unstale":"2024-05-27T09:37:37.350628+0000","last_undegraded":"2024-05-27T09:37:37.350628+0000","last_fullsized":"2024-05-27T09:37:37.350628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:21:17.734561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042444429999999997,"stat_sum":{"num_bytes":6471160,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":27,"num_read_kb":4122,"num_write":339,"num_write_kb":6465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"49'126","reported_seq":207,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:41.379954+0000","last_change":"2024-05-27T09:27:17.570300+0000","last_active":"2024-05-27T09:37:41.379954+0000","last_peered":"2024-05-27T09:37:41.379954+0000","last_clean":"2024-05-27T09:37:41.379954+0000","last_became_active":"2024-05-27T09:27:17.569685+0000","last_became_peered":"2024-05-27T09:27:17.569685+0000","last_unstale":"2024-05-27T09:37:41.379954+0000","last_undegraded":"2024-05-27T09:37:41.379954+0000","last_fullsized":"2024-05-27T09:37:41.379954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:26.210041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":4108,"num_write":127,"num_write_kb":4196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"50'355","reported_seq":2484,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.352377+0000","last_change":"2024-05-27T09:33:11.450475+0000","last_active":"2024-05-27T09:37:37.352377+0000","last_peered":"2024-05-27T09:37:37.352377+0000","last_clean":"2024-05-27T09:37:37.352377+0000","last_became_active":"2024-05-27T09:27:18.562455+0000","last_became_peered":"2024-05-27T09:27:18.562455+0000","last_unstale":"2024-05-27T09:37:37.352377+0000","last_undegraded":"2024-05-27T09:37:37.352377+0000","last_fullsized":"2024-05-27T09:37:37.352377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:31.423566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082016410000000008,"stat_sum":{"num_bytes":31948882,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":27,"num_read_kb":17799,"num_write":353,"num_write_kb":35479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"49'171","reported_seq":252,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:41.379258+0000","last_change":"2024-05-27T09:27:17.563172+0000","last_active":"2024-05-27T09:37:41.379258+0000","last_peered":"2024-05-27T09:37:41.379258+0000","last_clean":"2024-05-27T09:37:41.379258+0000","last_became_active":"2024-05-27T09:27:17.562941+0000","last_became_peered":"2024-05-27T09:27:17.562941+0000","last_unstale":"2024-05-27T09:37:41.379258+0000","last_undegraded":"2024-05-27T09:37:41.379258+0000","last_fullsized":"2024-05-27T09:37:41.379258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:51.742078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":171,"num_write_kb":4214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"50'349","reported_seq":2470,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.349831+0000","last_change":"2024-05-27T09:33:11.731891+0000","last_active":"2024-05-27T09:37:37.349831+0000","last_peered":"2024-05-27T09:37:37.349831+0000","last_clean":"2024-05-27T09:37:37.349831+0000","last_became_active":"2024-05-27T09:27:18.555280+0000","last_became_peered":"2024-05-27T09:27:18.555280+0000","last_unstale":"2024-05-27T09:37:37.349831+0000","last_undegraded":"2024-05-27T09:37:37.349831+0000","last_fullsized":"2024-05-27T09:37:37.349831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:31.147919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041574209999999997,"stat_sum":{"num_bytes":13440874,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":19,"num_read_kb":8208,"num_write":339,"num_write_kb":17394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"50'255","reported_seq":334,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.396188+0000","last_change":"2024-05-27T09:27:17.548266+0000","last_active":"2024-05-27T09:37:37.396188+0000","last_peered":"2024-05-27T09:37:37.396188+0000","last_clean":"2024-05-27T09:37:37.396188+0000","last_became_active":"2024-05-27T09:27:17.547938+0000","last_became_peered":"2024-05-27T09:27:17.547938+0000","last_unstale":"2024-05-27T09:37:37.396188+0000","last_undegraded":"2024-05-27T09:37:37.396188+0000","last_fullsized":"2024-05-27T09:37:37.396188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7113047,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":4100,"num_write":284,"num_write_kb":7099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"50'342","reported_seq":2460,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.352342+0000","last_change":"2024-05-27T09:33:11.625744+0000","last_active":"2024-05-27T09:37:37.352342+0000","last_peered":"2024-05-27T09:37:37.352342+0000","last_clean":"2024-05-27T09:37:37.352342+0000","last_became_active":"2024-05-27T09:27:18.563227+0000","last_became_peered":"2024-05-27T09:27:18.563227+0000","last_unstale":"2024-05-27T09:37:37.352342+0000","last_undegraded":"2024-05-27T09:37:37.352342+0000","last_fullsized":"2024-05-27T09:37:37.352342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:30:49.326058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010490250000000001,"stat_sum":{"num_bytes":10717337,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":15,"num_read_kb":2569,"num_write":334,"num_write_kb":14740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"50'342","reported_seq":424,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:41.370164+0000","last_change":"2024-05-27T09:27:17.562200+0000","last_active":"2024-05-27T09:37:41.370164+0000","last_peered":"2024-05-27T09:37:41.370164+0000","last_clean":"2024-05-27T09:37:41.370164+0000","last_became_active":"2024-05-27T09:27:17.561857+0000","last_became_peered":"2024-05-27T09:27:17.561857+0000","last_unstale":"2024-05-27T09:37:41.370164+0000","last_undegraded":"2024-05-27T09:37:41.370164+0000","last_fullsized":"2024-05-27T09:37:41.370164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:05.209528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":16,"num_read_kb":8206,"num_write":340,"num_write_kb":12481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"50'360","reported_seq":2475,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.372550+0000","last_change":"2024-05-27T09:33:11.455285+0000","last_active":"2024-05-27T09:37:37.372550+0000","last_peered":"2024-05-27T09:37:37.372550+0000","last_clean":"2024-05-27T09:37:37.372550+0000","last_became_active":"2024-05-27T09:27:18.572339+0000","last_became_peered":"2024-05-27T09:27:18.572339+0000","last_unstale":"2024-05-27T09:37:37.372550+0000","last_undegraded":"2024-05-27T09:37:37.372550+0000","last_fullsized":"2024-05-27T09:37:37.372550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:11:51.773780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011410574,"stat_sum":{"num_bytes":1288358,"num_objects":185,"num_object_clones":0,"num_object_copies":370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":185,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":355,"num_write_kb":9639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"50'330","reported_seq":2455,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.388768+0000","last_change":"2024-05-27T09:33:11.456154+0000","last_active":"2024-05-27T09:37:37.388768+0000","last_peered":"2024-05-27T09:37:37.388768+0000","last_clean":"2024-05-27T09:37:37.388768+0000","last_became_active":"2024-05-27T09:27:18.572307+0000","last_became_peered":"2024-05-27T09:27:18.572307+0000","last_unstale":"2024-05-27T09:37:37.388768+0000","last_undegraded":"2024-05-27T09:37:37.388768+0000","last_fullsized":"2024-05-27T09:37:37.388768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:38:38.928939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089418529999999996,"stat_sum":{"num_bytes":7551421,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":315,"num_write_kb":11635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"49'32","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.542484+0000","last_change":"2024-05-27T09:27:17.570634+0000","last_active":"2024-05-27T09:37:15.542484+0000","last_peered":"2024-05-27T09:37:15.542484+0000","last_clean":"2024-05-27T09:37:15.542484+0000","last_became_active":"2024-05-27T09:27:17.569866+0000","last_became_peered":"2024-05-27T09:27:17.569866+0000","last_unstale":"2024-05-27T09:37:15.542484+0000","last_undegraded":"2024-05-27T09:37:15.542484+0000","last_fullsized":"2024-05-27T09:37:15.542484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:21:38.268535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4099,"num_write":35,"num_write_kb":4131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"50'359","reported_seq":2476,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.380935+0000","last_change":"2024-05-27T09:33:11.737809+0000","last_active":"2024-05-27T09:37:37.380935+0000","last_peered":"2024-05-27T09:37:37.380935+0000","last_clean":"2024-05-27T09:37:37.380935+0000","last_became_active":"2024-05-27T09:27:18.549035+0000","last_became_peered":"2024-05-27T09:27:18.549035+0000","last_unstale":"2024-05-27T09:37:37.380935+0000","last_undegraded":"2024-05-27T09:37:37.380935+0000","last_fullsized":"2024-05-27T09:37:37.380935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:14:41.023261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0090041500000000007,"stat_sum":{"num_bytes":20118472,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":14,"num_read_kb":13,"num_write":350,"num_write_kb":27987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"49'143","reported_seq":226,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.543458+0000","last_change":"2024-05-27T09:27:17.568143+0000","last_active":"2024-05-27T09:37:15.543458+0000","last_peered":"2024-05-27T09:37:15.543458+0000","last_clean":"2024-05-27T09:37:15.543458+0000","last_became_active":"2024-05-27T09:27:17.567235+0000","last_became_peered":"2024-05-27T09:27:17.567235+0000","last_unstale":"2024-05-27T09:37:15.543458+0000","last_undegraded":"2024-05-27T09:37:15.543458+0000","last_fullsized":"2024-05-27T09:37:15.543458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.284270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":15,"num_read_kb":12312,"num_write":136,"num_write_kb":12381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"50'319","reported_seq":2443,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.372942+0000","last_change":"2024-05-27T09:33:11.732530+0000","last_active":"2024-05-27T09:37:37.372942+0000","last_peered":"2024-05-27T09:37:37.372942+0000","last_clean":"2024-05-27T09:37:37.372942+0000","last_became_active":"2024-05-27T09:27:18.562919+0000","last_became_peered":"2024-05-27T09:27:18.562919+0000","last_unstale":"2024-05-27T09:37:37.372942+0000","last_undegraded":"2024-05-27T09:37:37.372942+0000","last_fullsized":"2024-05-27T09:37:37.372942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:09:22.339045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0063913879999999996,"stat_sum":{"num_bytes":22598856,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":21,"num_read_kb":8211,"num_write":313,"num_write_kb":22222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"49'87","reported_seq":165,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.953251+0000","last_change":"2024-05-27T09:27:17.551193+0000","last_active":"2024-05-27T09:37:15.953251+0000","last_peered":"2024-05-27T09:37:15.953251+0000","last_clean":"2024-05-27T09:37:15.953251+0000","last_became_active":"2024-05-27T09:27:17.550849+0000","last_became_peered":"2024-05-27T09:27:17.550849+0000","last_unstale":"2024-05-27T09:37:15.953251+0000","last_undegraded":"2024-05-27T09:37:15.953251+0000","last_fullsized":"2024-05-27T09:37:15.953251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:51.742078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":81,"num_write_kb":4153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"50'336","reported_seq":2454,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.385192+0000","last_change":"2024-05-27T09:33:11.732054+0000","last_active":"2024-05-27T09:37:37.385192+0000","last_peered":"2024-05-27T09:37:37.385192+0000","last_clean":"2024-05-27T09:37:37.385192+0000","last_became_active":"2024-05-27T09:27:18.579574+0000","last_became_peered":"2024-05-27T09:27:18.579574+0000","last_unstale":"2024-05-27T09:37:37.385192+0000","last_undegraded":"2024-05-27T09:37:37.385192+0000","last_fullsized":"2024-05-27T09:37:37.385192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:56:05.185246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062209780000000003,"stat_sum":{"num_bytes":14492175,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":14,"num_read_kb":4108,"num_write":330,"num_write_kb":14321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"49'82","reported_seq":172,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.632721+0000","last_change":"2024-05-27T09:27:17.549711+0000","last_active":"2024-05-27T09:37:15.632721+0000","last_peered":"2024-05-27T09:37:15.632721+0000","last_clean":"2024-05-27T09:37:15.632721+0000","last_became_active":"2024-05-27T09:27:17.549291+0000","last_became_peered":"2024-05-27T09:27:17.549291+0000","last_unstale":"2024-05-27T09:37:15.632721+0000","last_undegraded":"2024-05-27T09:37:15.632721+0000","last_fullsized":"2024-05-27T09:37:15.632721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:16:22.057803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":41,"num_read_kb":4136,"num_write":85,"num_write_kb":8333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":952881208,"num_objects":10697,"num_object_clones":0,"num_object_copies":21394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10697,"num_whiteouts":0,"num_read":1250,"num_read_kb":378565,"num_write":21433,"num_write_kb":1379166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1734066176,"data_stored":1904222320,"data_compressed":168711384,"data_compressed_allocated":186425344,"data_compressed_original":412123136,"omap_allocated":0,"internal_metadata":0},"log_size":21961,"ondisk_log_size":21961,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":188008061,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":557,"num_read_kb":123805,"num_write":4420,"num_write_kb":189069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":105,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":83959808,"data_stored":376016122,"data_compressed":49745852,"data_compressed_allocated":62447616,"data_compressed_original":354656256,"omap_allocated":0,"internal_metadata":0},"log_size":4119,"ondisk_log_size":4119,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65078,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509570,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":381232,"kb_used_data":309800,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":93990608,"statfs":{"total":96636764160,"available":96246382592,"internally_reserved":0,"allocated":317235200,"data_stored":412122108,"data_compressed":41350796,"data_compressed_allocated":47357952,"data_compressed_original":150941696,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509570,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":282276,"kb_used_data":233052,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94089564,"statfs":{"total":96636764160,"available":96347713536,"internally_reserved":0,"allocated":238645248,"data_stored":280794177,"data_compressed":23230909,"data_compressed_allocated":26537984,"data_compressed_original":76705792,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509570,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246488,"kb_used_data":201424,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94125352,"statfs":{"total":96636764160,"available":96384360448,"internally_reserved":0,"allocated":206258176,"data_stored":257021246,"data_compressed":23229218,"data_compressed_allocated":26480640,"data_compressed_original":83582976,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509570,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356864,"kb_used_data":285432,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94014976,"statfs":{"total":96636764160,"available":96271335424,"internally_reserved":0,"allocated":292282368,"data_stored":379742172,"data_compressed":39531620,"data_compressed_allocated":45047808,"data_compressed_original":141885440,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542273,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":157168,"kb_used_data":130152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94214672,"statfs":{"total":96636764160,"available":96475824128,"internally_reserved":0,"allocated":133275648,"data_stored":168154774,"data_compressed":18202841,"data_compressed_allocated":20492288,"data_compressed_original":60579840,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542273,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":196060,"kb_used_data":146836,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94175780,"statfs":{"total":96636764160,"available":96435998720,"internally_reserved":0,"allocated":150360064,"data_stored":185784291,"data_compressed":18841571,"data_compressed_allocated":21557248,"data_compressed_original":63373312,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542273,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":358788,"kb_used_data":305404,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94013052,"statfs":{"total":96636764160,"available":96269365248,"internally_reserved":0,"allocated":312733696,"data_stored":372392046,"data_compressed":31436983,"data_compressed_allocated":35463168,"data_compressed_original":103411712,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542273,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":221244,"kb_used_data":167860,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94150596,"statfs":{"total":96636764160,"available":96410210304,"internally_reserved":0,"allocated":171888640,"data_stored":227475258,"data_compressed":22698376,"data_compressed_allocated":26025984,"data_compressed_original":87445504,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13996032,"data_stored":67110923,"data_compressed":8258411,"data_compressed_allocated":10555392,"data_compressed_original":63688704,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7446528,"data_stored":37748896,"data_compressed":4390710,"data_compressed_allocated":5623808,"data_compressed_original":35934208,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7921664,"data_stored":33555138,"data_compressed":4517094,"data_compressed_allocated":5791744,"data_compressed_original":31436800,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19890176,"data_stored":83888049,"data_compressed":11332765,"data_compressed_allocated":14295040,"data_compressed_original":78307328,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12206080,"data_stored":50333888,"data_compressed":7575831,"data_compressed_allocated":9129984,"data_compressed_original":47296512,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9977856,"data_stored":45397504,"data_compressed":5653863,"data_compressed_allocated":7127040,"data_compressed_original":42569728,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7090176,"data_stored":32280702,"data_compressed":4471564,"data_compressed_allocated":5599232,"data_compressed_original":30801920,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5431296,"data_stored":25701022,"data_compressed":3545614,"data_compressed_allocated":4325376,"data_compressed_original":24621056,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":277721088,"data_stored":312372852,"data_compressed":31273209,"data_compressed_allocated":34492416,"data_compressed_original":78196736,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":198180864,"data_stored":218423585,"data_compressed":18805969,"data_compressed_allocated":20811776,"data_compressed_original":47075328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":230158336,"data_stored":246980642,"data_compressed":18713815,"data_compressed_allocated":20746240,"data_compressed_original":45268992,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":296779776,"data_stored":327975662,"data_compressed":30018031,"data_compressed_allocated":33062912,"data_compressed_original":72634368,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":159117312,"data_stored":176882973,"data_compressed":15122545,"data_compressed_allocated":16896000,"data_compressed_original":40148992,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":302125056,"data_stored":326145918,"data_compressed":25750581,"data_compressed_allocated":28291072,"data_compressed_original":60268544,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":142704640,"data_stored":153245333,"data_compressed":14370007,"data_compressed_allocated":15958016,"data_compressed_original":32571392,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":127279104,"data_stored":142195355,"data_compressed":14657227,"data_compressed_allocated":16166912,"data_compressed_original":35958784,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T09:37:43.389 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T09:37:43.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.704+0000 7f706f000640 1 Processor -- start 2024-05-27T09:37:43.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.705+0000 7f706f000640 1 -- start start 2024-05-27T09:37:43.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.706+0000 7f706f000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f70680c2a00 0x7f7068159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:43.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.706+0000 7f706f000640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f7068159e40 0x7f7068074140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:43.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.706+0000 7f706f000640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70680c96d0 con 0x7f70681534c0 2024-05-27T09:37:43.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.706+0000 7f706f000640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f7068159190 con 0x7f7068159e40 2024-05-27T09:37:43.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.706+0000 7f706f000640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f70681538d0 con 0x7f70680c2a00 2024-05-27T09:37:43.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.707+0000 7f706dc00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f70681534c0 0x7f70680c8220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:56804/0 (socket says 172.21.15.114:56804) 2024-05-27T09:37:43.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.707+0000 7f706dc00640 1 -- 172.21.15.114:0/3180541760 learned_addr learned my addr 172.21.15.114:0/3180541760 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:43.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.707+0000 7f706e600640 1 --2- 172.21.15.114:0/3180541760 >> v2:172.21.15.114:3300/0 conn(0x7f7068159e40 0x7f7068074140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:43.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.707+0000 7f706d200640 1 --2- 172.21.15.114:0/3180541760 >> v2:172.21.15.163:3300/0 conn(0x7f70680c2a00 0x7f7068159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:43.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.707+0000 7f7067e00640 1 -- 172.21.15.114:0/3180541760 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3184445779 0 0) 0x7f70680c96d0 con 0x7f70681534c0 2024-05-27T09:37:43.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.707+0000 7f7067e00640 1 -- 172.21.15.114:0/3180541760 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7050005730 con 0x7f70681534c0 2024-05-27T09:37:43.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.707+0000 7f706e600640 1 -- 172.21.15.114:0/3180541760 >> v2:172.21.15.163:3300/0 conn(0x7f70680c2a00 msgr2=0x7f7068159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:43.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.707+0000 7f706e600640 1 --2- 172.21.15.114:0/3180541760 >> v2:172.21.15.163:3300/0 conn(0x7f70680c2a00 0x7f7068159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:43.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.707+0000 7f706e600640 1 -- 172.21.15.114:0/3180541760 >> v1:172.21.15.114:6789/0 conn(0x7f70681534c0 legacy=0x7f70680c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:43.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.707+0000 7f706e600640 1 -- 172.21.15.114:0/3180541760 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70680c8930 con 0x7f7068159e40 2024-05-27T09:37:43.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.708+0000 7f706e600640 1 --2- 172.21.15.114:0/3180541760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7068159e40 0x7f7068074140 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7068071be0 tx=0x7f706002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43070f43fd13ebb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:43.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.709+0000 7f7067e00640 1 -- 172.21.15.114:0/3180541760 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060007740 con 0x7f7068159e40 2024-05-27T09:37:43.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.709+0000 7f7067e00640 1 -- 172.21.15.114:0/3180541760 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70600078e0 con 0x7f7068159e40 2024-05-27T09:37:43.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.709+0000 7f7067e00640 1 -- 172.21.15.114:0/3180541760 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060007cb0 con 0x7f7068159e40 2024-05-27T09:37:43.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.709+0000 7f706f000640 1 -- 172.21.15.114:0/3180541760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7068159e40 msgr2=0x7f7068074140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:43.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.709+0000 7f706f000640 1 --2- 172.21.15.114:0/3180541760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7068159e40 0x7f7068074140 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7068071be0 tx=0x7f706002a680 comp rx=0 tx=0).stop 2024-05-27T09:37:43.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.709+0000 7f706f000640 1 -- 172.21.15.114:0/3180541760 shutdown_connections 2024-05-27T09:37:43.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.709+0000 7f706f000640 1 --2- 172.21.15.114:0/3180541760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7068159e40 0x7f7068074140 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:43.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.709+0000 7f706f000640 1 --2- 172.21.15.114:0/3180541760 >> v2:172.21.15.163:3300/0 conn(0x7f70680c2a00 0x7f7068159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:43.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.710+0000 7f706f000640 1 -- 172.21.15.114:0/3180541760 >> 172.21.15.114:0/3180541760 conn(0x7f70680ca320 msgr2=0x7f70680ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:43.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.710+0000 7f706f000640 1 -- 172.21.15.114:0/3180541760 shutdown_connections 2024-05-27T09:37:43.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.710+0000 7f706f000640 1 -- 172.21.15.114:0/3180541760 wait complete. 2024-05-27T09:37:43.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.712+0000 7f706f000640 1 Processor -- start 2024-05-27T09:37:43.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.712+0000 7f706f000640 1 -- start start 2024-05-27T09:37:43.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.713+0000 7f706f000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70680c2a00 0x7f70681f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:43.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.713+0000 7f706f000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70681534c0 0x7f7068073ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:43.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.713+0000 7f706f000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7068159e40 0x7f706814fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:43.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.713+0000 7f706f000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f70680c9f30 con 0x7f7068159e40 2024-05-27T09:37:43.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.713+0000 7f706f000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f70680c9c50 con 0x7f70680c2a00 2024-05-27T09:37:43.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.714+0000 7f706f000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f70681593e0 con 0x7f70681534c0 2024-05-27T09:37:43.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.714+0000 7f706e600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7068159e40 0x7f706814fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:43.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.714+0000 7f706e600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7068159e40 0x7f706814fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39792/0 (socket says 172.21.15.114:39792) 2024-05-27T09:37:43.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.714+0000 7f706e600640 1 -- 172.21.15.114:0/1053334750 learned_addr learned my addr 172.21.15.114:0/1053334750 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:43.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.714+0000 7f706dc00640 1 --2- 172.21.15.114:0/1053334750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70680c2a00 0x7f70681f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:43.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.714+0000 7f706d200640 1 --2- 172.21.15.114:0/1053334750 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70681534c0 0x7f7068073ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:43.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.714+0000 7f706e600640 1 -- 172.21.15.114:0/1053334750 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70681534c0 msgr2=0x7f7068073ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:43.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.714+0000 7f706e600640 1 --2- 172.21.15.114:0/1053334750 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70681534c0 0x7f7068073ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:43.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.714+0000 7f706e600640 1 -- 172.21.15.114:0/1053334750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70680c2a00 msgr2=0x7f70681f2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:43.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.714+0000 7f706e600640 1 --2- 172.21.15.114:0/1053334750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70680c2a00 0x7f70681f2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:43.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.714+0000 7f706e600640 1 -- 172.21.15.114:0/1053334750 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70681f2ff0 con 0x7f7068159e40 2024-05-27T09:37:43.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.715+0000 7f706e600640 1 --2- 172.21.15.114:0/1053334750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7068159e40 0x7f706814fb80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7060031630 tx=0x7f7060004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:43.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.715+0000 7f7065600640 1 -- 172.21.15.114:0/1053334750 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060009450 con 0x7f7068159e40 2024-05-27T09:37:43.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.715+0000 7f7065600640 1 -- 172.21.15.114:0/1053334750 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7060031c20 con 0x7f7068159e40 2024-05-27T09:37:43.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.715+0000 7f706f000640 1 -- 172.21.15.114:0/1053334750 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70681509c0 con 0x7f7068159e40 2024-05-27T09:37:43.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.715+0000 7f7065600640 1 -- 172.21.15.114:0/1053334750 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706003b410 con 0x7f7068159e40 2024-05-27T09:37:43.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.715+0000 7f706f000640 1 -- 172.21.15.114:0/1053334750 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70680b06f0 con 0x7f7068159e40 2024-05-27T09:37:43.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.716+0000 7f706f000640 1 -- 172.21.15.114:0/1053334750 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7068050860 con 0x7f7068159e40 2024-05-27T09:37:43.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.717+0000 7f7065600640 1 -- 172.21.15.114:0/1053334750 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f706003b5d0 con 0x7f7068159e40 2024-05-27T09:37:43.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.718+0000 7f7065600640 1 --2- 172.21.15.114:0/1053334750 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f70480762d0 0x7f70480787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:43.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.718+0000 7f7065600640 1 -- 172.21.15.114:0/1053334750 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7f70600b8be0 con 0x7f7068159e40 2024-05-27T09:37:43.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.719+0000 7f706dc00640 1 --2- 172.21.15.114:0/1053334750 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f70480762d0 0x7f70480787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:43.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.719+0000 7f706dc00640 1 --2- 172.21.15.114:0/1053334750 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f70480762d0 0x7f70480787e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f705c000f60 tx=0x7f705c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:43.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.723+0000 7f7065600640 1 -- 172.21.15.114:0/1053334750 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f706003f030 con 0x7f7068159e40 2024-05-27T09:37:43.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.936+0000 7f706f000640 1 -- 172.21.15.114:0/1053334750 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70680c40f0 con 0x7f7068159e40 2024-05-27T09:37:43.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.938+0000 7f7065600640 1 -- 172.21.15.114:0/1053334750 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+17505 (secure 0 0 0) 0x7f7060081d60 con 0x7f7068159e40 2024-05-27T09:37:43.938 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:37:43.939 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":50,"fsid":"2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48","created":"2024-05-27T09:27:00.307278+0000","modified":"2024-05-27T09:37:15.516868+0000","last_up_change":"2024-05-27T09:27:11.463226+0000","last_in_change":"2024-05-27T09:27:02.972880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T09:27:10.796859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T09:27:16.063199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T09:27:17.064857+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":46,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3561c750-1d63-42f6-a6ae-0ee63721e15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6817","nonce":1960550710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6819","nonce":1960550710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6824","nonce":1960550710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6821","nonce":1960550710}]},"public_addr":"172.21.15.114:6817/1960550710","cluster_addr":"172.21.15.114:6819/1960550710","heartbeat_back_addr":"172.21.15.114:6824/1960550710","heartbeat_front_addr":"172.21.15.114:6821/1960550710","state":["exists","up"]},{"osd":1,"uuid":"9a407d04-38c0-4676-928f-d04e8116bd19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6825","nonce":326580057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6827","nonce":326580057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6831","nonce":326580057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6829","nonce":326580057}]},"public_addr":"172.21.15.114:6825/326580057","cluster_addr":"172.21.15.114:6827/326580057","heartbeat_back_addr":"172.21.15.114:6831/326580057","heartbeat_front_addr":"172.21.15.114:6829/326580057","state":["exists","up"]},{"osd":2,"uuid":"255a1cc6-9357-43f7-9db5-d760f4806ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6809","nonce":865568619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6811","nonce":865568619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6815","nonce":865568619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6813","nonce":865568619}]},"public_addr":"172.21.15.114:6809/865568619","cluster_addr":"172.21.15.114:6811/865568619","heartbeat_back_addr":"172.21.15.114:6815/865568619","heartbeat_front_addr":"172.21.15.114:6813/865568619","state":["exists","up"]},{"osd":3,"uuid":"cb4d7692-9a17-41b9-927e-d8eac3e217c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6801","nonce":1601792164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6803","nonce":1601792164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6807","nonce":1601792164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6805","nonce":1601792164}]},"public_addr":"172.21.15.114:6801/1601792164","cluster_addr":"172.21.15.114:6803/1601792164","heartbeat_back_addr":"172.21.15.114:6807/1601792164","heartbeat_front_addr":"172.21.15.114:6805/1601792164","state":["exists","up"]},{"osd":4,"uuid":"cdef1940-3453-4fbd-b0f1-459f5f5c7142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6817","nonce":1982246015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6819","nonce":1982246015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6823","nonce":1982246015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6821","nonce":1982246015}]},"public_addr":"172.21.15.163:6817/1982246015","cluster_addr":"172.21.15.163:6819/1982246015","heartbeat_back_addr":"172.21.15.163:6823/1982246015","heartbeat_front_addr":"172.21.15.163:6821/1982246015","state":["exists","up"]},{"osd":5,"uuid":"220a7b32-abcc-492f-bca8-4615cc2ea111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6801","nonce":1695294004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6803","nonce":1695294004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6807","nonce":1695294004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6805","nonce":1695294004}]},"public_addr":"172.21.15.163:6801/1695294004","cluster_addr":"172.21.15.163:6803/1695294004","heartbeat_back_addr":"172.21.15.163:6807/1695294004","heartbeat_front_addr":"172.21.15.163:6805/1695294004","state":["exists","up"]},{"osd":6,"uuid":"43a74d13-d03f-493a-a6c0-e73b0a056c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6809","nonce":3649718505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6811","nonce":3649718505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6815","nonce":3649718505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6813","nonce":3649718505}]},"public_addr":"172.21.15.163:6809/3649718505","cluster_addr":"172.21.15.163:6811/3649718505","heartbeat_back_addr":"172.21.15.163:6815/3649718505","heartbeat_front_addr":"172.21.15.163:6813/3649718505","state":["exists","up"]},{"osd":7,"uuid":"134629c2-70af-4b84-83a6-7ddf53681baa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6825","nonce":2539451311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6827","nonce":2539451311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6831","nonce":2539451311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6829","nonce":2539451311}]},"public_addr":"172.21.15.163:6825/2539451311","cluster_addr":"172.21.15.163:6827/2539451311","heartbeat_back_addr":"172.21.15.163:6831/2539451311","heartbeat_front_addr":"172.21.15.163:6829/2539451311","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.388942+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.515675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.646761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.400212+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.220072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.079531+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.180419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.175008+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.34","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6835/2747052034":"2024-05-28T09:37:15.516770+0000","172.21.15.114:6833/2747052034":"2024-05-28T09:37:15.516770+0000","172.21.15.163:6837/3371890103":"2024-05-28T09:35:40.437044+0000","172.21.15.114:6837/1448950017":"2024-05-28T09:34:05.407657+0000","172.21.15.163:6839/3371890103":"2024-05-28T09:35:40.437044+0000","172.21.15.114:6834/657027519":"2024-05-28T09:32:30.373988+0000","172.21.15.114:6836/1448950017":"2024-05-28T09:34:05.407657+0000","172.21.15.114:6832/657027519":"2024-05-28T09:32:30.373988+0000","172.21.15.114:6833/1325619883":"2024-05-28T09:31:00.345342+0000","172.21.15.163:6839/444766663":"2024-05-28T09:29:45.323526+0000","172.21.15.114:6835/1325619883":"2024-05-28T09:31:00.345342+0000","172.21.15.163:6837/444766663":"2024-05-28T09:29:45.323526+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T09:37:43.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.941+0000 7f706f000640 1 -- 172.21.15.114:0/1053334750 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f70480762d0 msgr2=0x7f70480787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:43.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.941+0000 7f706f000640 1 --2- 172.21.15.114:0/1053334750 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f70480762d0 0x7f70480787e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f705c000f60 tx=0x7f705c009040 comp rx=0 tx=0).stop 2024-05-27T09:37:43.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.941+0000 7f706f000640 1 -- 172.21.15.114:0/1053334750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7068159e40 msgr2=0x7f706814fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:43.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.941+0000 7f706f000640 1 --2- 172.21.15.114:0/1053334750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7068159e40 0x7f706814fb80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7060031630 tx=0x7f7060004a40 comp rx=0 tx=0).stop 2024-05-27T09:37:43.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.941+0000 7f706f000640 1 -- 172.21.15.114:0/1053334750 shutdown_connections 2024-05-27T09:37:43.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.941+0000 7f706f000640 1 --2- 172.21.15.114:0/1053334750 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f70480762d0 0x7f70480787e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:43.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.941+0000 7f706f000640 1 --2- 172.21.15.114:0/1053334750 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7068159e40 0x7f706814fb80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:43.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.941+0000 7f706f000640 1 --2- 172.21.15.114:0/1053334750 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70681534c0 0x7f7068073ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:43.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.941+0000 7f706f000640 1 --2- 172.21.15.114:0/1053334750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70680c2a00 0x7f70681f2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:43.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.941+0000 7f706f000640 1 -- 172.21.15.114:0/1053334750 >> 172.21.15.114:0/1053334750 conn(0x7f70680ca320 msgr2=0x7f7068157bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:43.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.941+0000 7f706f000640 1 -- 172.21.15.114:0/1053334750 shutdown_connections 2024-05-27T09:37:43.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:43.941+0000 7f706f000640 1 -- 172.21.15.114:0/1053334750 wait complete. 2024-05-27T09:37:44.971 INFO:tasks.ceph:Scrubbing osd.0 2024-05-27T09:37:44.971 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-27T09:37:45.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.256+0000 7fbad9a00640 1 Processor -- start 2024-05-27T09:37:45.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.256+0000 7fbad9a00640 1 -- start start 2024-05-27T09:37:45.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.258+0000 7fbad9a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbad40b2ca0 0x7fbad415fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:45.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.258+0000 7fbad9a00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbad40b2730 0x7fbad4167050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:45.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.258+0000 7fbad9a00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbad40c9090 con 0x7fbad4156c70 2024-05-27T09:37:45.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.258+0000 7fbad9a00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fbad40c96a0 con 0x7fbad40b2ca0 2024-05-27T09:37:45.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.258+0000 7fbad9a00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fbad40c87b0 con 0x7fbad40b2730 2024-05-27T09:37:45.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.258+0000 7fbad2a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbad40b2ca0 0x7fbad415fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:45.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.258+0000 7fbad2a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fbad40b2ca0 0x7fbad415fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37282/0 (socket says 172.21.15.114:37282) 2024-05-27T09:37:45.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.258+0000 7fbad2a00640 1 -- 172.21.15.114:0/1304566218 learned_addr learned my addr 172.21.15.114:0/1304566218 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:45.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.258+0000 7fbad3400640 1 --2- 172.21.15.114:0/1304566218 >> v2:172.21.15.163:3300/0 conn(0x7fbad40b2730 0x7fbad4167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:45.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.259+0000 7fbad2a00640 1 -- 172.21.15.114:0/1304566218 >> v2:172.21.15.163:3300/0 conn(0x7fbad40b2730 msgr2=0x7fbad4167050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:45.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.259+0000 7fbad2a00640 1 --2- 172.21.15.114:0/1304566218 >> v2:172.21.15.163:3300/0 conn(0x7fbad40b2730 0x7fbad4167050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:45.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.259+0000 7fbad2a00640 1 -- 172.21.15.114:0/1304566218 >> v1:172.21.15.163:6789/0 conn(0x7fbad4156c70 legacy=0x7fbad406c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:45.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.259+0000 7fbad2a00640 1 -- 172.21.15.114:0/1304566218 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad40c8a90 con 0x7fbad40b2ca0 2024-05-27T09:37:45.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.260+0000 7fbad2a00640 1 --2- 172.21.15.114:0/1304566218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad40b2ca0 0x7fbad415fc90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbad40c9820 tx=0x7fbac402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce5e0a63c69ebfd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:45.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.261+0000 7fbad1600640 1 -- 172.21.15.114:0/1304566218 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4036070 con 0x7fbad40b2ca0 2024-05-27T09:37:45.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.261+0000 7fbad1600640 1 -- 172.21.15.114:0/1304566218 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac4031600 con 0x7fbad40b2ca0 2024-05-27T09:37:45.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.261+0000 7fbad1600640 1 -- 172.21.15.114:0/1304566218 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac40319d0 con 0x7fbad40b2ca0 2024-05-27T09:37:45.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.261+0000 7fbad9a00640 1 -- 172.21.15.114:0/1304566218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad40b2ca0 msgr2=0x7fbad415fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:45.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.261+0000 7fbad9a00640 1 --2- 172.21.15.114:0/1304566218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad40b2ca0 0x7fbad415fc90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbad40c9820 tx=0x7fbac402a730 comp rx=0 tx=0).stop 2024-05-27T09:37:45.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.261+0000 7fbad9a00640 1 -- 172.21.15.114:0/1304566218 shutdown_connections 2024-05-27T09:37:45.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.261+0000 7fbad9a00640 1 --2- 172.21.15.114:0/1304566218 >> v2:172.21.15.163:3300/0 conn(0x7fbad40b2730 0x7fbad4167050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:45.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.261+0000 7fbad9a00640 1 --2- 172.21.15.114:0/1304566218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad40b2ca0 0x7fbad415fc90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:45.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.261+0000 7fbad9a00640 1 -- 172.21.15.114:0/1304566218 >> 172.21.15.114:0/1304566218 conn(0x7fbad40cd9e0 msgr2=0x7fbad40cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:45.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.262+0000 7fbad9a00640 1 -- 172.21.15.114:0/1304566218 shutdown_connections 2024-05-27T09:37:45.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.262+0000 7fbad9a00640 1 -- 172.21.15.114:0/1304566218 wait complete. 2024-05-27T09:37:45.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.264+0000 7fbad9a00640 1 Processor -- start 2024-05-27T09:37:45.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.264+0000 7fbad9a00640 1 -- start start 2024-05-27T09:37:45.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.265+0000 7fbad9a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbad40b2730 0x7fbad415f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:45.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.266+0000 7fbad9a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbad40b2ca0 0x7fbad41f6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:45.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.266+0000 7fbad9a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad4156c70 0x7fbad414ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:45.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.266+0000 7fbad9a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbad41601d0 con 0x7fbad4156c70 2024-05-27T09:37:45.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.266+0000 7fbad9a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbad40c8d70 con 0x7fbad40b2730 2024-05-27T09:37:45.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.266+0000 7fbad9a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fbad4074bd0 con 0x7fbad40b2ca0 2024-05-27T09:37:45.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.266+0000 7fbad3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad4156c70 0x7fbad414ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:45.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.266+0000 7fbad3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad4156c70 0x7fbad414ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37292/0 (socket says 172.21.15.114:37292) 2024-05-27T09:37:45.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.266+0000 7fbad3400640 1 -- 172.21.15.114:0/2453642067 learned_addr learned my addr 172.21.15.114:0/2453642067 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:45.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.266+0000 7fbad2a00640 1 --2- 172.21.15.114:0/2453642067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbad40b2730 0x7fbad415f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:45.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.266+0000 7fbad2000640 1 --2- 172.21.15.114:0/2453642067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbad40b2ca0 0x7fbad41f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:45.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.266+0000 7fbad3400640 1 -- 172.21.15.114:0/2453642067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbad40b2ca0 msgr2=0x7fbad41f6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:45.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.266+0000 7fbad3400640 1 --2- 172.21.15.114:0/2453642067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbad40b2ca0 0x7fbad41f6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:45.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.266+0000 7fbad3400640 1 -- 172.21.15.114:0/2453642067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbad40b2730 msgr2=0x7fbad415f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:45.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.266+0000 7fbad3400640 1 --2- 172.21.15.114:0/2453642067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbad40b2730 0x7fbad415f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:45.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.266+0000 7fbad3400640 1 -- 172.21.15.114:0/2453642067 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad41f7030 con 0x7fbad4156c70 2024-05-27T09:37:45.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.266+0000 7fbad2a00640 1 --2- 172.21.15.114:0/2453642067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbad40b2730 0x7fbad415f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:45.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.267+0000 7fbad3400640 1 --2- 172.21.15.114:0/2453642067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad4156c70 0x7fbad414ada0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00e820 tx=0x7fbacc00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:45.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.267+0000 7fbacaa00640 1 -- 172.21.15.114:0/2453642067 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc017070 con 0x7fbad4156c70 2024-05-27T09:37:45.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.267+0000 7fbacaa00640 1 -- 172.21.15.114:0/2453642067 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbacc00ee70 con 0x7fbad4156c70 2024-05-27T09:37:45.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.267+0000 7fbad9a00640 1 -- 172.21.15.114:0/2453642067 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad40cd700 con 0x7fbad4156c70 2024-05-27T09:37:45.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.267+0000 7fbacaa00640 1 -- 172.21.15.114:0/2453642067 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc00cb60 con 0x7fbad4156c70 2024-05-27T09:37:45.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.267+0000 7fbad9a00640 1 -- 172.21.15.114:0/2453642067 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad41e74e0 con 0x7fbad4156c70 2024-05-27T09:37:45.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.268+0000 7fbad9a00640 1 -- 172.21.15.114:0/2453642067 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbad406a380 con 0x7fbad4156c70 2024-05-27T09:37:45.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.270+0000 7fbacaa00640 1 -- 172.21.15.114:0/2453642067 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbacc01a070 con 0x7fbad4156c70 2024-05-27T09:37:45.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.270+0000 7fbacaa00640 1 --2- 172.21.15.114:0/2453642067 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbab4076300 0x7fbab4078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:45.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.271+0000 7fbacaa00640 1 -- 172.21.15.114:0/2453642067 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7fbacc0976a0 con 0x7fbad4156c70 2024-05-27T09:37:45.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.271+0000 7fbacaa00640 1 --2- 172.21.15.114:0/2453642067 >> [v2:172.21.15.114:6816/1960550710,v1:172.21.15.114:6817/1960550710] conn(0x7fbab407e970 0x7fbab4080de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:45.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.271+0000 7fbacaa00640 1 -- 172.21.15.114:0/2453642067 --> [v2:172.21.15.114:6816/1960550710,v1:172.21.15.114:6817/1960550710] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbacc022e50 con 0x7fbab407e970 2024-05-27T09:37:45.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.271+0000 7fbacaa00640 1 -- 172.21.15.114:0/2453642067 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fbacc09e050 con 0x7fbad4156c70 2024-05-27T09:37:45.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.271+0000 7fbad2a00640 1 --2- 172.21.15.114:0/2453642067 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbab4076300 0x7fbab4078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:45.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.271+0000 7fbad2000640 1 --2- 172.21.15.114:0/2453642067 >> [v2:172.21.15.114:6816/1960550710,v1:172.21.15.114:6817/1960550710] conn(0x7fbab407e970 0x7fbab4080de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:45.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.272+0000 7fbad2a00640 1 --2- 172.21.15.114:0/2453642067 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbab4076300 0x7fbab4078810 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbac4033000 tx=0x7fbac4009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:45.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.272+0000 7fbad2000640 1 --2- 172.21.15.114:0/2453642067 >> [v2:172.21.15.114:6816/1960550710,v1:172.21.15.114:6817/1960550710] conn(0x7fbab407e970 0x7fbab4080de0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:45.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.273+0000 7fbacaa00640 1 -- 172.21.15.114:0/2453642067 <== osd.0 v2:172.21.15.114:6816/1960550710 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbacc022e50 con 0x7fbab407e970 2024-05-27T09:37:45.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.287+0000 7fbad9a00640 1 -- 172.21.15.114:0/2453642067 --> [v2:172.21.15.114:6816/1960550710,v1:172.21.15.114:6817/1960550710] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbad40747a0 con 0x7fbab407e970 2024-05-27T09:37:45.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.288+0000 7fbacaa00640 1 -- 172.21.15.114:0/2453642067 <== osd.0 v2:172.21.15.114:6816/1960550710 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbad40747a0 con 0x7fbab407e970 2024-05-27T09:37:45.288 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-05-27T09:37:45.288 INFO:teuthology.orchestra.run.smithi114.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-27T09:37:45.288 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-05-27T09:37:45.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.288+0000 7fbad9a00640 1 -- 172.21.15.114:0/2453642067 >> [v2:172.21.15.114:6816/1960550710,v1:172.21.15.114:6817/1960550710] conn(0x7fbab407e970 msgr2=0x7fbab4080de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:45.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.288+0000 7fbad9a00640 1 --2- 172.21.15.114:0/2453642067 >> [v2:172.21.15.114:6816/1960550710,v1:172.21.15.114:6817/1960550710] conn(0x7fbab407e970 0x7fbab4080de0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:45.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.288+0000 7fbad9a00640 1 -- 172.21.15.114:0/2453642067 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbab4076300 msgr2=0x7fbab4078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:45.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.288+0000 7fbad9a00640 1 --2- 172.21.15.114:0/2453642067 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbab4076300 0x7fbab4078810 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbac4033000 tx=0x7fbac4009000 comp rx=0 tx=0).stop 2024-05-27T09:37:45.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.288+0000 7fbad9a00640 1 -- 172.21.15.114:0/2453642067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad4156c70 msgr2=0x7fbad414ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:45.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.288+0000 7fbad9a00640 1 --2- 172.21.15.114:0/2453642067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbad4156c70 0x7fbad414ada0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00e820 tx=0x7fbacc00ec20 comp rx=0 tx=0).stop 2024-05-27T09:37:45.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.288+0000 7fbad3400640 1 -- 172.21.15.114:0/2453642067 reap_dead start 2024-05-27T09:37:45.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.289+0000 7fbad9a00640 1 -- 172.21.15.114:0/2453642067 shutdown_connections 2024-05-27T09:37:45.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.289+0000 7fbad9a00640 1 -- 172.21.15.114:0/2453642067 >> 172.21.15.114:0/2453642067 conn(0x7fbad40cd9e0 msgr2=0x7fbad4152f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:45.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.289+0000 7fbad9a00640 1 -- 172.21.15.114:0/2453642067 shutdown_connections 2024-05-27T09:37:45.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.289+0000 7fbad9a00640 1 -- 172.21.15.114:0/2453642067 wait complete. 2024-05-27T09:37:45.314 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-27T09:37:45.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.587+0000 7f732aa00640 1 Processor -- start 2024-05-27T09:37:45.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.587+0000 7f732aa00640 1 -- start start 2024-05-27T09:37:45.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.589+0000 7f732aa00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73240c2a00 0x7f7324159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:45.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.589+0000 7f732aa00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7324159e40 0x7f7324074140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:45.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.589+0000 7f732aa00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73240c96d0 con 0x7f73241534c0 2024-05-27T09:37:45.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.589+0000 7f732aa00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f7324159190 con 0x7f7324159e40 2024-05-27T09:37:45.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.589+0000 7f732aa00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f73241538d0 con 0x7f73240c2a00 2024-05-27T09:37:45.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.590+0000 7f7329600640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f73241534c0 0x7f73240c8220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55880/0 (socket says 172.21.15.114:55880) 2024-05-27T09:37:45.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.590+0000 7f7329600640 1 -- 172.21.15.114:0/3977860091 learned_addr learned my addr 172.21.15.114:0/3977860091 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:45.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.590+0000 7f7328c00640 1 --2- 172.21.15.114:0/3977860091 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73240c2a00 0x7f7324159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:45.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.590+0000 7f732a000640 1 --2- 172.21.15.114:0/3977860091 >> v2:172.21.15.163:3300/0 conn(0x7f7324159e40 0x7f7324074140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:45.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.590+0000 7f732a000640 1 -- 172.21.15.114:0/3977860091 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73240c2a00 msgr2=0x7f7324159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:45.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.590+0000 7f732a000640 1 --2- 172.21.15.114:0/3977860091 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73240c2a00 0x7f7324159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:45.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.590+0000 7f732a000640 1 -- 172.21.15.114:0/3977860091 >> v1:172.21.15.114:6789/0 conn(0x7f73241534c0 legacy=0x7f73240c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:45.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.590+0000 7f732a000640 1 -- 172.21.15.114:0/3977860091 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73240c8930 con 0x7f7324159e40 2024-05-27T09:37:45.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.590+0000 7f7323e00640 1 -- 172.21.15.114:0/3977860091 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2712315709 0 0) 0x7f73240c96d0 con 0x7f73241534c0 2024-05-27T09:37:45.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.590+0000 7f7328c00640 1 --2- 172.21.15.114:0/3977860091 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73240c2a00 0x7f7324159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:45.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.591+0000 7f732a000640 1 --2- 172.21.15.114:0/3977860091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7324159e40 0x7f7324074140 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f731c009d90 tx=0x7f731c02c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=472d92d865528bb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:45.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.592+0000 7f7323e00640 1 -- 172.21.15.114:0/3977860091 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c032770 con 0x7f7324159e40 2024-05-27T09:37:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.592+0000 7f7323e00640 1 -- 172.21.15.114:0/3977860091 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f731c038070 con 0x7f7324159e40 2024-05-27T09:37:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.592+0000 7f7323e00640 1 -- 172.21.15.114:0/3977860091 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c032b30 con 0x7f7324159e40 2024-05-27T09:37:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.593+0000 7f732aa00640 1 -- 172.21.15.114:0/3977860091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7324159e40 msgr2=0x7f7324074140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.593+0000 7f732aa00640 1 --2- 172.21.15.114:0/3977860091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7324159e40 0x7f7324074140 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f731c009d90 tx=0x7f731c02c870 comp rx=0 tx=0).stop 2024-05-27T09:37:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.593+0000 7f732aa00640 1 -- 172.21.15.114:0/3977860091 shutdown_connections 2024-05-27T09:37:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.593+0000 7f732aa00640 1 --2- 172.21.15.114:0/3977860091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7324159e40 0x7f7324074140 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:45.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.593+0000 7f732aa00640 1 --2- 172.21.15.114:0/3977860091 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73240c2a00 0x7f7324159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:45.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.593+0000 7f732aa00640 1 -- 172.21.15.114:0/3977860091 >> 172.21.15.114:0/3977860091 conn(0x7f73240ca320 msgr2=0x7f73240ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:45.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.593+0000 7f732aa00640 1 -- 172.21.15.114:0/3977860091 shutdown_connections 2024-05-27T09:37:45.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.593+0000 7f732aa00640 1 -- 172.21.15.114:0/3977860091 wait complete. 2024-05-27T09:37:45.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.596+0000 7f732aa00640 1 Processor -- start 2024-05-27T09:37:45.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.596+0000 7f732aa00640 1 -- start start 2024-05-27T09:37:45.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.597+0000 7f732aa00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73240c2a00 0x7f73241f2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:45.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.597+0000 7f732aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73241534c0 0x7f7324075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:45.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.597+0000 7f732aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7324159e40 0x7f732414fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:45.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.598+0000 7f732aa00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f73240c9f30 con 0x7f73241534c0 2024-05-27T09:37:45.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.598+0000 7f732aa00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f73240c9c50 con 0x7f7324159e40 2024-05-27T09:37:45.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.598+0000 7f732aa00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f73241593e0 con 0x7f73240c2a00 2024-05-27T09:37:45.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.598+0000 7f7328c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73241534c0 0x7f7324075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:45.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.598+0000 7f7328c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73241534c0 0x7f7324075f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37298/0 (socket says 172.21.15.114:37298) 2024-05-27T09:37:45.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.598+0000 7f7328c00640 1 -- 172.21.15.114:0/3520918258 learned_addr learned my addr 172.21.15.114:0/3520918258 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:45.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.598+0000 7f7329600640 1 --2- 172.21.15.114:0/3520918258 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73240c2a00 0x7f73241f2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:45.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.598+0000 7f732a000640 1 --2- 172.21.15.114:0/3520918258 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7324159e40 0x7f732414fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:45.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.598+0000 7f732a000640 1 -- 172.21.15.114:0/3520918258 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73240c2a00 msgr2=0x7f73241f2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:45.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.598+0000 7f732a000640 1 --2- 172.21.15.114:0/3520918258 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73240c2a00 0x7f73241f2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:45.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.598+0000 7f732a000640 1 -- 172.21.15.114:0/3520918258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73241534c0 msgr2=0x7f7324075f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:45.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.598+0000 7f732a000640 1 --2- 172.21.15.114:0/3520918258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73241534c0 0x7f7324075f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:45.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.598+0000 7f732a000640 1 -- 172.21.15.114:0/3520918258 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73241f2590 con 0x7f7324159e40 2024-05-27T09:37:45.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.598+0000 7f7328c00640 1 --2- 172.21.15.114:0/3520918258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73241534c0 0x7f7324075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:45.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.599+0000 7f732a000640 1 --2- 172.21.15.114:0/3520918258 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7324159e40 0x7f732414fef0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7324072d20 tx=0x7f731c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:45.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.599+0000 7f7321600640 1 -- 172.21.15.114:0/3520918258 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c009450 con 0x7f7324159e40 2024-05-27T09:37:45.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.599+0000 7f7321600640 1 -- 172.21.15.114:0/3520918258 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f731c033830 con 0x7f7324159e40 2024-05-27T09:37:45.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.599+0000 7f7321600640 1 -- 172.21.15.114:0/3520918258 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c033c20 con 0x7f7324159e40 2024-05-27T09:37:45.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.599+0000 7f732aa00640 1 -- 172.21.15.114:0/3520918258 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73241ee8e0 con 0x7f7324159e40 2024-05-27T09:37:45.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.599+0000 7f732aa00640 1 -- 172.21.15.114:0/3520918258 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73241eee60 con 0x7f7324159e40 2024-05-27T09:37:45.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.601+0000 7f732aa00640 1 -- 172.21.15.114:0/3520918258 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7324050860 con 0x7f7324159e40 2024-05-27T09:37:45.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.602+0000 7f7321600640 1 -- 172.21.15.114:0/3520918258 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f731c032070 con 0x7f7324159e40 2024-05-27T09:37:45.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.603+0000 7f7321600640 1 --2- 172.21.15.114:0/3520918258 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f73040762d0 0x7f73040787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:45.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.603+0000 7f7321600640 1 -- 172.21.15.114:0/3520918258 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7f731c0b8560 con 0x7f7324159e40 2024-05-27T09:37:45.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.603+0000 7f7329600640 1 --2- 172.21.15.114:0/3520918258 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f73040762d0 0x7f73040787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:45.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.604+0000 7f7329600640 1 --2- 172.21.15.114:0/3520918258 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f73040762d0 0x7f73040787e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7318007fd0 tx=0x7f7318007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:45.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.608+0000 7f7321600640 1 -- 172.21.15.114:0/3520918258 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f731c081520 con 0x7f7324159e40 2024-05-27T09:37:45.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.813+0000 7f732aa00640 1 -- 172.21.15.114:0/3520918258 --> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f73240701e0 con 0x7f73040762d0 2024-05-27T09:37:45.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.814+0000 7f7321600640 1 -- 172.21.15.114:0/3520918258 <== mgr.4108 v2:172.21.15.163:6832/1122018204 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f73240701e0 con 0x7f73040762d0 2024-05-27T09:37:45.814 INFO:teuthology.orchestra.run.smithi114.stderr:instructed osd(s) 0 to deep-scrub 2024-05-27T09:37:45.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.817+0000 7f732aa00640 1 -- 172.21.15.114:0/3520918258 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f73040762d0 msgr2=0x7f73040787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:45.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.817+0000 7f732aa00640 1 --2- 172.21.15.114:0/3520918258 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f73040762d0 0x7f73040787e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7318007fd0 tx=0x7f7318007de0 comp rx=0 tx=0).stop 2024-05-27T09:37:45.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.817+0000 7f732aa00640 1 -- 172.21.15.114:0/3520918258 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7324159e40 msgr2=0x7f732414fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:45.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.817+0000 7f732aa00640 1 --2- 172.21.15.114:0/3520918258 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7324159e40 0x7f732414fef0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7324072d20 tx=0x7f731c004ab0 comp rx=0 tx=0).stop 2024-05-27T09:37:45.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.817+0000 7f732aa00640 1 -- 172.21.15.114:0/3520918258 shutdown_connections 2024-05-27T09:37:45.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.817+0000 7f732aa00640 1 --2- 172.21.15.114:0/3520918258 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f73040762d0 0x7f73040787e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:45.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.817+0000 7f732aa00640 1 --2- 172.21.15.114:0/3520918258 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7324159e40 0x7f732414fef0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:45.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.817+0000 7f732aa00640 1 --2- 172.21.15.114:0/3520918258 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73241534c0 0x7f7324075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:45.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.817+0000 7f732aa00640 1 --2- 172.21.15.114:0/3520918258 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73240c2a00 0x7f73241f2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:45.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.817+0000 7f732aa00640 1 -- 172.21.15.114:0/3520918258 >> 172.21.15.114:0/3520918258 conn(0x7f73240ca320 msgr2=0x7f7324151350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:45.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.818+0000 7f732aa00640 1 -- 172.21.15.114:0/3520918258 shutdown_connections 2024-05-27T09:37:45.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:45.818+0000 7f732aa00640 1 -- 172.21.15.114:0/3520918258 wait complete. 2024-05-27T09:37:45.847 INFO:tasks.ceph:Scrubbing osd.1 2024-05-27T09:37:45.847 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-27T09:37:46.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.126+0000 7fe8ae000640 1 Processor -- start 2024-05-27T09:37:46.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.126+0000 7fe8ae000640 1 -- start start 2024-05-27T09:37:46.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.128+0000 7fe8ae000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe8a8154820 0x7fe8a80c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:46.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.128+0000 7fe8ae000640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8a806fd10 con 0x7fe8a8070150 2024-05-27T09:37:46.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.128+0000 7fe8ae000640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8a806f470 con 0x7fe8a8073e50 2024-05-27T09:37:46.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.128+0000 7fe8ae000640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fe8a8154c30 con 0x7fe8a8154820 2024-05-27T09:37:46.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.128+0000 7fe8acc00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fe8a8070150 0x7fe8a8073740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55886/0 (socket says 172.21.15.114:55886) 2024-05-27T09:37:46.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.129+0000 7fe8acc00640 1 -- 172.21.15.114:0/196476487 learned_addr learned my addr 172.21.15.114:0/196476487 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:46.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.129+0000 7fe8ad600640 1 --2- 172.21.15.114:0/196476487 >> v2:172.21.15.163:3300/0 conn(0x7fe8a8154820 0x7fe8a80c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:46.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.129+0000 7fe8a7400640 1 -- 172.21.15.114:0/196476487 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1291951485 0 0) 0x7fe8a806fd10 con 0x7fe8a8070150 2024-05-27T09:37:46.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.129+0000 7fe8a7400640 1 -- 172.21.15.114:0/196476487 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe890003520 con 0x7fe8a8070150 2024-05-27T09:37:46.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.129+0000 7fe8ad600640 1 -- 172.21.15.114:0/196476487 >> v1:172.21.15.163:6789/0 conn(0x7fe8a8073e50 legacy=0x7fe8a80c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:46.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.129+0000 7fe8a7400640 1 -- 172.21.15.114:0/196476487 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3920494618 0 0) 0x7fe8a806f470 con 0x7fe8a8073e50 2024-05-27T09:37:46.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.129+0000 7fe8ad600640 1 -- 172.21.15.114:0/196476487 >> v1:172.21.15.114:6789/0 conn(0x7fe8a8070150 legacy=0x7fe8a8073740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:46.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.129+0000 7fe8ad600640 1 -- 172.21.15.114:0/196476487 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a806f750 con 0x7fe8a8154820 2024-05-27T09:37:46.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.130+0000 7fe8ad600640 1 --2- 172.21.15.114:0/196476487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8a8154820 0x7fe8a80c6dd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009d90 tx=0x7fe8a002a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d18ce648ac9511e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:46.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.131+0000 7fe8a7400640 1 -- 172.21.15.114:0/196476487 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0037070 con 0x7fe8a8154820 2024-05-27T09:37:46.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.131+0000 7fe8a7400640 1 -- 172.21.15.114:0/196476487 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8a0007740 con 0x7fe8a8154820 2024-05-27T09:37:46.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.131+0000 7fe8a7400640 1 -- 172.21.15.114:0/196476487 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0007b10 con 0x7fe8a8154820 2024-05-27T09:37:46.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.131+0000 7fe8ae000640 1 -- 172.21.15.114:0/196476487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8a8154820 msgr2=0x7fe8a80c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:46.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.131+0000 7fe8ae000640 1 --2- 172.21.15.114:0/196476487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8a8154820 0x7fe8a80c6dd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009d90 tx=0x7fe8a002a780 comp rx=0 tx=0).stop 2024-05-27T09:37:46.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.131+0000 7fe8ae000640 1 -- 172.21.15.114:0/196476487 shutdown_connections 2024-05-27T09:37:46.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.131+0000 7fe8ae000640 1 --2- 172.21.15.114:0/196476487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8a8154820 0x7fe8a80c6dd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:46.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.131+0000 7fe8ae000640 1 -- 172.21.15.114:0/196476487 >> 172.21.15.114:0/196476487 conn(0x7fe8a806c690 msgr2=0x7fe8a806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:46.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.132+0000 7fe8ae000640 1 -- 172.21.15.114:0/196476487 shutdown_connections 2024-05-27T09:37:46.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.132+0000 7fe8ae000640 1 -- 172.21.15.114:0/196476487 wait complete. 2024-05-27T09:37:46.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.134+0000 7fe8ae000640 1 Processor -- start 2024-05-27T09:37:46.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.134+0000 7fe8ae000640 1 -- start start 2024-05-27T09:37:46.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.135+0000 7fe8ae000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8a8070150 0x7fe8a814ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:46.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.135+0000 7fe8ae000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8a8073e50 0x7fe8a815c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:46.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.136+0000 7fe8ae000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8a8154820 0x7fe8a81f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:46.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.136+0000 7fe8ae000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a815b320 con 0x7fe8a8070150 2024-05-27T09:37:46.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.136+0000 7fe8ae000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a8073220 con 0x7fe8a8154820 2024-05-27T09:37:46.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.136+0000 7fe8ae000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fe8a815aee0 con 0x7fe8a8073e50 2024-05-27T09:37:46.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.136+0000 7fe8acc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8a8070150 0x7fe8a814ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:46.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.136+0000 7fe8acc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8a8070150 0x7fe8a814ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37302/0 (socket says 172.21.15.114:37302) 2024-05-27T09:37:46.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.136+0000 7fe8acc00640 1 -- 172.21.15.114:0/1656155397 learned_addr learned my addr 172.21.15.114:0/1656155397 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:46.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.136+0000 7fe8ad600640 1 --2- 172.21.15.114:0/1656155397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8a8154820 0x7fe8a81f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:46.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.136+0000 7fe8a7e00640 1 --2- 172.21.15.114:0/1656155397 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8a8073e50 0x7fe8a815c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:46.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.136+0000 7fe8acc00640 1 -- 172.21.15.114:0/1656155397 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8a8073e50 msgr2=0x7fe8a815c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:46.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.136+0000 7fe8acc00640 1 --2- 172.21.15.114:0/1656155397 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8a8073e50 0x7fe8a815c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:46.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.136+0000 7fe8acc00640 1 -- 172.21.15.114:0/1656155397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8a8154820 msgr2=0x7fe8a81f3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:46.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.136+0000 7fe8acc00640 1 --2- 172.21.15.114:0/1656155397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8a8154820 0x7fe8a81f3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:46.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.136+0000 7fe8acc00640 1 -- 172.21.15.114:0/1656155397 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a815c590 con 0x7fe8a8070150 2024-05-27T09:37:46.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.136+0000 7fe8a7e00640 1 --2- 172.21.15.114:0/1656155397 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8a8073e50 0x7fe8a815c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:37:46.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.137+0000 7fe8acc00640 1 --2- 172.21.15.114:0/1656155397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8a8070150 0x7fe8a814ca00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00c410 tx=0x7fe89c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:46.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.137+0000 7fe8a4c00640 1 -- 172.21.15.114:0/1656155397 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c017070 con 0x7fe8a8070150 2024-05-27T09:37:46.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.137+0000 7fe8a4c00640 1 -- 172.21.15.114:0/1656155397 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe89c005ce0 con 0x7fe8a8070150 2024-05-27T09:37:46.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.137+0000 7fe8ae000640 1 -- 172.21.15.114:0/1656155397 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a81f4390 con 0x7fe8a8070150 2024-05-27T09:37:46.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.137+0000 7fe8a4c00640 1 -- 172.21.15.114:0/1656155397 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c0072d0 con 0x7fe8a8070150 2024-05-27T09:37:46.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.137+0000 7fe8ae000640 1 -- 172.21.15.114:0/1656155397 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a81525f0 con 0x7fe8a8070150 2024-05-27T09:37:46.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.138+0000 7fe8ae000640 1 -- 172.21.15.114:0/1656155397 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe8a806a380 con 0x7fe8a8070150 2024-05-27T09:37:46.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.139+0000 7fe8a4c00640 1 -- 172.21.15.114:0/1656155397 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe89c01a070 con 0x7fe8a8070150 2024-05-27T09:37:46.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.140+0000 7fe8a4c00640 1 --2- 172.21.15.114:0/1656155397 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe88c0762d0 0x7fe88c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:46.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.141+0000 7fe8a4c00640 1 -- 172.21.15.114:0/1656155397 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7fe89c096b10 con 0x7fe8a8070150 2024-05-27T09:37:46.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.141+0000 7fe8a7e00640 1 --2- 172.21.15.114:0/1656155397 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe88c0762d0 0x7fe88c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:46.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.141+0000 7fe8a4c00640 1 --2- 172.21.15.114:0/1656155397 >> [v2:172.21.15.114:6823/326580057,v1:172.21.15.114:6825/326580057] conn(0x7fe88c07e940 0x7fe88c080db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:46.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.141+0000 7fe8a4c00640 1 -- 172.21.15.114:0/1656155397 --> [v2:172.21.15.114:6823/326580057,v1:172.21.15.114:6825/326580057] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe88c081480 con 0x7fe88c07e940 2024-05-27T09:37:46.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.141+0000 7fe8a4c00640 1 -- 172.21.15.114:0/1656155397 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fe89c09b2a0 con 0x7fe8a8070150 2024-05-27T09:37:46.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.141+0000 7fe8ad600640 1 --2- 172.21.15.114:0/1656155397 >> [v2:172.21.15.114:6823/326580057,v1:172.21.15.114:6825/326580057] conn(0x7fe88c07e940 0x7fe88c080db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:46.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.141+0000 7fe8a7e00640 1 --2- 172.21.15.114:0/1656155397 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe88c0762d0 0x7fe88c0787e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe898000c00 tx=0x7fe898009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:46.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.142+0000 7fe8ad600640 1 --2- 172.21.15.114:0/1656155397 >> [v2:172.21.15.114:6823/326580057,v1:172.21.15.114:6825/326580057] conn(0x7fe88c07e940 0x7fe88c080db0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:46.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.143+0000 7fe8a4c00640 1 -- 172.21.15.114:0/1656155397 <== osd.1 v2:172.21.15.114:6823/326580057 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe88c081480 con 0x7fe88c07e940 2024-05-27T09:37:46.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.157+0000 7fe8ae000640 1 -- 172.21.15.114:0/1656155397 --> [v2:172.21.15.114:6823/326580057,v1:172.21.15.114:6825/326580057] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe8a81f3a70 con 0x7fe88c07e940 2024-05-27T09:37:46.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.158+0000 7fe8a4c00640 1 -- 172.21.15.114:0/1656155397 <== osd.1 v2:172.21.15.114:6823/326580057 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe8a81f3a70 con 0x7fe88c07e940 2024-05-27T09:37:46.159 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-05-27T09:37:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-27T09:37:46.159 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-05-27T09:37:46.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.158+0000 7fe8ae000640 1 -- 172.21.15.114:0/1656155397 >> [v2:172.21.15.114:6823/326580057,v1:172.21.15.114:6825/326580057] conn(0x7fe88c07e940 msgr2=0x7fe88c080db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:46.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.158+0000 7fe8ae000640 1 --2- 172.21.15.114:0/1656155397 >> [v2:172.21.15.114:6823/326580057,v1:172.21.15.114:6825/326580057] conn(0x7fe88c07e940 0x7fe88c080db0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:46.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.158+0000 7fe8ae000640 1 -- 172.21.15.114:0/1656155397 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe88c0762d0 msgr2=0x7fe88c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:46.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.158+0000 7fe8ae000640 1 --2- 172.21.15.114:0/1656155397 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe88c0762d0 0x7fe88c0787e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe898000c00 tx=0x7fe898009040 comp rx=0 tx=0).stop 2024-05-27T09:37:46.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.158+0000 7fe8ae000640 1 -- 172.21.15.114:0/1656155397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8a8070150 msgr2=0x7fe8a814ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:46.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.158+0000 7fe8ae000640 1 --2- 172.21.15.114:0/1656155397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8a8070150 0x7fe8a814ca00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00c410 tx=0x7fe89c00c8d0 comp rx=0 tx=0).stop 2024-05-27T09:37:46.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.158+0000 7fe8ad600640 1 -- 172.21.15.114:0/1656155397 reap_dead start 2024-05-27T09:37:46.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.158+0000 7fe8ae000640 1 -- 172.21.15.114:0/1656155397 shutdown_connections 2024-05-27T09:37:46.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.158+0000 7fe8ae000640 1 -- 172.21.15.114:0/1656155397 >> 172.21.15.114:0/1656155397 conn(0x7fe8a806c690 msgr2=0x7fe8a80cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:46.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.159+0000 7fe8ae000640 1 -- 172.21.15.114:0/1656155397 shutdown_connections 2024-05-27T09:37:46.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.159+0000 7fe8ae000640 1 -- 172.21.15.114:0/1656155397 wait complete. 2024-05-27T09:37:46.185 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-27T09:37:46.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.460+0000 7f910da00640 1 Processor -- start 2024-05-27T09:37:46.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.460+0000 7f910da00640 1 -- start start 2024-05-27T09:37:46.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.462+0000 7f910da00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9108154820 0x7f91080ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:46.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.462+0000 7f910da00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f91080ca810 0x7f910815dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:46.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.462+0000 7f910da00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91080cac20 con 0x7f9108074a60 2024-05-27T09:37:46.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.462+0000 7f910da00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f9108154c30 con 0x7f91080ca810 2024-05-27T09:37:46.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.462+0000 7f910da00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f910815db60 con 0x7f9108154820 2024-05-27T09:37:46.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.462+0000 7f9107400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f91080ca810 0x7f910815dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:46.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.462+0000 7f9106000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9108154820 0x7f91080ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:46.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.462+0000 7f9106a00640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f9108074a60 0x7f9108073260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:52838/0 (socket says 172.21.15.114:52838) 2024-05-27T09:37:46.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.463+0000 7f9106a00640 1 -- 172.21.15.114:0/3826063889 learned_addr learned my addr 172.21.15.114:0/3826063889 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:46.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.463+0000 7f9106000640 1 -- 172.21.15.114:0/3826063889 >> v2:172.21.15.114:3300/0 conn(0x7f91080ca810 msgr2=0x7f910815dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:46.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.463+0000 7f9106000640 1 --2- 172.21.15.114:0/3826063889 >> v2:172.21.15.114:3300/0 conn(0x7f91080ca810 0x7f910815dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:46.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.463+0000 7f9106000640 1 -- 172.21.15.114:0/3826063889 >> v1:172.21.15.163:6789/0 conn(0x7f9108074a60 legacy=0x7f9108073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:46.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.463+0000 7f9106000640 1 -- 172.21.15.114:0/3826063889 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910815a810 con 0x7f9108154820 2024-05-27T09:37:46.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.464+0000 7f9106000640 1 --2- 172.21.15.114:0/3826063889 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9108154820 0x7f91080ca2d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f90f40097a0 tx=0x7f90f402a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c986971c203e614c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:46.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.464+0000 7f9105600640 1 -- 172.21.15.114:0/3826063889 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f4037070 con 0x7f9108154820 2024-05-27T09:37:46.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.464+0000 7f9105600640 1 -- 172.21.15.114:0/3826063889 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f4032440 con 0x7f9108154820 2024-05-27T09:37:46.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.464+0000 7f9105600640 1 -- 172.21.15.114:0/3826063889 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f4032810 con 0x7f9108154820 2024-05-27T09:37:46.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.465+0000 7f910da00640 1 -- 172.21.15.114:0/3826063889 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9108154820 msgr2=0x7f91080ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:46.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.465+0000 7f910da00640 1 --2- 172.21.15.114:0/3826063889 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9108154820 0x7f91080ca2d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f90f40097a0 tx=0x7f90f402a230 comp rx=0 tx=0).stop 2024-05-27T09:37:46.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.465+0000 7f910da00640 1 -- 172.21.15.114:0/3826063889 shutdown_connections 2024-05-27T09:37:46.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.465+0000 7f910da00640 1 --2- 172.21.15.114:0/3826063889 >> v2:172.21.15.114:3300/0 conn(0x7f91080ca810 0x7f910815dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:46.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.465+0000 7f910da00640 1 --2- 172.21.15.114:0/3826063889 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9108154820 0x7f91080ca2d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:46.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.465+0000 7f910da00640 1 -- 172.21.15.114:0/3826063889 >> 172.21.15.114:0/3826063889 conn(0x7f910806f6e0 msgr2=0x7f9108070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:46.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.466+0000 7f910da00640 1 -- 172.21.15.114:0/3826063889 shutdown_connections 2024-05-27T09:37:46.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.466+0000 7f910da00640 1 -- 172.21.15.114:0/3826063889 wait complete. 2024-05-27T09:37:46.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.468+0000 7f910da00640 1 Processor -- start 2024-05-27T09:37:46.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.469+0000 7f910da00640 1 -- start start 2024-05-27T09:37:46.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.470+0000 7f910da00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9108074a60 0x7f91081e5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:46.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.470+0000 7f910da00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91080ca810 0x7f910815cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:46.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.470+0000 7f910da00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9108154820 0x7f910813f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:46.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.470+0000 7f910da00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f910815b0b0 con 0x7f9108074a60 2024-05-27T09:37:46.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.470+0000 7f910da00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f910815add0 con 0x7f9108154820 2024-05-27T09:37:46.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.470+0000 7f910da00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f910815b390 con 0x7f91080ca810 2024-05-27T09:37:46.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.470+0000 7f9106a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9108074a60 0x7f91081e5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:46.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.471+0000 7f9106000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91080ca810 0x7f910815cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:46.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.471+0000 7f9107400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9108154820 0x7f910813f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:46.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.471+0000 7f9106a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9108074a60 0x7f91081e5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37308/0 (socket says 172.21.15.114:37308) 2024-05-27T09:37:46.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.471+0000 7f9106a00640 1 -- 172.21.15.114:0/1211622213 learned_addr learned my addr 172.21.15.114:0/1211622213 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:46.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.471+0000 7f9106000640 1 -- 172.21.15.114:0/1211622213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9108154820 msgr2=0x7f910813f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:46.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.471+0000 7f9106000640 1 --2- 172.21.15.114:0/1211622213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9108154820 0x7f910813f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:46.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.471+0000 7f9106000640 1 -- 172.21.15.114:0/1211622213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9108074a60 msgr2=0x7f91081e5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:46.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.471+0000 7f9106000640 1 --2- 172.21.15.114:0/1211622213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9108074a60 0x7f91081e5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:46.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.471+0000 7f9106000640 1 -- 172.21.15.114:0/1211622213 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91081e5b50 con 0x7f91080ca810 2024-05-27T09:37:46.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.471+0000 7f9106a00640 1 --2- 172.21.15.114:0/1211622213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9108074a60 0x7f91081e5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:46.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.471+0000 7f9107400640 1 --2- 172.21.15.114:0/1211622213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9108154820 0x7f910813f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:46.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.472+0000 7f9106000640 1 --2- 172.21.15.114:0/1211622213 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91080ca810 0x7f910815cc80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f90f4031000 tx=0x7f90f40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:46.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.472+0000 7f90fea00640 1 -- 172.21.15.114:0/1211622213 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f4041030 con 0x7f91080ca810 2024-05-27T09:37:46.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.472+0000 7f90fea00640 1 -- 172.21.15.114:0/1211622213 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f4030500 con 0x7f91080ca810 2024-05-27T09:37:46.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.472+0000 7f910da00640 1 -- 172.21.15.114:0/1211622213 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91081e6be0 con 0x7f91080ca810 2024-05-27T09:37:46.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.472+0000 7f90fea00640 1 -- 172.21.15.114:0/1211622213 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f4037040 con 0x7f91080ca810 2024-05-27T09:37:46.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.472+0000 7f910da00640 1 -- 172.21.15.114:0/1211622213 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f91081f4a20 con 0x7f91080ca810 2024-05-27T09:37:46.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.473+0000 7f910da00640 1 -- 172.21.15.114:0/1211622213 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91080764f0 con 0x7f91080ca810 2024-05-27T09:37:46.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.475+0000 7f90fea00640 1 -- 172.21.15.114:0/1211622213 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f90f4044070 con 0x7f91080ca810 2024-05-27T09:37:46.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.476+0000 7f90fea00640 1 --2- 172.21.15.114:0/1211622213 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f90e8076300 0x7f90e8078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:46.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.476+0000 7f90fea00640 1 -- 172.21.15.114:0/1211622213 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7f90f40b8520 con 0x7f91080ca810 2024-05-27T09:37:46.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.476+0000 7f9106a00640 1 --2- 172.21.15.114:0/1211622213 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f90e8076300 0x7f90e8078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:46.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.477+0000 7f9106a00640 1 --2- 172.21.15.114:0/1211622213 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f90e8076300 0x7f90e8078810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f90f8000f60 tx=0x7f90f8007f60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:46.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.480+0000 7f90fea00640 1 -- 172.21.15.114:0/1211622213 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f91080764f0 con 0x7f91080ca810 2024-05-27T09:37:46.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.699+0000 7f910da00640 1 -- 172.21.15.114:0/1211622213 --> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f9108152f50 con 0x7f90e8076300 2024-05-27T09:37:46.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.700+0000 7f90fea00640 1 -- 172.21.15.114:0/1211622213 <== mgr.4108 v2:172.21.15.163:6832/1122018204 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9108152f50 con 0x7f90e8076300 2024-05-27T09:37:46.701 INFO:teuthology.orchestra.run.smithi114.stderr:instructed osd(s) 1 to deep-scrub 2024-05-27T09:37:46.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.703+0000 7f910da00640 1 -- 172.21.15.114:0/1211622213 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f90e8076300 msgr2=0x7f90e8078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:46.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.703+0000 7f910da00640 1 --2- 172.21.15.114:0/1211622213 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f90e8076300 0x7f90e8078810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f90f8000f60 tx=0x7f90f8007f60 comp rx=0 tx=0).stop 2024-05-27T09:37:46.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.703+0000 7f910da00640 1 -- 172.21.15.114:0/1211622213 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91080ca810 msgr2=0x7f910815cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:46.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.703+0000 7f910da00640 1 --2- 172.21.15.114:0/1211622213 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91080ca810 0x7f910815cc80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f90f4031000 tx=0x7f90f40046c0 comp rx=0 tx=0).stop 2024-05-27T09:37:46.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.703+0000 7f910da00640 1 -- 172.21.15.114:0/1211622213 shutdown_connections 2024-05-27T09:37:46.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.703+0000 7f910da00640 1 --2- 172.21.15.114:0/1211622213 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f90e8076300 0x7f90e8078810 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:46.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.703+0000 7f910da00640 1 --2- 172.21.15.114:0/1211622213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9108154820 0x7f910813f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:46.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.703+0000 7f910da00640 1 --2- 172.21.15.114:0/1211622213 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91080ca810 0x7f910815cc80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:46.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.703+0000 7f910da00640 1 --2- 172.21.15.114:0/1211622213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9108074a60 0x7f91081e5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:46.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.703+0000 7f910da00640 1 -- 172.21.15.114:0/1211622213 >> 172.21.15.114:0/1211622213 conn(0x7f910806f6e0 msgr2=0x7f91081525b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:46.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.703+0000 7f910da00640 1 -- 172.21.15.114:0/1211622213 shutdown_connections 2024-05-27T09:37:46.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:46.703+0000 7f910da00640 1 -- 172.21.15.114:0/1211622213 wait complete. 2024-05-27T09:37:46.731 INFO:tasks.ceph:Scrubbing osd.2 2024-05-27T09:37:46.731 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-27T09:37:47.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.007+0000 7f5343a00640 1 Processor -- start 2024-05-27T09:37:47.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.007+0000 7f5343a00640 1 -- start start 2024-05-27T09:37:47.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.009+0000 7f5343a00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f533c0c9830 0x7f533c06c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:47.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.009+0000 7f5343a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f533c073260 0x7f533c06cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:47.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.009+0000 7f5343a00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f533c074cb0 con 0x7f533c15a830 2024-05-27T09:37:47.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.009+0000 7f5343a00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f533c073ee0 con 0x7f533c073260 2024-05-27T09:37:47.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.009+0000 7f5343a00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f533c074130 con 0x7f533c0c9830 2024-05-27T09:37:47.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.010+0000 7f5341a00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f533c15a830 0x7f533c15ac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55902/0 (socket says 172.21.15.114:55902) 2024-05-27T09:37:47.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.010+0000 7f5341a00640 1 -- 172.21.15.114:0/587517974 learned_addr learned my addr 172.21.15.114:0/587517974 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:47.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.010+0000 7f533be00640 1 --2- 172.21.15.114:0/587517974 >> v2:172.21.15.114:3300/0 conn(0x7f533c073260 0x7f533c06cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:47.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.010+0000 7f5341000640 1 --2- 172.21.15.114:0/587517974 >> v2:172.21.15.163:3300/0 conn(0x7f533c0c9830 0x7f533c06c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:47.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.010+0000 7f5341000640 1 -- 172.21.15.114:0/587517974 >> v2:172.21.15.114:3300/0 conn(0x7f533c073260 msgr2=0x7f533c06cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:47.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.010+0000 7f5341000640 1 --2- 172.21.15.114:0/587517974 >> v2:172.21.15.114:3300/0 conn(0x7f533c073260 0x7f533c06cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:47.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.010+0000 7f533b400640 1 -- 172.21.15.114:0/587517974 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 275299664 0 0) 0x7f533c074cb0 con 0x7f533c15a830 2024-05-27T09:37:47.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.010+0000 7f5341000640 1 -- 172.21.15.114:0/587517974 >> v1:172.21.15.114:6789/0 conn(0x7f533c15a830 legacy=0x7f533c15ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:47.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.010+0000 7f5341000640 1 -- 172.21.15.114:0/587517974 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533c074410 con 0x7f533c0c9830 2024-05-27T09:37:47.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.012+0000 7f5341000640 1 --2- 172.21.15.114:0/587517974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f533c0c9830 0x7f533c06c690 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f533c0ce740 tx=0x7f533002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abe8e7365fa676c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:47.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.012+0000 7f533b400640 1 -- 172.21.15.114:0/587517974 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5330037070 con 0x7f533c0c9830 2024-05-27T09:37:47.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.012+0000 7f533b400640 1 -- 172.21.15.114:0/587517974 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5330032440 con 0x7f533c0c9830 2024-05-27T09:37:47.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.012+0000 7f533b400640 1 -- 172.21.15.114:0/587517974 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5330032810 con 0x7f533c0c9830 2024-05-27T09:37:47.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.013+0000 7f5343a00640 1 -- 172.21.15.114:0/587517974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f533c0c9830 msgr2=0x7f533c06c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:47.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.013+0000 7f5343a00640 1 --2- 172.21.15.114:0/587517974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f533c0c9830 0x7f533c06c690 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f533c0ce740 tx=0x7f533002a250 comp rx=0 tx=0).stop 2024-05-27T09:37:47.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.013+0000 7f5343a00640 1 -- 172.21.15.114:0/587517974 shutdown_connections 2024-05-27T09:37:47.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.013+0000 7f5343a00640 1 --2- 172.21.15.114:0/587517974 >> v2:172.21.15.114:3300/0 conn(0x7f533c073260 0x7f533c06cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:47.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.013+0000 7f5343a00640 1 --2- 172.21.15.114:0/587517974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f533c0c9830 0x7f533c06c690 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:47.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.013+0000 7f5343a00640 1 -- 172.21.15.114:0/587517974 >> 172.21.15.114:0/587517974 conn(0x7f533c0cd9e0 msgr2=0x7f533c0cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:47.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.013+0000 7f5343a00640 1 -- 172.21.15.114:0/587517974 shutdown_connections 2024-05-27T09:37:47.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.013+0000 7f5343a00640 1 -- 172.21.15.114:0/587517974 wait complete. 2024-05-27T09:37:47.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.015+0000 7f5343a00640 1 Processor -- start 2024-05-27T09:37:47.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.015+0000 7f5343a00640 1 -- start start 2024-05-27T09:37:47.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.017+0000 7f5343a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f533c073260 0x7f533c1f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:47.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.017+0000 7f5343a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f533c0c9830 0x7f533c1e7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:47.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.017+0000 7f5343a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533c15a830 0x7f533c1f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:47.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.017+0000 7f5343a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f533c0706c0 con 0x7f533c15a830 2024-05-27T09:37:47.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.017+0000 7f5343a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f533c0749d0 con 0x7f533c073260 2024-05-27T09:37:47.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.017+0000 7f5343a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f533c073940 con 0x7f533c0c9830 2024-05-27T09:37:47.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.017+0000 7f5341a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533c15a830 0x7f533c1f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:47.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.017+0000 7f533be00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f533c0c9830 0x7f533c1e7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:47.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.017+0000 7f5341a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533c15a830 0x7f533c1f39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37320/0 (socket says 172.21.15.114:37320) 2024-05-27T09:37:47.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.017+0000 7f5341a00640 1 -- 172.21.15.114:0/3710818701 learned_addr learned my addr 172.21.15.114:0/3710818701 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:47.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.017+0000 7f5341000640 1 --2- 172.21.15.114:0/3710818701 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f533c073260 0x7f533c1f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:47.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.018+0000 7f5341a00640 1 -- 172.21.15.114:0/3710818701 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f533c0c9830 msgr2=0x7f533c1e7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:47.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.018+0000 7f5341a00640 1 --2- 172.21.15.114:0/3710818701 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f533c0c9830 0x7f533c1e7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:47.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.018+0000 7f5341a00640 1 -- 172.21.15.114:0/3710818701 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f533c073260 msgr2=0x7f533c1f0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:47.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.018+0000 7f5341a00640 1 --2- 172.21.15.114:0/3710818701 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f533c073260 0x7f533c1f0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:47.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.018+0000 7f5341a00640 1 -- 172.21.15.114:0/3710818701 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533c14ee40 con 0x7f533c15a830 2024-05-27T09:37:47.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.018+0000 7f533be00640 1 --2- 172.21.15.114:0/3710818701 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f533c0c9830 0x7f533c1e7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:47.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.018+0000 7f5341000640 1 --2- 172.21.15.114:0/3710818701 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f533c073260 0x7f533c1f0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:47.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.018+0000 7f5341a00640 1 --2- 172.21.15.114:0/3710818701 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533c15a830 0x7f533c1f39a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f533400c6e0 tx=0x7f533400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:47.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.018+0000 7f5338c00640 1 -- 172.21.15.114:0/3710818701 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5334018070 con 0x7f533c15a830 2024-05-27T09:37:47.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.018+0000 7f5338c00640 1 -- 172.21.15.114:0/3710818701 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5334004d60 con 0x7f533c15a830 2024-05-27T09:37:47.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.018+0000 7f5338c00640 1 -- 172.21.15.114:0/3710818701 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533400a880 con 0x7f533c15a830 2024-05-27T09:37:47.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.018+0000 7f5343a00640 1 -- 172.21.15.114:0/3710818701 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f533c14a5c0 con 0x7f533c15a830 2024-05-27T09:37:47.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.018+0000 7f5343a00640 1 -- 172.21.15.114:0/3710818701 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f533c0c5df0 con 0x7f533c15a830 2024-05-27T09:37:47.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.019+0000 7f5343a00640 1 -- 172.21.15.114:0/3710818701 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f533c06a380 con 0x7f533c15a830 2024-05-27T09:37:47.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.021+0000 7f5338c00640 1 -- 172.21.15.114:0/3710818701 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f533400aa20 con 0x7f533c15a830 2024-05-27T09:37:47.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.022+0000 7f5338c00640 1 --2- 172.21.15.114:0/3710818701 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f531c076300 0x7f531c078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:47.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.022+0000 7f5338c00640 1 -- 172.21.15.114:0/3710818701 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7f5334096ce0 con 0x7f533c15a830 2024-05-27T09:37:47.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.022+0000 7f5338c00640 1 --2- 172.21.15.114:0/3710818701 >> [v2:172.21.15.114:6808/865568619,v1:172.21.15.114:6809/865568619] conn(0x7f531c07e9f0 0x7f531c080e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:47.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.022+0000 7f5341000640 1 --2- 172.21.15.114:0/3710818701 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f531c076300 0x7f531c078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:47.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.022+0000 7f5338c00640 1 -- 172.21.15.114:0/3710818701 --> [v2:172.21.15.114:6808/865568619,v1:172.21.15.114:6809/865568619] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f531c081530 con 0x7f531c07e9f0 2024-05-27T09:37:47.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.022+0000 7f5338c00640 1 -- 172.21.15.114:0/3710818701 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f533409c2c0 con 0x7f533c15a830 2024-05-27T09:37:47.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.022+0000 7f533be00640 1 --2- 172.21.15.114:0/3710818701 >> [v2:172.21.15.114:6808/865568619,v1:172.21.15.114:6809/865568619] conn(0x7f531c07e9f0 0x7f531c080e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:47.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.023+0000 7f5341000640 1 --2- 172.21.15.114:0/3710818701 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f531c076300 0x7f531c078810 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5330034000 tx=0x7f533002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:47.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.023+0000 7f533be00640 1 --2- 172.21.15.114:0/3710818701 >> [v2:172.21.15.114:6808/865568619,v1:172.21.15.114:6809/865568619] conn(0x7f531c07e9f0 0x7f531c080e60 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:47.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.024+0000 7f5338c00640 1 -- 172.21.15.114:0/3710818701 <== osd.2 v2:172.21.15.114:6808/865568619 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f531c081530 con 0x7f531c07e9f0 2024-05-27T09:37:47.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.038+0000 7f5343a00640 1 -- 172.21.15.114:0/3710818701 --> [v2:172.21.15.114:6808/865568619,v1:172.21.15.114:6809/865568619] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f533c0c9f10 con 0x7f531c07e9f0 2024-05-27T09:37:47.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.039+0000 7f5338c00640 1 -- 172.21.15.114:0/3710818701 <== osd.2 v2:172.21.15.114:6808/865568619 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f533c0c9f10 con 0x7f531c07e9f0 2024-05-27T09:37:47.040 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-05-27T09:37:47.040 INFO:teuthology.orchestra.run.smithi114.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-27T09:37:47.040 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-05-27T09:37:47.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.040+0000 7f5343a00640 1 -- 172.21.15.114:0/3710818701 >> [v2:172.21.15.114:6808/865568619,v1:172.21.15.114:6809/865568619] conn(0x7f531c07e9f0 msgr2=0x7f531c080e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:47.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.040+0000 7f5343a00640 1 --2- 172.21.15.114:0/3710818701 >> [v2:172.21.15.114:6808/865568619,v1:172.21.15.114:6809/865568619] conn(0x7f531c07e9f0 0x7f531c080e60 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:47.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.040+0000 7f5343a00640 1 -- 172.21.15.114:0/3710818701 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f531c076300 msgr2=0x7f531c078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:47.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.040+0000 7f5343a00640 1 --2- 172.21.15.114:0/3710818701 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f531c076300 0x7f531c078810 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5330034000 tx=0x7f533002e000 comp rx=0 tx=0).stop 2024-05-27T09:37:47.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.040+0000 7f5343a00640 1 -- 172.21.15.114:0/3710818701 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533c15a830 msgr2=0x7f533c1f39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:47.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.040+0000 7f5343a00640 1 --2- 172.21.15.114:0/3710818701 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533c15a830 0x7f533c1f39a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f533400c6e0 tx=0x7f533400cba0 comp rx=0 tx=0).stop 2024-05-27T09:37:47.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.040+0000 7f5341a00640 1 -- 172.21.15.114:0/3710818701 reap_dead start 2024-05-27T09:37:47.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.040+0000 7f5343a00640 1 -- 172.21.15.114:0/3710818701 shutdown_connections 2024-05-27T09:37:47.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.040+0000 7f5343a00640 1 -- 172.21.15.114:0/3710818701 >> 172.21.15.114:0/3710818701 conn(0x7f533c0cd9e0 msgr2=0x7f533c0cd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:47.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.041+0000 7f5343a00640 1 -- 172.21.15.114:0/3710818701 shutdown_connections 2024-05-27T09:37:47.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.041+0000 7f5343a00640 1 -- 172.21.15.114:0/3710818701 wait complete. 2024-05-27T09:37:47.066 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-27T09:37:47.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.341+0000 7fa442200640 1 Processor -- start 2024-05-27T09:37:47.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.342+0000 7fa442200640 1 -- start start 2024-05-27T09:37:47.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.344+0000 7fa442200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa43c0741f0 0x7fa43c0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:47.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.344+0000 7fa442200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa43c1547d0 0x7fa43c15ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:47.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.344+0000 7fa442200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa43c06fe50 con 0x7fa43c074840 2024-05-27T09:37:47.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.344+0000 7fa442200640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fa43c0703d0 con 0x7fa43c1547d0 2024-05-27T09:37:47.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.344+0000 7fa442200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa43c15b260 con 0x7fa43c0741f0 2024-05-27T09:37:47.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.344+0000 7fa43be00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa43c1547d0 0x7fa43c15ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:47.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.344+0000 7fa440e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa43c0741f0 0x7fa43c0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:47.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.344+0000 7fa43be00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa43c1547d0 0x7fa43c15ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:42948/0 (socket says 172.21.15.114:42948) 2024-05-27T09:37:47.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.344+0000 7fa43be00640 1 -- 172.21.15.114:0/385039181 learned_addr learned my addr 172.21.15.114:0/385039181 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:47.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.345+0000 7fa43be00640 1 -- 172.21.15.114:0/385039181 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa43c0741f0 msgr2=0x7fa43c0c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:47.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.345+0000 7fa43be00640 1 --2- 172.21.15.114:0/385039181 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa43c0741f0 0x7fa43c0c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:47.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.345+0000 7fa43be00640 1 -- 172.21.15.114:0/385039181 >> v1:172.21.15.163:6789/0 conn(0x7fa43c074840 legacy=0x7fa43c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:47.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.345+0000 7fa43be00640 1 -- 172.21.15.114:0/385039181 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43c0700f0 con 0x7fa43c1547d0 2024-05-27T09:37:47.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.346+0000 7fa43be00640 1 --2- 172.21.15.114:0/385039181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa43c1547d0 0x7fa43c15ad20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa428009870 tx=0x7fa42802a2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f82a2d1251454f8a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:47.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.346+0000 7fa43b400640 1 -- 172.21.15.114:0/385039181 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa428037070 con 0x7fa43c1547d0 2024-05-27T09:37:47.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.346+0000 7fa43b400640 1 -- 172.21.15.114:0/385039181 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa428032640 con 0x7fa43c1547d0 2024-05-27T09:37:47.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.346+0000 7fa43b400640 1 -- 172.21.15.114:0/385039181 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa428032a10 con 0x7fa43c1547d0 2024-05-27T09:37:47.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.347+0000 7fa442200640 1 -- 172.21.15.114:0/385039181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa43c1547d0 msgr2=0x7fa43c15ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:47.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.347+0000 7fa442200640 1 --2- 172.21.15.114:0/385039181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa43c1547d0 0x7fa43c15ad20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa428009870 tx=0x7fa42802a2f0 comp rx=0 tx=0).stop 2024-05-27T09:37:47.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.347+0000 7fa442200640 1 -- 172.21.15.114:0/385039181 shutdown_connections 2024-05-27T09:37:47.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.347+0000 7fa442200640 1 --2- 172.21.15.114:0/385039181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa43c1547d0 0x7fa43c15ad20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:47.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.347+0000 7fa442200640 1 --2- 172.21.15.114:0/385039181 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa43c0741f0 0x7fa43c0c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:47.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.347+0000 7fa442200640 1 -- 172.21.15.114:0/385039181 >> 172.21.15.114:0/385039181 conn(0x7fa43c0ca320 msgr2=0x7fa43c0ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:47.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.347+0000 7fa442200640 1 -- 172.21.15.114:0/385039181 shutdown_connections 2024-05-27T09:37:47.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.347+0000 7fa442200640 1 -- 172.21.15.114:0/385039181 wait complete. 2024-05-27T09:37:47.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.350+0000 7fa442200640 1 Processor -- start 2024-05-27T09:37:47.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.350+0000 7fa442200640 1 -- start start 2024-05-27T09:37:47.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.352+0000 7fa442200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa43c0741f0 0x7fa43c0b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:47.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.352+0000 7fa442200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa43c074840 0x7fa43c1f2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:47.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.352+0000 7fa442200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa43c1547d0 0x7fa43c0c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:47.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.352+0000 7fa442200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c073790 con 0x7fa43c1547d0 2024-05-27T09:37:47.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.352+0000 7fa442200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c0734b0 con 0x7fa43c0741f0 2024-05-27T09:37:47.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.352+0000 7fa442200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa43c06f500 con 0x7fa43c074840 2024-05-27T09:37:47.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.352+0000 7fa440e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa43c0741f0 0x7fa43c0b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:47.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.352+0000 7fa43be00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa43c074840 0x7fa43c1f2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:47.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.352+0000 7fa440e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa43c0741f0 0x7fa43c0b0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:42954/0 (socket says 172.21.15.114:42954) 2024-05-27T09:37:47.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.352+0000 7fa440e00640 1 -- 172.21.15.114:0/1276351601 learned_addr learned my addr 172.21.15.114:0/1276351601 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:47.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.352+0000 7fa441800640 1 --2- 172.21.15.114:0/1276351601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa43c1547d0 0x7fa43c0c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:47.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.353+0000 7fa440e00640 1 -- 172.21.15.114:0/1276351601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa43c074840 msgr2=0x7fa43c1f2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:47.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.353+0000 7fa440e00640 1 --2- 172.21.15.114:0/1276351601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa43c074840 0x7fa43c1f2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:47.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.353+0000 7fa440e00640 1 -- 172.21.15.114:0/1276351601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa43c1547d0 msgr2=0x7fa43c0c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:47.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.353+0000 7fa440e00640 1 --2- 172.21.15.114:0/1276351601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa43c1547d0 0x7fa43c0c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:47.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.353+0000 7fa440e00640 1 -- 172.21.15.114:0/1276351601 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43c0b1f40 con 0x7fa43c0741f0 2024-05-27T09:37:47.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.353+0000 7fa43be00640 1 --2- 172.21.15.114:0/1276351601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa43c074840 0x7fa43c1f2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:47.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.353+0000 7fa441800640 1 --2- 172.21.15.114:0/1276351601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa43c1547d0 0x7fa43c0c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:37:47.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.353+0000 7fa440e00640 1 --2- 172.21.15.114:0/1276351601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa43c0741f0 0x7fa43c0b0f00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa43000e870 tx=0x7fa43000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:47.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.353+0000 7fa438c00640 1 -- 172.21.15.114:0/1276351601 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430017070 con 0x7fa43c0741f0 2024-05-27T09:37:47.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.353+0000 7fa438c00640 1 -- 172.21.15.114:0/1276351601 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43000c940 con 0x7fa43c0741f0 2024-05-27T09:37:47.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.353+0000 7fa442200640 1 -- 172.21.15.114:0/1276351601 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa43c1f5a40 con 0x7fa43c0741f0 2024-05-27T09:37:47.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.353+0000 7fa438c00640 1 -- 172.21.15.114:0/1276351601 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43000cd30 con 0x7fa43c0741f0 2024-05-27T09:37:47.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.354+0000 7fa442200640 1 -- 172.21.15.114:0/1276351601 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa43c1f6000 con 0x7fa43c0741f0 2024-05-27T09:37:47.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.355+0000 7fa442200640 1 -- 172.21.15.114:0/1276351601 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa43c14e8e0 con 0x7fa43c0741f0 2024-05-27T09:37:47.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.356+0000 7fa438c00640 1 -- 172.21.15.114:0/1276351601 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa43001a070 con 0x7fa43c0741f0 2024-05-27T09:37:47.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.357+0000 7fa438c00640 1 --2- 172.21.15.114:0/1276351601 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa41c076300 0x7fa41c078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:47.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.357+0000 7fa438c00640 1 -- 172.21.15.114:0/1276351601 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7fa430098240 con 0x7fa43c0741f0 2024-05-27T09:37:47.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.357+0000 7fa43be00640 1 --2- 172.21.15.114:0/1276351601 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa41c076300 0x7fa41c078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:47.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.358+0000 7fa43be00640 1 --2- 172.21.15.114:0/1276351601 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa41c076300 0x7fa41c078810 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa428034000 tx=0x7fa42802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:47.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.360+0000 7fa438c00640 1 -- 172.21.15.114:0/1276351601 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa430061310 con 0x7fa43c0741f0 2024-05-27T09:37:47.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.569+0000 7fa442200640 1 -- 172.21.15.114:0/1276351601 --> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fa43c153e00 con 0x7fa41c076300 2024-05-27T09:37:47.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.571+0000 7fa438c00640 1 -- 172.21.15.114:0/1276351601 <== mgr.4108 v2:172.21.15.163:6832/1122018204 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa43c153e00 con 0x7fa41c076300 2024-05-27T09:37:47.572 INFO:teuthology.orchestra.run.smithi114.stderr:instructed osd(s) 2 to deep-scrub 2024-05-27T09:37:47.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.574+0000 7fa442200640 1 -- 172.21.15.114:0/1276351601 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa41c076300 msgr2=0x7fa41c078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:47.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.574+0000 7fa442200640 1 --2- 172.21.15.114:0/1276351601 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa41c076300 0x7fa41c078810 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa428034000 tx=0x7fa42802e000 comp rx=0 tx=0).stop 2024-05-27T09:37:47.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.574+0000 7fa442200640 1 -- 172.21.15.114:0/1276351601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa43c0741f0 msgr2=0x7fa43c0b0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:47.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.574+0000 7fa442200640 1 --2- 172.21.15.114:0/1276351601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa43c0741f0 0x7fa43c0b0f00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa43000e870 tx=0x7fa43000ed30 comp rx=0 tx=0).stop 2024-05-27T09:37:47.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.574+0000 7fa442200640 1 -- 172.21.15.114:0/1276351601 shutdown_connections 2024-05-27T09:37:47.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.574+0000 7fa442200640 1 --2- 172.21.15.114:0/1276351601 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa41c076300 0x7fa41c078810 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:47.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.574+0000 7fa442200640 1 --2- 172.21.15.114:0/1276351601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa43c1547d0 0x7fa43c0c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:47.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.574+0000 7fa442200640 1 --2- 172.21.15.114:0/1276351601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa43c074840 0x7fa43c1f2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:47.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.574+0000 7fa442200640 1 --2- 172.21.15.114:0/1276351601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa43c0741f0 0x7fa43c0b0f00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:47.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.574+0000 7fa442200640 1 -- 172.21.15.114:0/1276351601 >> 172.21.15.114:0/1276351601 conn(0x7fa43c0ca320 msgr2=0x7fa43c0764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:47.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.574+0000 7fa442200640 1 -- 172.21.15.114:0/1276351601 shutdown_connections 2024-05-27T09:37:47.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.574+0000 7fa442200640 1 -- 172.21.15.114:0/1276351601 wait complete. 2024-05-27T09:37:47.600 INFO:tasks.ceph:Scrubbing osd.3 2024-05-27T09:37:47.600 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-27T09:37:47.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.868+0000 7f0e25400640 1 Processor -- start 2024-05-27T09:37:47.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.869+0000 7f0e25400640 1 -- start start 2024-05-27T09:37:47.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.871+0000 7f0e25400640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0e200c9830 0x7f0e2006c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:47.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.871+0000 7f0e25400640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e20074cb0 con 0x7f0e2015af40 2024-05-27T09:37:47.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.871+0000 7f0e25400640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e20074130 con 0x7f0e20073260 2024-05-27T09:37:47.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.871+0000 7f0e25400640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f0e20073ee0 con 0x7f0e200c9830 2024-05-27T09:37:47.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.871+0000 7f0e1f200640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f0e2015af40 0x7f0e20164dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55918/0 (socket says 172.21.15.114:55918) 2024-05-27T09:37:47.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.871+0000 7f0e1e800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f0e200c9830 0x7f0e2006c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:47.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.871+0000 7f0e1f200640 1 -- 172.21.15.114:0/530065757 learned_addr learned my addr 172.21.15.114:0/530065757 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:47.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.872+0000 7f0e1de00640 1 -- 172.21.15.114:0/530065757 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1610860682 0 0) 0x7f0e20074130 con 0x7f0e20073260 2024-05-27T09:37:47.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.872+0000 7f0e1de00640 1 -- 172.21.15.114:0/530065757 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e04003520 con 0x7f0e20073260 2024-05-27T09:37:47.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.872+0000 7f0e1e800640 1 -- 172.21.15.114:0/530065757 >> v1:172.21.15.163:6789/0 conn(0x7f0e20073260 legacy=0x7f0e2015a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:47.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.872+0000 7f0e1e800640 1 -- 172.21.15.114:0/530065757 >> v1:172.21.15.114:6789/0 conn(0x7f0e2015af40 legacy=0x7f0e20164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:47.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.872+0000 7f0e1de00640 1 -- 172.21.15.114:0/530065757 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 369351615 0 0) 0x7f0e20074cb0 con 0x7f0e2015af40 2024-05-27T09:37:47.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.872+0000 7f0e1e800640 1 -- 172.21.15.114:0/530065757 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e20074410 con 0x7f0e200c9830 2024-05-27T09:37:47.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.873+0000 7f0e1e800640 1 --2- 172.21.15.114:0/530065757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e200c9830 0x7f0e2006c690 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0e100097a0 tx=0x7f0e1002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff1e014e196ddac server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:47.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.873+0000 7f0e1de00640 1 -- 172.21.15.114:0/530065757 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e10037070 con 0x7f0e200c9830 2024-05-27T09:37:47.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.873+0000 7f0e1de00640 1 -- 172.21.15.114:0/530065757 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e10032640 con 0x7f0e200c9830 2024-05-27T09:37:47.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.873+0000 7f0e1de00640 1 -- 172.21.15.114:0/530065757 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e100329f0 con 0x7f0e200c9830 2024-05-27T09:37:47.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.874+0000 7f0e25400640 1 -- 172.21.15.114:0/530065757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e200c9830 msgr2=0x7f0e2006c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:47.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.874+0000 7f0e25400640 1 --2- 172.21.15.114:0/530065757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e200c9830 0x7f0e2006c690 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0e100097a0 tx=0x7f0e1002a2c0 comp rx=0 tx=0).stop 2024-05-27T09:37:47.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.874+0000 7f0e25400640 1 -- 172.21.15.114:0/530065757 shutdown_connections 2024-05-27T09:37:47.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.874+0000 7f0e25400640 1 --2- 172.21.15.114:0/530065757 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e200c9830 0x7f0e2006c690 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:47.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.874+0000 7f0e25400640 1 -- 172.21.15.114:0/530065757 >> 172.21.15.114:0/530065757 conn(0x7f0e200cd9e0 msgr2=0x7f0e200cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:47.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.874+0000 7f0e25400640 1 -- 172.21.15.114:0/530065757 shutdown_connections 2024-05-27T09:37:47.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.874+0000 7f0e25400640 1 -- 172.21.15.114:0/530065757 wait complete. 2024-05-27T09:37:47.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.876+0000 7f0e25400640 1 Processor -- start 2024-05-27T09:37:47.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.877+0000 7f0e25400640 1 -- start start 2024-05-27T09:37:47.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.878+0000 7f0e25400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e20073260 0x7f0e200c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:47.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.878+0000 7f0e25400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200c9830 0x7f0e201f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:47.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.878+0000 7f0e25400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e2015af40 0x7f0e200c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:47.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.878+0000 7f0e25400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0e200706c0 con 0x7f0e2015af40 2024-05-27T09:37:47.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.878+0000 7f0e25400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0e200749d0 con 0x7f0e200c9830 2024-05-27T09:37:47.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.878+0000 7f0e25400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f0e20073940 con 0x7f0e20073260 2024-05-27T09:37:47.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.879+0000 7f0e1e800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e20073260 0x7f0e200c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:47.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.879+0000 7f0e1e800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e20073260 0x7f0e200c61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:43750/0 (socket says 172.21.15.114:43750) 2024-05-27T09:37:47.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.879+0000 7f0e17e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200c9830 0x7f0e201f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:47.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.879+0000 7f0e1e800640 1 -- 172.21.15.114:0/3272903982 learned_addr learned my addr 172.21.15.114:0/3272903982 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:47.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.879+0000 7f0e1f200640 1 --2- 172.21.15.114:0/3272903982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e2015af40 0x7f0e200c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:47.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.879+0000 7f0e17e00640 1 -- 172.21.15.114:0/3272903982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e20073260 msgr2=0x7f0e200c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:47.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.879+0000 7f0e17e00640 1 --2- 172.21.15.114:0/3272903982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e20073260 0x7f0e200c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:47.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.879+0000 7f0e17e00640 1 -- 172.21.15.114:0/3272903982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e2015af40 msgr2=0x7f0e200c6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:47.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.879+0000 7f0e17e00640 1 --2- 172.21.15.114:0/3272903982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e2015af40 0x7f0e200c6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:47.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.879+0000 7f0e17e00640 1 -- 172.21.15.114:0/3272903982 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e201f6540 con 0x7f0e200c9830 2024-05-27T09:37:47.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.879+0000 7f0e1e800640 1 --2- 172.21.15.114:0/3272903982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e20073260 0x7f0e200c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:47.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.879+0000 7f0e1f200640 1 --2- 172.21.15.114:0/3272903982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e2015af40 0x7f0e200c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:47.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.880+0000 7f0e17e00640 1 --2- 172.21.15.114:0/3272903982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200c9830 0x7f0e201f6000 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800c3c0 tx=0x7f0e0800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:47.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.880+0000 7f0e16a00640 1 -- 172.21.15.114:0/3272903982 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08017070 con 0x7f0e200c9830 2024-05-27T09:37:47.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.880+0000 7f0e16a00640 1 -- 172.21.15.114:0/3272903982 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e080052f0 con 0x7f0e200c9830 2024-05-27T09:37:47.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.880+0000 7f0e25400640 1 -- 172.21.15.114:0/3272903982 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e200b2af0 con 0x7f0e200c9830 2024-05-27T09:37:47.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.880+0000 7f0e16a00640 1 -- 172.21.15.114:0/3272903982 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08003830 con 0x7f0e200c9830 2024-05-27T09:37:47.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.880+0000 7f0e25400640 1 -- 172.21.15.114:0/3272903982 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e200c3660 con 0x7f0e200c9830 2024-05-27T09:37:47.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.881+0000 7f0e25400640 1 -- 172.21.15.114:0/3272903982 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0e201583d0 con 0x7f0e200c9830 2024-05-27T09:37:47.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.883+0000 7f0e16a00640 1 -- 172.21.15.114:0/3272903982 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0e0801a070 con 0x7f0e200c9830 2024-05-27T09:37:47.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.884+0000 7f0e16a00640 1 --2- 172.21.15.114:0/3272903982 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0dfc076300 0x7f0dfc078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:47.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.884+0000 7f0e16a00640 1 -- 172.21.15.114:0/3272903982 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7f0e08096e40 con 0x7f0e200c9830 2024-05-27T09:37:47.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.884+0000 7f0e1e800640 1 --2- 172.21.15.114:0/3272903982 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0dfc076300 0x7f0dfc078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:47.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.884+0000 7f0e16a00640 1 --2- 172.21.15.114:0/3272903982 >> [v2:172.21.15.114:6800/1601792164,v1:172.21.15.114:6801/1601792164] conn(0x7f0dfc07e980 0x7f0dfc080df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:47.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.884+0000 7f0e16a00640 1 -- 172.21.15.114:0/3272903982 --> [v2:172.21.15.114:6800/1601792164,v1:172.21.15.114:6801/1601792164] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e08009e50 con 0x7f0dfc07e980 2024-05-27T09:37:47.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.884+0000 7f0e16a00640 1 -- 172.21.15.114:0/3272903982 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0e080971f0 con 0x7f0e200c9830 2024-05-27T09:37:47.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.884+0000 7f0e1f200640 1 --2- 172.21.15.114:0/3272903982 >> [v2:172.21.15.114:6800/1601792164,v1:172.21.15.114:6801/1601792164] conn(0x7f0dfc07e980 0x7f0dfc080df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:47.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.885+0000 7f0e1e800640 1 --2- 172.21.15.114:0/3272903982 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0dfc076300 0x7f0dfc078810 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0e100097a0 tx=0x7f0e1002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:47.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.885+0000 7f0e1f200640 1 --2- 172.21.15.114:0/3272903982 >> [v2:172.21.15.114:6800/1601792164,v1:172.21.15.114:6801/1601792164] conn(0x7f0dfc07e980 0x7f0dfc080df0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:47.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.886+0000 7f0e16a00640 1 -- 172.21.15.114:0/3272903982 <== osd.3 v2:172.21.15.114:6800/1601792164 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e08009e50 con 0x7f0dfc07e980 2024-05-27T09:37:47.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.901+0000 7f0e25400640 1 -- 172.21.15.114:0/3272903982 --> [v2:172.21.15.114:6800/1601792164,v1:172.21.15.114:6801/1601792164] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0e2006a380 con 0x7f0dfc07e980 2024-05-27T09:37:47.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.901+0000 7f0e16a00640 1 -- 172.21.15.114:0/3272903982 <== osd.3 v2:172.21.15.114:6800/1601792164 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0e2006a380 con 0x7f0dfc07e980 2024-05-27T09:37:47.903 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-05-27T09:37:47.903 INFO:teuthology.orchestra.run.smithi114.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-27T09:37:47.903 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-05-27T09:37:47.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.902+0000 7f0e25400640 1 -- 172.21.15.114:0/3272903982 >> [v2:172.21.15.114:6800/1601792164,v1:172.21.15.114:6801/1601792164] conn(0x7f0dfc07e980 msgr2=0x7f0dfc080df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:47.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.902+0000 7f0e25400640 1 --2- 172.21.15.114:0/3272903982 >> [v2:172.21.15.114:6800/1601792164,v1:172.21.15.114:6801/1601792164] conn(0x7f0dfc07e980 0x7f0dfc080df0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:47.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.902+0000 7f0e25400640 1 -- 172.21.15.114:0/3272903982 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0dfc076300 msgr2=0x7f0dfc078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:47.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.902+0000 7f0e25400640 1 --2- 172.21.15.114:0/3272903982 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0dfc076300 0x7f0dfc078810 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0e100097a0 tx=0x7f0e1002e000 comp rx=0 tx=0).stop 2024-05-27T09:37:47.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.902+0000 7f0e25400640 1 -- 172.21.15.114:0/3272903982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200c9830 msgr2=0x7f0e201f6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:47.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.902+0000 7f0e25400640 1 --2- 172.21.15.114:0/3272903982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200c9830 0x7f0e201f6000 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800c3c0 tx=0x7f0e0800c880 comp rx=0 tx=0).stop 2024-05-27T09:37:47.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.902+0000 7f0e1f200640 1 -- 172.21.15.114:0/3272903982 reap_dead start 2024-05-27T09:37:47.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.902+0000 7f0e25400640 1 -- 172.21.15.114:0/3272903982 shutdown_connections 2024-05-27T09:37:47.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.902+0000 7f0e25400640 1 -- 172.21.15.114:0/3272903982 >> 172.21.15.114:0/3272903982 conn(0x7f0e200cd9e0 msgr2=0x7f0e200cd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:47.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.903+0000 7f0e25400640 1 -- 172.21.15.114:0/3272903982 shutdown_connections 2024-05-27T09:37:47.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:47.903+0000 7f0e25400640 1 -- 172.21.15.114:0/3272903982 wait complete. 2024-05-27T09:37:47.932 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-27T09:37:48.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.202+0000 7fdbce800640 1 Processor -- start 2024-05-27T09:37:48.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.203+0000 7fdbce800640 1 -- start start 2024-05-27T09:37:48.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.205+0000 7fdbce800640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdbc8074a60 0x7fdbc8154820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:48.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.205+0000 7fdbce800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbc8073260 0x7fdbc8073670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:48.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.205+0000 7fdbce800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdbc80ca2d0 0x7fdbc80ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:48.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.205+0000 7fdbce800640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fdbc815db60 con 0x7fdbc80ca2d0 2024-05-27T09:37:48.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.205+0000 7fdbce800640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fdbc80cac20 con 0x7fdbc8074a60 2024-05-27T09:37:48.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.205+0000 7fdbce800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc815a810 con 0x7fdbc8073260 2024-05-27T09:37:48.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.205+0000 7fdbc7e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdbc80ca2d0 0x7fdbc80ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:48.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.205+0000 7fdbc7e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fdbc80ca2d0 0x7fdbc80ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37356/0 (socket says 172.21.15.114:37356) 2024-05-27T09:37:48.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.205+0000 7fdbc7e00640 1 -- 172.21.15.114:0/1269688349 learned_addr learned my addr 172.21.15.114:0/1269688349 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:48.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.205+0000 7fdbbea00640 1 --2- 172.21.15.114:0/1269688349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbc8073260 0x7fdbc8073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:48.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.206+0000 7fdbc7400640 1 --2- 172.21.15.114:0/1269688349 >> v2:172.21.15.163:3300/0 conn(0x7fdbc8074a60 0x7fdbc8154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:48.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.206+0000 7fdbc7e00640 1 -- 172.21.15.114:0/1269688349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbc8073260 msgr2=0x7fdbc8073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:48.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.206+0000 7fdbc7e00640 1 --2- 172.21.15.114:0/1269688349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbc8073260 0x7fdbc8073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:48.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.206+0000 7fdbc7e00640 1 -- 172.21.15.114:0/1269688349 >> v2:172.21.15.163:3300/0 conn(0x7fdbc8074a60 msgr2=0x7fdbc8154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:48.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.206+0000 7fdbc7e00640 1 --2- 172.21.15.114:0/1269688349 >> v2:172.21.15.163:3300/0 conn(0x7fdbc8074a60 0x7fdbc8154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:48.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.206+0000 7fdbc7e00640 1 -- 172.21.15.114:0/1269688349 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc815aaf0 con 0x7fdbc80ca2d0 2024-05-27T09:37:48.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.207+0000 7fdbc7e00640 1 --2- 172.21.15.114:0/1269688349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbc80ca2d0 0x7fdbc80ca6e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80c4ce0 tx=0x7fdbc002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237f73cd7b1a499d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:48.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.207+0000 7fdbc6a00640 1 -- 172.21.15.114:0/1269688349 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0036070 con 0x7fdbc80ca2d0 2024-05-27T09:37:48.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.207+0000 7fdbc6a00640 1 -- 172.21.15.114:0/1269688349 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc0031600 con 0x7fdbc80ca2d0 2024-05-27T09:37:48.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.207+0000 7fdbc6a00640 1 -- 172.21.15.114:0/1269688349 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc00319d0 con 0x7fdbc80ca2d0 2024-05-27T09:37:48.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.208+0000 7fdbce800640 1 -- 172.21.15.114:0/1269688349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbc80ca2d0 msgr2=0x7fdbc80ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:48.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.208+0000 7fdbce800640 1 --2- 172.21.15.114:0/1269688349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbc80ca2d0 0x7fdbc80ca6e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80c4ce0 tx=0x7fdbc002a730 comp rx=0 tx=0).stop 2024-05-27T09:37:48.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.208+0000 7fdbce800640 1 -- 172.21.15.114:0/1269688349 shutdown_connections 2024-05-27T09:37:48.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.208+0000 7fdbce800640 1 --2- 172.21.15.114:0/1269688349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbc80ca2d0 0x7fdbc80ca6e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:48.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.208+0000 7fdbce800640 1 --2- 172.21.15.114:0/1269688349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbc8073260 0x7fdbc8073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:48.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.208+0000 7fdbce800640 1 --2- 172.21.15.114:0/1269688349 >> v2:172.21.15.163:3300/0 conn(0x7fdbc8074a60 0x7fdbc8154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:48.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.208+0000 7fdbce800640 1 -- 172.21.15.114:0/1269688349 >> 172.21.15.114:0/1269688349 conn(0x7fdbc806f6e0 msgr2=0x7fdbc8070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:48.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.209+0000 7fdbce800640 1 -- 172.21.15.114:0/1269688349 shutdown_connections 2024-05-27T09:37:48.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.209+0000 7fdbce800640 1 -- 172.21.15.114:0/1269688349 wait complete. 2024-05-27T09:37:48.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.211+0000 7fdbce800640 1 Processor -- start 2024-05-27T09:37:48.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.211+0000 7fdbce800640 1 -- start start 2024-05-27T09:37:48.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.212+0000 7fdbce800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbc8073260 0x7fdbc81f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:48.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.212+0000 7fdbce800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbc8074a60 0x7fdbc81403d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:48.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.212+0000 7fdbce800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbc80ca2d0 0x7fdbc8140910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:48.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.212+0000 7fdbce800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc815b390 con 0x7fdbc8073260 2024-05-27T09:37:48.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.212+0000 7fdbce800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc815b0b0 con 0x7fdbc8074a60 2024-05-27T09:37:48.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.212+0000 7fdbce800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc815a4e0 con 0x7fdbc80ca2d0 2024-05-27T09:37:48.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.213+0000 7fdbc7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbc8073260 0x7fdbc81f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:48.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.213+0000 7fdbc7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbc8073260 0x7fdbc81f6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37372/0 (socket says 172.21.15.114:37372) 2024-05-27T09:37:48.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.213+0000 7fdbc7400640 1 -- 172.21.15.114:0/1646279932 learned_addr learned my addr 172.21.15.114:0/1646279932 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:48.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.213+0000 7fdbbea00640 1 --2- 172.21.15.114:0/1646279932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbc8074a60 0x7fdbc81403d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:48.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.213+0000 7fdbc7e00640 1 --2- 172.21.15.114:0/1646279932 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbc80ca2d0 0x7fdbc8140910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:48.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.213+0000 7fdbc7400640 1 -- 172.21.15.114:0/1646279932 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbc80ca2d0 msgr2=0x7fdbc8140910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:48.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.213+0000 7fdbc7400640 1 --2- 172.21.15.114:0/1646279932 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbc80ca2d0 0x7fdbc8140910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:48.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.213+0000 7fdbc7400640 1 -- 172.21.15.114:0/1646279932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbc8074a60 msgr2=0x7fdbc81403d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:48.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.213+0000 7fdbc7400640 1 --2- 172.21.15.114:0/1646279932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbc8074a60 0x7fdbc81403d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:48.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.213+0000 7fdbc7400640 1 -- 172.21.15.114:0/1646279932 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc80c6c10 con 0x7fdbc8073260 2024-05-27T09:37:48.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.213+0000 7fdbc7400640 1 --2- 172.21.15.114:0/1646279932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbc8073260 0x7fdbc81f6c90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800e870 tx=0x7fdbb800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:48.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.214+0000 7fdbbfe00640 1 -- 172.21.15.114:0/1646279932 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8017070 con 0x7fdbc8073260 2024-05-27T09:37:48.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.214+0000 7fdbbfe00640 1 -- 172.21.15.114:0/1646279932 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbb800c940 con 0x7fdbc8073260 2024-05-27T09:37:48.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.214+0000 7fdbbfe00640 1 -- 172.21.15.114:0/1646279932 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb800cd30 con 0x7fdbc8073260 2024-05-27T09:37:48.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.214+0000 7fdbce800640 1 -- 172.21.15.114:0/1646279932 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc80c6eb0 con 0x7fdbc8073260 2024-05-27T09:37:48.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.214+0000 7fdbce800640 1 -- 172.21.15.114:0/1646279932 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc80c7470 con 0x7fdbc8073260 2024-05-27T09:37:48.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.215+0000 7fdbce800640 1 -- 172.21.15.114:0/1646279932 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc806a380 con 0x7fdbc8073260 2024-05-27T09:37:48.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.216+0000 7fdbbfe00640 1 -- 172.21.15.114:0/1646279932 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdbb801a070 con 0x7fdbc8073260 2024-05-27T09:37:48.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.217+0000 7fdbbfe00640 1 --2- 172.21.15.114:0/1646279932 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdba4076300 0x7fdba4078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:48.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.217+0000 7fdbbfe00640 1 -- 172.21.15.114:0/1646279932 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7fdbb80976a0 con 0x7fdbc8073260 2024-05-27T09:37:48.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.218+0000 7fdbbea00640 1 --2- 172.21.15.114:0/1646279932 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdba4076300 0x7fdba4078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:48.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.218+0000 7fdbbea00640 1 --2- 172.21.15.114:0/1646279932 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdba4076300 0x7fdba4078810 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8164cf0 tx=0x7fdbb0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:48.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.223+0000 7fdbbfe00640 1 -- 172.21.15.114:0/1646279932 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdbb8060660 con 0x7fdbc8073260 2024-05-27T09:37:48.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.440+0000 7fdbce800640 1 -- 172.21.15.114:0/1646279932 --> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fdbc81599a0 con 0x7fdba4076300 2024-05-27T09:37:48.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.441+0000 7fdbbfe00640 1 -- 172.21.15.114:0/1646279932 <== mgr.4108 v2:172.21.15.163:6832/1122018204 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdbc81599a0 con 0x7fdba4076300 2024-05-27T09:37:48.442 INFO:teuthology.orchestra.run.smithi114.stderr:instructed osd(s) 3 to deep-scrub 2024-05-27T09:37:48.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.444+0000 7fdbce800640 1 -- 172.21.15.114:0/1646279932 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdba4076300 msgr2=0x7fdba4078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:48.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.444+0000 7fdbce800640 1 --2- 172.21.15.114:0/1646279932 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdba4076300 0x7fdba4078810 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8164cf0 tx=0x7fdbb0009190 comp rx=0 tx=0).stop 2024-05-27T09:37:48.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.444+0000 7fdbce800640 1 -- 172.21.15.114:0/1646279932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbc8073260 msgr2=0x7fdbc81f6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:48.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.444+0000 7fdbce800640 1 --2- 172.21.15.114:0/1646279932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbc8073260 0x7fdbc81f6c90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800e870 tx=0x7fdbb800ed30 comp rx=0 tx=0).stop 2024-05-27T09:37:48.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.444+0000 7fdbce800640 1 -- 172.21.15.114:0/1646279932 shutdown_connections 2024-05-27T09:37:48.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.444+0000 7fdbce800640 1 --2- 172.21.15.114:0/1646279932 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fdba4076300 0x7fdba4078810 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:48.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.444+0000 7fdbce800640 1 --2- 172.21.15.114:0/1646279932 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbc80ca2d0 0x7fdbc8140910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:48.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.444+0000 7fdbce800640 1 --2- 172.21.15.114:0/1646279932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbc8074a60 0x7fdbc81403d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:48.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.444+0000 7fdbce800640 1 --2- 172.21.15.114:0/1646279932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbc8073260 0x7fdbc81f6c90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:48.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.444+0000 7fdbce800640 1 -- 172.21.15.114:0/1646279932 >> 172.21.15.114:0/1646279932 conn(0x7fdbc806f6e0 msgr2=0x7fdbc80c9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:48.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.444+0000 7fdbce800640 1 -- 172.21.15.114:0/1646279932 shutdown_connections 2024-05-27T09:37:48.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.444+0000 7fdbce800640 1 -- 172.21.15.114:0/1646279932 wait complete. 2024-05-27T09:37:48.471 INFO:tasks.ceph:Scrubbing osd.4 2024-05-27T09:37:48.471 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-27T09:37:48.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.744+0000 7f550d800640 1 Processor -- start 2024-05-27T09:37:48.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.744+0000 7f550d800640 1 -- start start 2024-05-27T09:37:48.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.746+0000 7f550d800640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5508073260 0x7f550815a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:48.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.746+0000 7f550d800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f550815ad70 0x7f5508164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:48.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.746+0000 7f550d800640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5508074cb0 con 0x7f55080c9830 2024-05-27T09:37:48.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.746+0000 7f550d800640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f5508073ee0 con 0x7f5508073260 2024-05-27T09:37:48.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.746+0000 7f550d800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f5508074130 con 0x7f550815ad70 2024-05-27T09:37:48.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.747+0000 7f5507400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f550815ad70 0x7f5508164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:48.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.747+0000 7f5506000640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f5508073260 0x7f550815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:48.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.747+0000 7f5506a00640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f55080c9830 0x7f550806c690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:52870/0 (socket says 172.21.15.114:52870) 2024-05-27T09:37:48.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.747+0000 7f5506a00640 1 -- 172.21.15.114:0/2620900586 learned_addr learned my addr 172.21.15.114:0/2620900586 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:48.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.747+0000 7f5505600640 1 -- 172.21.15.114:0/2620900586 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3299510496 0 0) 0x7f5508074cb0 con 0x7f55080c9830 2024-05-27T09:37:48.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.747+0000 7f5505600640 1 -- 172.21.15.114:0/2620900586 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54ec003520 con 0x7f55080c9830 2024-05-27T09:37:48.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.747+0000 7f5507400640 1 -- 172.21.15.114:0/2620900586 >> v2:172.21.15.114:3300/0 conn(0x7f5508073260 msgr2=0x7f550815a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:48.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.747+0000 7f5507400640 1 --2- 172.21.15.114:0/2620900586 >> v2:172.21.15.114:3300/0 conn(0x7f5508073260 0x7f550815a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:48.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.747+0000 7f5507400640 1 -- 172.21.15.114:0/2620900586 >> v1:172.21.15.163:6789/0 conn(0x7f55080c9830 legacy=0x7f550806c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:48.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.747+0000 7f5507400640 1 -- 172.21.15.114:0/2620900586 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5508074410 con 0x7f550815ad70 2024-05-27T09:37:48.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.747+0000 7f5506000640 1 --2- 172.21.15.114:0/2620900586 >> v2:172.21.15.114:3300/0 conn(0x7f5508073260 0x7f550815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:48.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.748+0000 7f5507400640 1 --2- 172.21.15.114:0/2620900586 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f550815ad70 0x7f5508164dd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f55000097a0 tx=0x7f550002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c3442c62547a1ef server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:48.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.749+0000 7f5505600640 1 -- 172.21.15.114:0/2620900586 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500037070 con 0x7f550815ad70 2024-05-27T09:37:48.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.749+0000 7f5505600640 1 -- 172.21.15.114:0/2620900586 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5500032440 con 0x7f550815ad70 2024-05-27T09:37:48.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.749+0000 7f5505600640 1 -- 172.21.15.114:0/2620900586 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55000327f0 con 0x7f550815ad70 2024-05-27T09:37:48.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.749+0000 7f550d800640 1 -- 172.21.15.114:0/2620900586 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f550815ad70 msgr2=0x7f5508164dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:48.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.749+0000 7f550d800640 1 --2- 172.21.15.114:0/2620900586 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f550815ad70 0x7f5508164dd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f55000097a0 tx=0x7f550002a210 comp rx=0 tx=0).stop 2024-05-27T09:37:48.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.749+0000 7f550d800640 1 -- 172.21.15.114:0/2620900586 shutdown_connections 2024-05-27T09:37:48.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.749+0000 7f550d800640 1 --2- 172.21.15.114:0/2620900586 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f550815ad70 0x7f5508164dd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:48.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.749+0000 7f550d800640 1 --2- 172.21.15.114:0/2620900586 >> v2:172.21.15.114:3300/0 conn(0x7f5508073260 0x7f550815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:48.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.749+0000 7f550d800640 1 -- 172.21.15.114:0/2620900586 >> 172.21.15.114:0/2620900586 conn(0x7f55080cd9e0 msgr2=0x7f55080cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:48.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.749+0000 7f550d800640 1 -- 172.21.15.114:0/2620900586 shutdown_connections 2024-05-27T09:37:48.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.749+0000 7f550d800640 1 -- 172.21.15.114:0/2620900586 wait complete. 2024-05-27T09:37:48.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.752+0000 7f550d800640 1 Processor -- start 2024-05-27T09:37:48.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.752+0000 7f550d800640 1 -- start start 2024-05-27T09:37:48.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.753+0000 7f550d800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5508073260 0x7f55080c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:48.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.753+0000 7f550d800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55080c9830 0x7f5508158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:48.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.753+0000 7f550d800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f550815ad70 0x7f55081592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:48.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.753+0000 7f550d800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f550815b180 con 0x7f55080c9830 2024-05-27T09:37:48.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.753+0000 7f550d800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f55080749d0 con 0x7f5508073260 2024-05-27T09:37:48.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.753+0000 7f550d800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f55080706c0 con 0x7f550815ad70 2024-05-27T09:37:48.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.754+0000 7f5506000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55080c9830 0x7f5508158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:48.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.754+0000 7f5506000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55080c9830 0x7f5508158db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37384/0 (socket says 172.21.15.114:37384) 2024-05-27T09:37:48.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.754+0000 7f5506000640 1 -- 172.21.15.114:0/1510239580 learned_addr learned my addr 172.21.15.114:0/1510239580 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:48.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.754+0000 7f5506a00640 1 --2- 172.21.15.114:0/1510239580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5508073260 0x7f55080c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:48.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.754+0000 7f5507400640 1 --2- 172.21.15.114:0/1510239580 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f550815ad70 0x7f55081592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:48.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.754+0000 7f5506000640 1 -- 172.21.15.114:0/1510239580 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f550815ad70 msgr2=0x7f55081592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:48.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.754+0000 7f5506000640 1 --2- 172.21.15.114:0/1510239580 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f550815ad70 0x7f55081592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:48.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.754+0000 7f5506000640 1 -- 172.21.15.114:0/1510239580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5508073260 msgr2=0x7f55080c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:48.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.754+0000 7f5506000640 1 --2- 172.21.15.114:0/1510239580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5508073260 0x7f55080c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:48.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.754+0000 7f5506000640 1 -- 172.21.15.114:0/1510239580 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55080c6200 con 0x7f55080c9830 2024-05-27T09:37:48.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.754+0000 7f5506a00640 1 --2- 172.21.15.114:0/1510239580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5508073260 0x7f55080c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:48.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.754+0000 7f5506000640 1 --2- 172.21.15.114:0/1510239580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55080c9830 0x7f5508158db0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f54f000e990 tx=0x7f54f000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:48.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.755+0000 7f54fea00640 1 -- 172.21.15.114:0/1510239580 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f0017070 con 0x7f55080c9830 2024-05-27T09:37:48.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.755+0000 7f54fea00640 1 -- 172.21.15.114:0/1510239580 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f000c9e0 con 0x7f55080c9830 2024-05-27T09:37:48.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.755+0000 7f550d800640 1 -- 172.21.15.114:0/1510239580 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55080c44e0 con 0x7f55080c9830 2024-05-27T09:37:48.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.755+0000 7f54fea00640 1 -- 172.21.15.114:0/1510239580 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f000cdb0 con 0x7f55080c9830 2024-05-27T09:37:48.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.755+0000 7f550d800640 1 -- 172.21.15.114:0/1510239580 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55080c3a60 con 0x7f55080c9830 2024-05-27T09:37:48.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.756+0000 7f550d800640 1 -- 172.21.15.114:0/1510239580 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5508153870 con 0x7f55080c9830 2024-05-27T09:37:48.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.757+0000 7f54fea00640 1 -- 172.21.15.114:0/1510239580 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f54f001a070 con 0x7f55080c9830 2024-05-27T09:37:48.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.758+0000 7f54fea00640 1 --2- 172.21.15.114:0/1510239580 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f54e4076300 0x7f54e4078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:48.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.758+0000 7f54fea00640 1 -- 172.21.15.114:0/1510239580 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7f54f0097660 con 0x7f55080c9830 2024-05-27T09:37:48.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.759+0000 7f54fea00640 1 --2- 172.21.15.114:0/1510239580 >> [v2:172.21.15.163:6816/1982246015,v1:172.21.15.163:6817/1982246015] conn(0x7f54e407e970 0x7f54e4080de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:48.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.759+0000 7f54fea00640 1 -- 172.21.15.114:0/1510239580 --> [v2:172.21.15.163:6816/1982246015,v1:172.21.15.163:6817/1982246015] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f54f0015e50 con 0x7f54e407e970 2024-05-27T09:37:48.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.759+0000 7f54fea00640 1 -- 172.21.15.114:0/1510239580 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f54f009e050 con 0x7f55080c9830 2024-05-27T09:37:48.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.759+0000 7f5506a00640 1 --2- 172.21.15.114:0/1510239580 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f54e4076300 0x7f54e4078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:48.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.759+0000 7f5507400640 1 --2- 172.21.15.114:0/1510239580 >> [v2:172.21.15.163:6816/1982246015,v1:172.21.15.163:6817/1982246015] conn(0x7f54e407e970 0x7f54e4080de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:48.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.760+0000 7f5506a00640 1 --2- 172.21.15.114:0/1510239580 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f54e4076300 0x7f54e4078810 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f54f8005c00 tx=0x7f54f8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:48.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.760+0000 7f5507400640 1 --2- 172.21.15.114:0/1510239580 >> [v2:172.21.15.163:6816/1982246015,v1:172.21.15.163:6817/1982246015] conn(0x7f54e407e970 0x7f54e4080de0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:48.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.761+0000 7f54fea00640 1 -- 172.21.15.114:0/1510239580 <== osd.4 v2:172.21.15.163:6816/1982246015 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f54f0015e50 con 0x7f54e407e970 2024-05-27T09:37:48.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.775+0000 7f550d800640 1 -- 172.21.15.114:0/1510239580 --> [v2:172.21.15.163:6816/1982246015,v1:172.21.15.163:6817/1982246015] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f55080c4290 con 0x7f54e407e970 2024-05-27T09:37:48.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.775+0000 7f54fea00640 1 -- 172.21.15.114:0/1510239580 <== osd.4 v2:172.21.15.163:6816/1982246015 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f55080c4290 con 0x7f54e407e970 2024-05-27T09:37:48.777 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-05-27T09:37:48.777 INFO:teuthology.orchestra.run.smithi114.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-27T09:37:48.777 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-05-27T09:37:48.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.776+0000 7f550d800640 1 -- 172.21.15.114:0/1510239580 >> [v2:172.21.15.163:6816/1982246015,v1:172.21.15.163:6817/1982246015] conn(0x7f54e407e970 msgr2=0x7f54e4080de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:48.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.776+0000 7f550d800640 1 --2- 172.21.15.114:0/1510239580 >> [v2:172.21.15.163:6816/1982246015,v1:172.21.15.163:6817/1982246015] conn(0x7f54e407e970 0x7f54e4080de0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:48.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.776+0000 7f550d800640 1 -- 172.21.15.114:0/1510239580 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f54e4076300 msgr2=0x7f54e4078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:48.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.776+0000 7f550d800640 1 --2- 172.21.15.114:0/1510239580 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f54e4076300 0x7f54e4078810 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f54f8005c00 tx=0x7f54f8009040 comp rx=0 tx=0).stop 2024-05-27T09:37:48.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.776+0000 7f550d800640 1 -- 172.21.15.114:0/1510239580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55080c9830 msgr2=0x7f5508158db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:48.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.776+0000 7f550d800640 1 --2- 172.21.15.114:0/1510239580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55080c9830 0x7f5508158db0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f54f000e990 tx=0x7f54f000ee50 comp rx=0 tx=0).stop 2024-05-27T09:37:48.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.776+0000 7f5507400640 1 -- 172.21.15.114:0/1510239580 reap_dead start 2024-05-27T09:37:48.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.776+0000 7f550d800640 1 -- 172.21.15.114:0/1510239580 shutdown_connections 2024-05-27T09:37:48.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.776+0000 7f550d800640 1 -- 172.21.15.114:0/1510239580 >> 172.21.15.114:0/1510239580 conn(0x7f55080cd9e0 msgr2=0x7f5508153060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:48.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.776+0000 7f550d800640 1 -- 172.21.15.114:0/1510239580 shutdown_connections 2024-05-27T09:37:48.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:48.776+0000 7f550d800640 1 -- 172.21.15.114:0/1510239580 wait complete. 2024-05-27T09:37:48.801 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-27T09:37:49.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.106+0000 7f018ee00640 1 Processor -- start 2024-05-27T09:37:49.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.106+0000 7f018ee00640 1 -- start start 2024-05-27T09:37:49.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.108+0000 7f018ee00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01880c8780 0x7f018815fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:49.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.108+0000 7f018ee00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f01880b46e0 0x7f0188166fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:49.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.108+0000 7f018ee00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01880cd1e0 con 0x7f01880b75b0 2024-05-27T09:37:49.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.108+0000 7f018ee00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f01880cd4c0 con 0x7f01880b46e0 2024-05-27T09:37:49.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.108+0000 7f018ee00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f01880c7f80 con 0x7f01880c8780 2024-05-27T09:37:49.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.108+0000 7f018e400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f01880b46e0 0x7f0188166fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:49.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.108+0000 7f018da00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01880c8780 0x7f018815fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:49.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.108+0000 7f018da00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01880c8780 0x7f018815fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:43812/0 (socket says 172.21.15.114:43812) 2024-05-27T09:37:49.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.108+0000 7f018e400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f01880b46e0 0x7f0188166fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:42992/0 (socket says 172.21.15.114:42992) 2024-05-27T09:37:49.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.108+0000 7f018da00640 1 -- 172.21.15.114:0/39767419 learned_addr learned my addr 172.21.15.114:0/39767419 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:49.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.109+0000 7f018da00640 1 -- 172.21.15.114:0/39767419 >> v2:172.21.15.163:3300/0 conn(0x7f01880b46e0 msgr2=0x7f0188166fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:49.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.109+0000 7f018da00640 1 --2- 172.21.15.114:0/39767419 >> v2:172.21.15.163:3300/0 conn(0x7f01880b46e0 0x7f0188166fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:49.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.109+0000 7f018da00640 1 -- 172.21.15.114:0/39767419 >> v1:172.21.15.163:6789/0 conn(0x7f01880b75b0 legacy=0x7f01880d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:49.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.109+0000 7f0187e00640 1 -- 172.21.15.114:0/39767419 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2839260919 0 0) 0x7f01880cd1e0 con 0x7f01880b75b0 2024-05-27T09:37:49.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.109+0000 7f018da00640 1 -- 172.21.15.114:0/39767419 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0188160160 con 0x7f01880c8780 2024-05-27T09:37:49.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.110+0000 7f018da00640 1 --2- 172.21.15.114:0/39767419 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01880c8780 0x7f018815fc20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f017c009870 tx=0x7f017c02a320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c76e28070468fad server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:49.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.111+0000 7f0187e00640 1 -- 172.21.15.114:0/39767419 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c037070 con 0x7f01880c8780 2024-05-27T09:37:49.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.111+0000 7f0187e00640 1 -- 172.21.15.114:0/39767419 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f017c032640 con 0x7f01880c8780 2024-05-27T09:37:49.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.111+0000 7f0187e00640 1 -- 172.21.15.114:0/39767419 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c032a10 con 0x7f01880c8780 2024-05-27T09:37:49.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.112+0000 7f018ee00640 1 -- 172.21.15.114:0/39767419 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01880c8780 msgr2=0x7f018815fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:49.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.112+0000 7f018ee00640 1 --2- 172.21.15.114:0/39767419 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01880c8780 0x7f018815fc20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f017c009870 tx=0x7f017c02a320 comp rx=0 tx=0).stop 2024-05-27T09:37:49.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.112+0000 7f018ee00640 1 -- 172.21.15.114:0/39767419 shutdown_connections 2024-05-27T09:37:49.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.112+0000 7f018ee00640 1 --2- 172.21.15.114:0/39767419 >> v2:172.21.15.163:3300/0 conn(0x7f01880b46e0 0x7f0188166fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:49.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.112+0000 7f018ee00640 1 --2- 172.21.15.114:0/39767419 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01880c8780 0x7f018815fc20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:49.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.112+0000 7f018ee00640 1 -- 172.21.15.114:0/39767419 >> 172.21.15.114:0/39767419 conn(0x7f018806e040 msgr2=0x7f018806e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:49.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.112+0000 7f018ee00640 1 -- 172.21.15.114:0/39767419 shutdown_connections 2024-05-27T09:37:49.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.112+0000 7f018ee00640 1 -- 172.21.15.114:0/39767419 wait complete. 2024-05-27T09:37:49.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.115+0000 7f018ee00640 1 Processor -- start 2024-05-27T09:37:49.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.115+0000 7f018ee00640 1 -- start start 2024-05-27T09:37:49.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.116+0000 7f018ee00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01880b46e0 0x7f01881f6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:49.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.116+0000 7f018ee00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01880b75b0 0x7f01881f2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:49.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.116+0000 7f018ee00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01880c8780 0x7f018815ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:49.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.116+0000 7f018ee00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f01880c6800 con 0x7f01880b46e0 2024-05-27T09:37:49.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.116+0000 7f018ee00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f01880cdf30 con 0x7f01880b75b0 2024-05-27T09:37:49.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.116+0000 7f018ee00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f01881f2670 con 0x7f01880c8780 2024-05-27T09:37:49.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.117+0000 7f018e400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01880c8780 0x7f018815ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:49.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.117+0000 7f018e400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01880c8780 0x7f018815ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:43818/0 (socket says 172.21.15.114:43818) 2024-05-27T09:37:49.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.117+0000 7f018e400640 1 -- 172.21.15.114:0/4059402657 learned_addr learned my addr 172.21.15.114:0/4059402657 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:49.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.117+0000 7f018d000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01880b75b0 0x7f01881f2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:49.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.117+0000 7f018da00640 1 --2- 172.21.15.114:0/4059402657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01880b46e0 0x7f01881f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:49.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.117+0000 7f018d000640 1 -- 172.21.15.114:0/4059402657 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01880c8780 msgr2=0x7f018815ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:49.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.117+0000 7f018d000640 1 --2- 172.21.15.114:0/4059402657 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01880c8780 0x7f018815ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:49.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.117+0000 7f018d000640 1 -- 172.21.15.114:0/4059402657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01880b46e0 msgr2=0x7f01881f6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:49.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.117+0000 7f018d000640 1 --2- 172.21.15.114:0/4059402657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01880b46e0 0x7f01881f6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:49.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.117+0000 7f018d000640 1 -- 172.21.15.114:0/4059402657 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01881f6740 con 0x7f01880b75b0 2024-05-27T09:37:49.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.118+0000 7f018d000640 1 --2- 172.21.15.114:0/4059402657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01880b75b0 0x7f01881f2d10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f01880cc2a0 tx=0x7f017400c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:49.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.118+0000 7f0185600640 1 -- 172.21.15.114:0/4059402657 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174017070 con 0x7f01880b75b0 2024-05-27T09:37:49.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.118+0000 7f0185600640 1 -- 172.21.15.114:0/4059402657 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01740052d0 con 0x7f01880b75b0 2024-05-27T09:37:49.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.118+0000 7f0185600640 1 -- 172.21.15.114:0/4059402657 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174003830 con 0x7f01880b75b0 2024-05-27T09:37:49.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.118+0000 7f018ee00640 1 -- 172.21.15.114:0/4059402657 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f018814b2f0 con 0x7f01880b75b0 2024-05-27T09:37:49.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.118+0000 7f018ee00640 1 -- 172.21.15.114:0/4059402657 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f018814b8b0 con 0x7f01880b75b0 2024-05-27T09:37:49.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.120+0000 7f018ee00640 1 -- 172.21.15.114:0/4059402657 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0188050860 con 0x7f01880b75b0 2024-05-27T09:37:49.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.121+0000 7f0185600640 1 -- 172.21.15.114:0/4059402657 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f017401a070 con 0x7f01880b75b0 2024-05-27T09:37:49.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.123+0000 7f0185600640 1 --2- 172.21.15.114:0/4059402657 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0168076300 0x7f0168078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:49.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.123+0000 7f0185600640 1 -- 172.21.15.114:0/4059402657 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7f01740968d0 con 0x7f01880b75b0 2024-05-27T09:37:49.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.123+0000 7f018da00640 1 --2- 172.21.15.114:0/4059402657 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0168076300 0x7f0168078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:49.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.124+0000 7f018da00640 1 --2- 172.21.15.114:0/4059402657 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0168076300 0x7f0168078810 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f017c034000 tx=0x7f017c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:49.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.127+0000 7f0185600640 1 -- 172.21.15.114:0/4059402657 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f017405f9e0 con 0x7f01880b75b0 2024-05-27T09:37:49.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.390+0000 7f018ee00640 1 -- 172.21.15.114:0/4059402657 --> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f018814b0a0 con 0x7f0168076300 2024-05-27T09:37:49.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.391+0000 7f0185600640 1 -- 172.21.15.114:0/4059402657 <== mgr.4108 v2:172.21.15.163:6832/1122018204 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f018814b0a0 con 0x7f0168076300 2024-05-27T09:37:49.392 INFO:teuthology.orchestra.run.smithi114.stderr:instructed osd(s) 4 to deep-scrub 2024-05-27T09:37:49.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.394+0000 7f018ee00640 1 -- 172.21.15.114:0/4059402657 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0168076300 msgr2=0x7f0168078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:49.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.394+0000 7f018ee00640 1 --2- 172.21.15.114:0/4059402657 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0168076300 0x7f0168078810 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f017c034000 tx=0x7f017c02e000 comp rx=0 tx=0).stop 2024-05-27T09:37:49.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.394+0000 7f018ee00640 1 -- 172.21.15.114:0/4059402657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01880b75b0 msgr2=0x7f01881f2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:49.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.394+0000 7f018ee00640 1 --2- 172.21.15.114:0/4059402657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01880b75b0 0x7f01881f2d10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f01880cc2a0 tx=0x7f017400c820 comp rx=0 tx=0).stop 2024-05-27T09:37:49.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.394+0000 7f018ee00640 1 -- 172.21.15.114:0/4059402657 shutdown_connections 2024-05-27T09:37:49.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.394+0000 7f018ee00640 1 --2- 172.21.15.114:0/4059402657 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0168076300 0x7f0168078810 secure :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f017c034000 tx=0x7f017c02e000 comp rx=0 tx=0).stop 2024-05-27T09:37:49.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.394+0000 7f018ee00640 1 --2- 172.21.15.114:0/4059402657 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01880c8780 0x7f018815ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:49.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.394+0000 7f018ee00640 1 --2- 172.21.15.114:0/4059402657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01880b75b0 0x7f01881f2d10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:49.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.394+0000 7f018ee00640 1 --2- 172.21.15.114:0/4059402657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f01880b46e0 0x7f01881f6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:49.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.394+0000 7f018ee00640 1 -- 172.21.15.114:0/4059402657 >> 172.21.15.114:0/4059402657 conn(0x7f018806e040 msgr2=0x7f0188154690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:49.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.395+0000 7f018ee00640 1 -- 172.21.15.114:0/4059402657 shutdown_connections 2024-05-27T09:37:49.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.395+0000 7f018ee00640 1 -- 172.21.15.114:0/4059402657 wait complete. 2024-05-27T09:37:49.429 INFO:tasks.ceph:Scrubbing osd.5 2024-05-27T09:37:49.430 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-27T09:37:49.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.712+0000 7f0f0d400640 1 Processor -- start 2024-05-27T09:37:49.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.712+0000 7f0f0d400640 1 -- start start 2024-05-27T09:37:49.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.714+0000 7f0f0d400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0f08142820 0x7f0f08074670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:49.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.714+0000 7f0f0d400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f0806c690 0x7f0f0806caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:49.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.715+0000 7f0f0d400640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f0815ca80 con 0x7f0f08073540 2024-05-27T09:37:49.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.715+0000 7f0f0d400640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f0f0815cd20 con 0x7f0f08142820 2024-05-27T09:37:49.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.715+0000 7f0f0d400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f0f0815d000 con 0x7f0f0806c690 2024-05-27T09:37:49.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.715+0000 7f0f05e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f0806c690 0x7f0f0806caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:49.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.715+0000 7f0f06800640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0f08142820 0x7f0f08074670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:49.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.715+0000 7f0f07200640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f0f08073540 0x7f0f08073950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:52888/0 (socket says 172.21.15.114:52888) 2024-05-27T09:37:49.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.715+0000 7f0f07200640 1 -- 172.21.15.114:0/2617272743 learned_addr learned my addr 172.21.15.114:0/2617272743 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:49.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.715+0000 7f0f05e00640 1 -- 172.21.15.114:0/2617272743 >> v2:172.21.15.163:3300/0 conn(0x7f0f08142820 msgr2=0x7f0f08074670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:49.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.715+0000 7f0f05e00640 1 --2- 172.21.15.114:0/2617272743 >> v2:172.21.15.163:3300/0 conn(0x7f0f08142820 0x7f0f08074670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:49.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.715+0000 7f0f05e00640 1 -- 172.21.15.114:0/2617272743 >> v1:172.21.15.163:6789/0 conn(0x7f0f08073540 legacy=0x7f0f08073950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:49.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.715+0000 7f0f05e00640 1 -- 172.21.15.114:0/2617272743 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0815d2e0 con 0x7f0f0806c690 2024-05-27T09:37:49.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.715+0000 7f0f06800640 1 --2- 172.21.15.114:0/2617272743 >> v2:172.21.15.163:3300/0 conn(0x7f0f08142820 0x7f0f08074670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:49.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.715+0000 7f0f05400640 1 -- 172.21.15.114:0/2617272743 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 643819838 0 0) 0x7f0f0815ca80 con 0x7f0f08073540 2024-05-27T09:37:49.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.716+0000 7f0f05e00640 1 --2- 172.21.15.114:0/2617272743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f0806c690 0x7f0f0806caa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40097a0 tx=0x7f0ef402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad7b278b2ad84e51 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:49.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.717+0000 7f0f05400640 1 -- 172.21.15.114:0/2617272743 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4037070 con 0x7f0f0806c690 2024-05-27T09:37:49.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.717+0000 7f0f05400640 1 -- 172.21.15.114:0/2617272743 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef4032440 con 0x7f0f0806c690 2024-05-27T09:37:49.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.717+0000 7f0f05400640 1 -- 172.21.15.114:0/2617272743 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4032810 con 0x7f0f0806c690 2024-05-27T09:37:49.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.718+0000 7f0f0d400640 1 -- 172.21.15.114:0/2617272743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f0806c690 msgr2=0x7f0f0806caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:49.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.718+0000 7f0f0d400640 1 --2- 172.21.15.114:0/2617272743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f0806c690 0x7f0f0806caa0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40097a0 tx=0x7f0ef402a180 comp rx=0 tx=0).stop 2024-05-27T09:37:49.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.718+0000 7f0f0d400640 1 -- 172.21.15.114:0/2617272743 shutdown_connections 2024-05-27T09:37:49.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.718+0000 7f0f0d400640 1 --2- 172.21.15.114:0/2617272743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f0806c690 0x7f0f0806caa0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:49.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.718+0000 7f0f0d400640 1 --2- 172.21.15.114:0/2617272743 >> v2:172.21.15.163:3300/0 conn(0x7f0f08142820 0x7f0f08074670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:49.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.718+0000 7f0f0d400640 1 -- 172.21.15.114:0/2617272743 >> 172.21.15.114:0/2617272743 conn(0x7f0f080cd9e0 msgr2=0x7f0f080cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:49.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.718+0000 7f0f0d400640 1 -- 172.21.15.114:0/2617272743 shutdown_connections 2024-05-27T09:37:49.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.718+0000 7f0f0d400640 1 -- 172.21.15.114:0/2617272743 wait complete. 2024-05-27T09:37:49.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.721+0000 7f0f0d400640 1 Processor -- start 2024-05-27T09:37:49.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.721+0000 7f0f0d400640 1 -- start start 2024-05-27T09:37:49.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.722+0000 7f0f0d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f0806c690 0x7f0f081e3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:49.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.722+0000 7f0f0d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f08073540 0x7f0f081e60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:49.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.722+0000 7f0f0d400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f08142820 0x7f0f080cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:49.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.722+0000 7f0f0d400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0f080c3410 con 0x7f0f08073540 2024-05-27T09:37:49.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.722+0000 7f0f0d400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0815d8a0 con 0x7f0f0806c690 2024-05-27T09:37:49.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.722+0000 7f0f0d400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f0f080c8740 con 0x7f0f08142820 2024-05-27T09:37:49.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.723+0000 7f0f06800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f0806c690 0x7f0f081e3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:49.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.723+0000 7f0f07200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f08142820 0x7f0f080cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:49.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.723+0000 7f0f06800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f0806c690 0x7f0f081e3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:43016/0 (socket says 172.21.15.114:43016) 2024-05-27T09:37:49.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.723+0000 7f0f06800640 1 -- 172.21.15.114:0/3582687530 learned_addr learned my addr 172.21.15.114:0/3582687530 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:49.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.723+0000 7f0f05e00640 1 --2- 172.21.15.114:0/3582687530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f08073540 0x7f0f081e60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:49.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.723+0000 7f0f06800640 1 -- 172.21.15.114:0/3582687530 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f08142820 msgr2=0x7f0f080cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:49.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.723+0000 7f0f06800640 1 --2- 172.21.15.114:0/3582687530 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f08142820 0x7f0f080cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:49.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.723+0000 7f0f06800640 1 -- 172.21.15.114:0/3582687530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f08073540 msgr2=0x7f0f081e60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:49.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.723+0000 7f0f06800640 1 --2- 172.21.15.114:0/3582687530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f08073540 0x7f0f081e60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:49.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.723+0000 7f0f06800640 1 -- 172.21.15.114:0/3582687530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f08155330 con 0x7f0f0806c690 2024-05-27T09:37:49.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.723+0000 7f0f06800640 1 --2- 172.21.15.114:0/3582687530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f0806c690 0x7f0f081e3060 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800e940 tx=0x7f0ef800ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:49.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.724+0000 7f0efea00640 1 -- 172.21.15.114:0/3582687530 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8017070 con 0x7f0f0806c690 2024-05-27T09:37:49.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.724+0000 7f0efea00640 1 -- 172.21.15.114:0/3582687530 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef800ca00 con 0x7f0f0806c690 2024-05-27T09:37:49.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.724+0000 7f0efea00640 1 -- 172.21.15.114:0/3582687530 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef800cdd0 con 0x7f0f0806c690 2024-05-27T09:37:49.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.724+0000 7f0f0d400640 1 -- 172.21.15.114:0/3582687530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f0815b900 con 0x7f0f0806c690 2024-05-27T09:37:49.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.724+0000 7f0f0d400640 1 -- 172.21.15.114:0/3582687530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f0815bec0 con 0x7f0f0806c690 2024-05-27T09:37:49.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.725+0000 7f0f0d400640 1 -- 172.21.15.114:0/3582687530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0f08070510 con 0x7f0f0806c690 2024-05-27T09:37:49.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.726+0000 7f0efea00640 1 -- 172.21.15.114:0/3582687530 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0ef801a070 con 0x7f0f0806c690 2024-05-27T09:37:49.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.727+0000 7f0efea00640 1 --2- 172.21.15.114:0/3582687530 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0ee8076300 0x7f0ee8078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:49.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.727+0000 7f0efea00640 1 -- 172.21.15.114:0/3582687530 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7f0ef80976e0 con 0x7f0f0806c690 2024-05-27T09:37:49.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.727+0000 7f0efea00640 1 --2- 172.21.15.114:0/3582687530 >> [v2:172.21.15.163:6800/1695294004,v1:172.21.15.163:6801/1695294004] conn(0x7f0ee807e970 0x7f0ee8080de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:49.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.728+0000 7f0f05e00640 1 --2- 172.21.15.114:0/3582687530 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0ee8076300 0x7f0ee8078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:49.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.728+0000 7f0efea00640 1 -- 172.21.15.114:0/3582687530 --> [v2:172.21.15.163:6800/1695294004,v1:172.21.15.163:6801/1695294004] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0ef8008eb0 con 0x7f0ee807e970 2024-05-27T09:37:49.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.728+0000 7f0efea00640 1 -- 172.21.15.114:0/3582687530 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0ef809e050 con 0x7f0f0806c690 2024-05-27T09:37:49.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.728+0000 7f0f07200640 1 --2- 172.21.15.114:0/3582687530 >> [v2:172.21.15.163:6800/1695294004,v1:172.21.15.163:6801/1695294004] conn(0x7f0ee807e970 0x7f0ee8080de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:49.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.728+0000 7f0f05e00640 1 --2- 172.21.15.114:0/3582687530 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0ee8076300 0x7f0ee8078810 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4034000 tx=0x7f0ef402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:49.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.728+0000 7f0f07200640 1 --2- 172.21.15.114:0/3582687530 >> [v2:172.21.15.163:6800/1695294004,v1:172.21.15.163:6801/1695294004] conn(0x7f0ee807e970 0x7f0ee8080de0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:49.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.729+0000 7f0efea00640 1 -- 172.21.15.114:0/3582687530 <== osd.5 v2:172.21.15.163:6800/1695294004 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0ef8008eb0 con 0x7f0ee807e970 2024-05-27T09:37:49.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.744+0000 7f0f0d400640 1 -- 172.21.15.114:0/3582687530 --> [v2:172.21.15.163:6800/1695294004,v1:172.21.15.163:6801/1695294004] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0f080767d0 con 0x7f0ee807e970 2024-05-27T09:37:49.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.745+0000 7f0efea00640 1 -- 172.21.15.114:0/3582687530 <== osd.5 v2:172.21.15.163:6800/1695294004 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0f080767d0 con 0x7f0ee807e970 2024-05-27T09:37:49.746 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-05-27T09:37:49.746 INFO:teuthology.orchestra.run.smithi114.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-27T09:37:49.746 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-05-27T09:37:49.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.745+0000 7f0f0d400640 1 -- 172.21.15.114:0/3582687530 >> [v2:172.21.15.163:6800/1695294004,v1:172.21.15.163:6801/1695294004] conn(0x7f0ee807e970 msgr2=0x7f0ee8080de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:49.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.745+0000 7f0f0d400640 1 --2- 172.21.15.114:0/3582687530 >> [v2:172.21.15.163:6800/1695294004,v1:172.21.15.163:6801/1695294004] conn(0x7f0ee807e970 0x7f0ee8080de0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:49.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.745+0000 7f0f0d400640 1 -- 172.21.15.114:0/3582687530 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0ee8076300 msgr2=0x7f0ee8078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:49.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.745+0000 7f0f0d400640 1 --2- 172.21.15.114:0/3582687530 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f0ee8076300 0x7f0ee8078810 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4034000 tx=0x7f0ef402e000 comp rx=0 tx=0).stop 2024-05-27T09:37:49.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.745+0000 7f0f0d400640 1 -- 172.21.15.114:0/3582687530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f0806c690 msgr2=0x7f0f081e3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:49.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.745+0000 7f0f0d400640 1 --2- 172.21.15.114:0/3582687530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f0806c690 0x7f0f081e3060 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800e940 tx=0x7f0ef800ee00 comp rx=0 tx=0).stop 2024-05-27T09:37:49.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.745+0000 7f0f07200640 1 -- 172.21.15.114:0/3582687530 reap_dead start 2024-05-27T09:37:49.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.746+0000 7f0f0d400640 1 -- 172.21.15.114:0/3582687530 shutdown_connections 2024-05-27T09:37:49.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.746+0000 7f0f0d400640 1 -- 172.21.15.114:0/3582687530 >> 172.21.15.114:0/3582687530 conn(0x7f0f080cd9e0 msgr2=0x7f0f0806cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:49.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.746+0000 7f0f0d400640 1 -- 172.21.15.114:0/3582687530 shutdown_connections 2024-05-27T09:37:49.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:49.746+0000 7f0f0d400640 1 -- 172.21.15.114:0/3582687530 wait complete. 2024-05-27T09:37:49.775 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-27T09:37:50.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.056+0000 7fa643e00640 1 Processor -- start 2024-05-27T09:37:50.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.057+0000 7fa643e00640 1 -- start start 2024-05-27T09:37:50.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.059+0000 7fa643e00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa6440741f0 0x7fa6440c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:50.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.059+0000 7fa643e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fa644074840 0x7fa64415ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:50.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.059+0000 7fa643e00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa64406fe50 con 0x7fa6441547d0 2024-05-27T09:37:50.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.059+0000 7fa643e00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fa6440703d0 con 0x7fa644074840 2024-05-27T09:37:50.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.059+0000 7fa643e00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fa64415b260 con 0x7fa6440741f0 2024-05-27T09:37:50.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.059+0000 7fa642000640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fa6441547d0 0x7fa64406edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55930/0 (socket says 172.21.15.114:55930) 2024-05-27T09:37:50.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.059+0000 7fa642000640 1 -- 172.21.15.114:0/1721570724 learned_addr learned my addr 172.21.15.114:0/1721570724 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:50.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.060+0000 7fa642a00640 1 --2- 172.21.15.114:0/1721570724 >> v2:172.21.15.163:3300/0 conn(0x7fa6440741f0 0x7fa6440c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:50.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.060+0000 7fa643400640 1 --2- 172.21.15.114:0/1721570724 >> v2:172.21.15.114:3300/0 conn(0x7fa644074840 0x7fa64415ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:50.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.060+0000 7fa642a00640 1 -- 172.21.15.114:0/1721570724 >> v2:172.21.15.114:3300/0 conn(0x7fa644074840 msgr2=0x7fa64415ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:50.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.060+0000 7fa642a00640 1 --2- 172.21.15.114:0/1721570724 >> v2:172.21.15.114:3300/0 conn(0x7fa644074840 0x7fa64415ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:50.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.060+0000 7fa642a00640 1 -- 172.21.15.114:0/1721570724 >> v1:172.21.15.114:6789/0 conn(0x7fa6441547d0 legacy=0x7fa64406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:50.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.060+0000 7fa642a00640 1 -- 172.21.15.114:0/1721570724 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6440700f0 con 0x7fa6440741f0 2024-05-27T09:37:50.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.062+0000 7fa642a00640 1 --2- 172.21.15.114:0/1721570724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6440741f0 0x7fa6440c63a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa6440caf30 tx=0x7fa63402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1b410cbe2a8b6f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:50.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.063+0000 7fa641600640 1 -- 172.21.15.114:0/1721570724 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa634037070 con 0x7fa6440741f0 2024-05-27T09:37:50.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.063+0000 7fa641600640 1 -- 172.21.15.114:0/1721570724 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa634032440 con 0x7fa6440741f0 2024-05-27T09:37:50.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.063+0000 7fa641600640 1 -- 172.21.15.114:0/1721570724 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa634032810 con 0x7fa6440741f0 2024-05-27T09:37:50.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.064+0000 7fa643e00640 1 -- 172.21.15.114:0/1721570724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6440741f0 msgr2=0x7fa6440c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:50.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.064+0000 7fa643e00640 1 --2- 172.21.15.114:0/1721570724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6440741f0 0x7fa6440c63a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa6440caf30 tx=0x7fa63402a230 comp rx=0 tx=0).stop 2024-05-27T09:37:50.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.064+0000 7fa643e00640 1 -- 172.21.15.114:0/1721570724 shutdown_connections 2024-05-27T09:37:50.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.064+0000 7fa643e00640 1 --2- 172.21.15.114:0/1721570724 >> v2:172.21.15.114:3300/0 conn(0x7fa644074840 0x7fa64415ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:50.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.064+0000 7fa643e00640 1 --2- 172.21.15.114:0/1721570724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6440741f0 0x7fa6440c63a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:50.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.064+0000 7fa643e00640 1 -- 172.21.15.114:0/1721570724 >> 172.21.15.114:0/1721570724 conn(0x7fa6440ca320 msgr2=0x7fa6440ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:50.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.064+0000 7fa643e00640 1 -- 172.21.15.114:0/1721570724 shutdown_connections 2024-05-27T09:37:50.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.064+0000 7fa643e00640 1 -- 172.21.15.114:0/1721570724 wait complete. 2024-05-27T09:37:50.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.067+0000 7fa643e00640 1 Processor -- start 2024-05-27T09:37:50.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.067+0000 7fa643e00640 1 -- start start 2024-05-27T09:37:50.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.069+0000 7fa643e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6440741f0 0x7fa6441504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:50.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.069+0000 7fa643e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa644074840 0x7fa644152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:50.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.069+0000 7fa643e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6441547d0 0x7fa64414ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:50.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.069+0000 7fa643e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa644073790 con 0x7fa644074840 2024-05-27T09:37:50.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.069+0000 7fa643e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa6440734b0 con 0x7fa6440741f0 2024-05-27T09:37:50.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.069+0000 7fa643e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa64406f500 con 0x7fa6441547d0 2024-05-27T09:37:50.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.069+0000 7fa642000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa644074840 0x7fa644152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:50.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.069+0000 7fa643400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6441547d0 0x7fa64414ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:50.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.069+0000 7fa642000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa644074840 0x7fa644152230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37428/0 (socket says 172.21.15.114:37428) 2024-05-27T09:37:50.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.069+0000 7fa642000640 1 -- 172.21.15.114:0/120097357 learned_addr learned my addr 172.21.15.114:0/120097357 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:50.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.069+0000 7fa643400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6441547d0 0x7fa64414ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:43848/0 (socket says 172.21.15.114:43848) 2024-05-27T09:37:50.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.070+0000 7fa642a00640 1 --2- 172.21.15.114:0/120097357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6440741f0 0x7fa6441504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:50.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.070+0000 7fa643400640 1 -- 172.21.15.114:0/120097357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6440741f0 msgr2=0x7fa6441504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:50.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.070+0000 7fa643400640 1 --2- 172.21.15.114:0/120097357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6440741f0 0x7fa6441504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:50.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.070+0000 7fa643400640 1 -- 172.21.15.114:0/120097357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa644074840 msgr2=0x7fa644152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:50.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.070+0000 7fa643400640 1 --2- 172.21.15.114:0/120097357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa644074840 0x7fa644152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:50.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.070+0000 7fa643400640 1 -- 172.21.15.114:0/120097357 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa644152770 con 0x7fa6441547d0 2024-05-27T09:37:50.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.070+0000 7fa643400640 1 --2- 172.21.15.114:0/120097357 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6441547d0 0x7fa64414ed70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00eba0 tx=0x7fa63c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:50.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.070+0000 7fa63aa00640 1 -- 172.21.15.114:0/120097357 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c018020 con 0x7fa6441547d0 2024-05-27T09:37:50.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.071+0000 7fa63aa00640 1 -- 172.21.15.114:0/120097357 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa63c00cd20 con 0x7fa6441547d0 2024-05-27T09:37:50.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.071+0000 7fa643e00640 1 -- 172.21.15.114:0/120097357 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa644143000 con 0x7fa6441547d0 2024-05-27T09:37:50.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.071+0000 7fa63aa00640 1 -- 172.21.15.114:0/120097357 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c0146c0 con 0x7fa6441547d0 2024-05-27T09:37:50.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.071+0000 7fa643e00640 1 -- 172.21.15.114:0/120097357 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6441435c0 con 0x7fa6441547d0 2024-05-27T09:37:50.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.072+0000 7fa643e00640 1 -- 172.21.15.114:0/120097357 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa644074620 con 0x7fa6441547d0 2024-05-27T09:37:50.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.073+0000 7fa63aa00640 1 -- 172.21.15.114:0/120097357 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa63c01b020 con 0x7fa6441547d0 2024-05-27T09:37:50.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.074+0000 7fa63aa00640 1 --2- 172.21.15.114:0/120097357 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa6240762d0 0x7fa6240787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:50.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.075+0000 7fa63aa00640 1 -- 172.21.15.114:0/120097357 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7fa63c0983e0 con 0x7fa6441547d0 2024-05-27T09:37:50.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.075+0000 7fa642a00640 1 --2- 172.21.15.114:0/120097357 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa6240762d0 0x7fa6240787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:50.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.075+0000 7fa642a00640 1 --2- 172.21.15.114:0/120097357 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa6240762d0 0x7fa6240787e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa634034000 tx=0x7fa63402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:50.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.079+0000 7fa63aa00640 1 -- 172.21.15.114:0/120097357 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa63c022020 con 0x7fa6441547d0 2024-05-27T09:37:50.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.340+0000 7fa643e00640 1 -- 172.21.15.114:0/120097357 --> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fa6440c66b0 con 0x7fa6240762d0 2024-05-27T09:37:50.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.340+0000 7fa63aa00640 1 -- 172.21.15.114:0/120097357 <== mgr.4108 v2:172.21.15.163:6832/1122018204 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa6440c66b0 con 0x7fa6240762d0 2024-05-27T09:37:50.342 INFO:teuthology.orchestra.run.smithi114.stderr:instructed osd(s) 5 to deep-scrub 2024-05-27T09:37:50.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.343+0000 7fa643e00640 1 -- 172.21.15.114:0/120097357 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa6240762d0 msgr2=0x7fa6240787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:50.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.343+0000 7fa643e00640 1 --2- 172.21.15.114:0/120097357 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa6240762d0 0x7fa6240787e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa634034000 tx=0x7fa63402e000 comp rx=0 tx=0).stop 2024-05-27T09:37:50.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.343+0000 7fa643e00640 1 -- 172.21.15.114:0/120097357 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6441547d0 msgr2=0x7fa64414ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:50.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.343+0000 7fa643e00640 1 --2- 172.21.15.114:0/120097357 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6441547d0 0x7fa64414ed70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00eba0 tx=0x7fa63c00c490 comp rx=0 tx=0).stop 2024-05-27T09:37:50.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.343+0000 7fa643e00640 1 -- 172.21.15.114:0/120097357 shutdown_connections 2024-05-27T09:37:50.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.343+0000 7fa643e00640 1 --2- 172.21.15.114:0/120097357 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa6240762d0 0x7fa6240787e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:50.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.343+0000 7fa643e00640 1 --2- 172.21.15.114:0/120097357 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6441547d0 0x7fa64414ed70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:50.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.343+0000 7fa643e00640 1 --2- 172.21.15.114:0/120097357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa644074840 0x7fa644152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:50.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.343+0000 7fa643e00640 1 --2- 172.21.15.114:0/120097357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6440741f0 0x7fa6441504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:50.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.344+0000 7fa643e00640 1 -- 172.21.15.114:0/120097357 >> 172.21.15.114:0/120097357 conn(0x7fa6440ca320 msgr2=0x7fa644150e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:50.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.344+0000 7fa643e00640 1 -- 172.21.15.114:0/120097357 shutdown_connections 2024-05-27T09:37:50.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.344+0000 7fa643e00640 1 -- 172.21.15.114:0/120097357 wait complete. 2024-05-27T09:37:50.377 INFO:tasks.ceph:Scrubbing osd.6 2024-05-27T09:37:50.377 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-27T09:37:50.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.660+0000 7fbf1d400640 1 Processor -- start 2024-05-27T09:37:50.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.660+0000 7fbf1d400640 1 -- start start 2024-05-27T09:37:50.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.662+0000 7fbf1d400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf18073260 0x7fbf1815a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:50.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.663+0000 7fbf1d400640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf18074cb0 con 0x7fbf180c9830 2024-05-27T09:37:50.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.663+0000 7fbf1d400640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf18074130 con 0x7fbf1815ad70 2024-05-27T09:37:50.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.663+0000 7fbf1d400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fbf18073ee0 con 0x7fbf18073260 2024-05-27T09:37:50.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.663+0000 7fbf16800640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fbf180c9830 0x7fbf1806c690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55938/0 (socket says 172.21.15.114:55938) 2024-05-27T09:37:50.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.663+0000 7fbf16800640 1 -- 172.21.15.114:0/3039957098 learned_addr learned my addr 172.21.15.114:0/3039957098 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:50.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.663+0000 7fbf15e00640 1 --2- 172.21.15.114:0/3039957098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf18073260 0x7fbf1815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:50.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.663+0000 7fbf15e00640 1 -- 172.21.15.114:0/3039957098 >> v1:172.21.15.163:6789/0 conn(0x7fbf1815ad70 legacy=0x7fbf18164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:50.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.663+0000 7fbf15e00640 1 -- 172.21.15.114:0/3039957098 >> v1:172.21.15.114:6789/0 conn(0x7fbf180c9830 legacy=0x7fbf1806c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:50.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.663+0000 7fbf15400640 1 -- 172.21.15.114:0/3039957098 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2796533436 0 0) 0x7fbf18074130 con 0x7fbf1815ad70 2024-05-27T09:37:50.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.664+0000 7fbf15e00640 1 -- 172.21.15.114:0/3039957098 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf18074410 con 0x7fbf18073260 2024-05-27T09:37:50.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.665+0000 7fbf15e00640 1 --2- 172.21.15.114:0/3039957098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf18073260 0x7fbf1815a830 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbf040097d0 tx=0x7fbf0402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb2acb7a00810c71 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:50.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.665+0000 7fbf15400640 1 -- 172.21.15.114:0/3039957098 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf04037070 con 0x7fbf18073260 2024-05-27T09:37:50.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.665+0000 7fbf15400640 1 -- 172.21.15.114:0/3039957098 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf0402ae30 con 0x7fbf18073260 2024-05-27T09:37:50.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.665+0000 7fbf15400640 1 -- 172.21.15.114:0/3039957098 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf04032640 con 0x7fbf18073260 2024-05-27T09:37:50.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.666+0000 7fbf1d400640 1 -- 172.21.15.114:0/3039957098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf18073260 msgr2=0x7fbf1815a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:50.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.666+0000 7fbf1d400640 1 --2- 172.21.15.114:0/3039957098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf18073260 0x7fbf1815a830 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbf040097d0 tx=0x7fbf0402a1a0 comp rx=0 tx=0).stop 2024-05-27T09:37:50.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.666+0000 7fbf1d400640 1 -- 172.21.15.114:0/3039957098 shutdown_connections 2024-05-27T09:37:50.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.666+0000 7fbf1d400640 1 --2- 172.21.15.114:0/3039957098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf18073260 0x7fbf1815a830 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:50.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.666+0000 7fbf1d400640 1 -- 172.21.15.114:0/3039957098 >> 172.21.15.114:0/3039957098 conn(0x7fbf180cd9e0 msgr2=0x7fbf180cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:50.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.666+0000 7fbf1d400640 1 -- 172.21.15.114:0/3039957098 shutdown_connections 2024-05-27T09:37:50.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.666+0000 7fbf1d400640 1 -- 172.21.15.114:0/3039957098 wait complete. 2024-05-27T09:37:50.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.668+0000 7fbf1d400640 1 Processor -- start 2024-05-27T09:37:50.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.669+0000 7fbf1d400640 1 -- start start 2024-05-27T09:37:50.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.670+0000 7fbf1d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf18073260 0x7fbf180cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:50.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.670+0000 7fbf1d400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf180c9830 0x7fbf180c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:50.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.670+0000 7fbf1d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf1815ad70 0x7fbf180c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:50.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.670+0000 7fbf1d400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1815b180 con 0x7fbf18073260 2024-05-27T09:37:50.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.670+0000 7fbf1d400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbf180749d0 con 0x7fbf1815ad70 2024-05-27T09:37:50.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.670+0000 7fbf1d400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fbf180706c0 con 0x7fbf180c9830 2024-05-27T09:37:50.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.670+0000 7fbf16800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf18073260 0x7fbf180cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:50.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.670+0000 7fbf16800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf18073260 0x7fbf180cbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37432/0 (socket says 172.21.15.114:37432) 2024-05-27T09:37:50.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.670+0000 7fbf16800640 1 -- 172.21.15.114:0/1061358684 learned_addr learned my addr 172.21.15.114:0/1061358684 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:50.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.670+0000 7fbf15e00640 1 --2- 172.21.15.114:0/1061358684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf180c9830 0x7fbf180c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:50.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.670+0000 7fbf17200640 1 --2- 172.21.15.114:0/1061358684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf1815ad70 0x7fbf180c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:50.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.671+0000 7fbf15e00640 1 -- 172.21.15.114:0/1061358684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf1815ad70 msgr2=0x7fbf180c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:50.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.671+0000 7fbf15e00640 1 --2- 172.21.15.114:0/1061358684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf1815ad70 0x7fbf180c3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:50.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.671+0000 7fbf15e00640 1 -- 172.21.15.114:0/1061358684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf18073260 msgr2=0x7fbf180cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:50.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.671+0000 7fbf15e00640 1 --2- 172.21.15.114:0/1061358684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf18073260 0x7fbf180cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:50.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.671+0000 7fbf15e00640 1 -- 172.21.15.114:0/1061358684 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf180cc460 con 0x7fbf180c9830 2024-05-27T09:37:50.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.671+0000 7fbf17200640 1 --2- 172.21.15.114:0/1061358684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf1815ad70 0x7fbf180c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:50.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.671+0000 7fbf16800640 1 --2- 172.21.15.114:0/1061358684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf18073260 0x7fbf180cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:50.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.671+0000 7fbf15e00640 1 --2- 172.21.15.114:0/1061358684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf180c9830 0x7fbf180c3a20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbf1806d170 tx=0x7fbf040046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:50.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.672+0000 7fbf0ea00640 1 -- 172.21.15.114:0/1061358684 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf04037040 con 0x7fbf180c9830 2024-05-27T09:37:50.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.672+0000 7fbf0ea00640 1 -- 172.21.15.114:0/1061358684 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf04004030 con 0x7fbf180c9830 2024-05-27T09:37:50.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.672+0000 7fbf1d400640 1 -- 172.21.15.114:0/1061358684 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf180c2c10 con 0x7fbf180c9830 2024-05-27T09:37:50.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.672+0000 7fbf0ea00640 1 -- 172.21.15.114:0/1061358684 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf04006e10 con 0x7fbf180c9830 2024-05-27T09:37:50.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.672+0000 7fbf1d400640 1 -- 172.21.15.114:0/1061358684 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf181f6690 con 0x7fbf180c9830 2024-05-27T09:37:50.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.673+0000 7fbf1d400640 1 -- 172.21.15.114:0/1061358684 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbf18153950 con 0x7fbf180c9830 2024-05-27T09:37:50.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.675+0000 7fbf0ea00640 1 -- 172.21.15.114:0/1061358684 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbf0403a040 con 0x7fbf180c9830 2024-05-27T09:37:50.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.676+0000 7fbf0ea00640 1 --2- 172.21.15.114:0/1061358684 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbef8076300 0x7fbef8078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:50.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.676+0000 7fbf0ea00640 1 -- 172.21.15.114:0/1061358684 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7fbf040b8820 con 0x7fbf180c9830 2024-05-27T09:37:50.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.677+0000 7fbf0ea00640 1 --2- 172.21.15.114:0/1061358684 >> [v2:172.21.15.163:6808/3649718505,v1:172.21.15.163:6809/3649718505] conn(0x7fbef807e970 0x7fbef8080de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:50.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.677+0000 7fbf0ea00640 1 -- 172.21.15.114:0/1061358684 --> [v2:172.21.15.163:6808/3649718505,v1:172.21.15.163:6809/3649718505] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbef80814b0 con 0x7fbef807e970 2024-05-27T09:37:50.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.677+0000 7fbf0ea00640 1 -- 172.21.15.114:0/1061358684 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fbf040b8cb0 con 0x7fbf180c9830 2024-05-27T09:37:50.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.677+0000 7fbf16800640 1 --2- 172.21.15.114:0/1061358684 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbef8076300 0x7fbef8078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:50.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.677+0000 7fbf17200640 1 --2- 172.21.15.114:0/1061358684 >> [v2:172.21.15.163:6808/3649718505,v1:172.21.15.163:6809/3649718505] conn(0x7fbef807e970 0x7fbef8080de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:50.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.678+0000 7fbf16800640 1 --2- 172.21.15.114:0/1061358684 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbef8076300 0x7fbef8078810 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbf08000f60 tx=0x7fbf08000bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:50.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.678+0000 7fbf17200640 1 --2- 172.21.15.114:0/1061358684 >> [v2:172.21.15.163:6808/3649718505,v1:172.21.15.163:6809/3649718505] conn(0x7fbef807e970 0x7fbef8080de0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:50.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.679+0000 7fbf0ea00640 1 -- 172.21.15.114:0/1061358684 <== osd.6 v2:172.21.15.163:6808/3649718505 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbef80814b0 con 0x7fbef807e970 2024-05-27T09:37:50.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.693+0000 7fbf1d400640 1 -- 172.21.15.114:0/1061358684 --> [v2:172.21.15.163:6808/3649718505,v1:172.21.15.163:6809/3649718505] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbf181f6160 con 0x7fbef807e970 2024-05-27T09:37:50.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.694+0000 7fbf0ea00640 1 -- 172.21.15.114:0/1061358684 <== osd.6 v2:172.21.15.163:6808/3649718505 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbf181f6160 con 0x7fbef807e970 2024-05-27T09:37:50.696 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-05-27T09:37:50.696 INFO:teuthology.orchestra.run.smithi114.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-27T09:37:50.696 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-05-27T09:37:50.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.694+0000 7fbf1d400640 1 -- 172.21.15.114:0/1061358684 >> [v2:172.21.15.163:6808/3649718505,v1:172.21.15.163:6809/3649718505] conn(0x7fbef807e970 msgr2=0x7fbef8080de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:50.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.694+0000 7fbf1d400640 1 --2- 172.21.15.114:0/1061358684 >> [v2:172.21.15.163:6808/3649718505,v1:172.21.15.163:6809/3649718505] conn(0x7fbef807e970 0x7fbef8080de0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:50.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.694+0000 7fbf1d400640 1 -- 172.21.15.114:0/1061358684 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbef8076300 msgr2=0x7fbef8078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:50.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.694+0000 7fbf1d400640 1 --2- 172.21.15.114:0/1061358684 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbef8076300 0x7fbef8078810 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbf08000f60 tx=0x7fbf08000bc0 comp rx=0 tx=0).stop 2024-05-27T09:37:50.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.694+0000 7fbf1d400640 1 -- 172.21.15.114:0/1061358684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf180c9830 msgr2=0x7fbf180c3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:50.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.694+0000 7fbf1d400640 1 --2- 172.21.15.114:0/1061358684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf180c9830 0x7fbf180c3a20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbf1806d170 tx=0x7fbf040046c0 comp rx=0 tx=0).stop 2024-05-27T09:37:50.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.694+0000 7fbf17200640 1 -- 172.21.15.114:0/1061358684 reap_dead start 2024-05-27T09:37:50.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.694+0000 7fbf1d400640 1 -- 172.21.15.114:0/1061358684 shutdown_connections 2024-05-27T09:37:50.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.694+0000 7fbf1d400640 1 -- 172.21.15.114:0/1061358684 >> 172.21.15.114:0/1061358684 conn(0x7fbf180cd9e0 msgr2=0x7fbf1806f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:50.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.695+0000 7fbf1d400640 1 -- 172.21.15.114:0/1061358684 shutdown_connections 2024-05-27T09:37:50.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.695+0000 7fbf1d400640 1 -- 172.21.15.114:0/1061358684 wait complete. 2024-05-27T09:37:50.719 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-27T09:37:50.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.991+0000 7fcff6400640 1 Processor -- start 2024-05-27T09:37:50.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.991+0000 7fcff6400640 1 -- start start 2024-05-27T09:37:50.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.993+0000 7fcff6400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcff00741f0 0x7fcff00c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:50.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.993+0000 7fcff6400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcff01547d0 0x7fcff015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:50.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.993+0000 7fcff6400640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcff006fe50 con 0x7fcff0074840 2024-05-27T09:37:50.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.993+0000 7fcff6400640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fcff00703d0 con 0x7fcff00741f0 2024-05-27T09:37:50.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.993+0000 7fcff6400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fcff015b260 con 0x7fcff01547d0 2024-05-27T09:37:50.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.993+0000 7fcff5000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcff00741f0 0x7fcff00c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:50.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.993+0000 7fcfefe00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcff01547d0 0x7fcff015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:50.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.993+0000 7fcff5000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcff00741f0 0x7fcff00c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:43048/0 (socket says 172.21.15.114:43048) 2024-05-27T09:37:50.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.993+0000 7fcfefe00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcff01547d0 0x7fcff015ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:43882/0 (socket says 172.21.15.114:43882) 2024-05-27T09:37:50.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.993+0000 7fcff5000640 1 -- 172.21.15.114:0/1615305310 learned_addr learned my addr 172.21.15.114:0/1615305310 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:51.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.994+0000 7fcff5000640 1 -- 172.21.15.114:0/1615305310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcff01547d0 msgr2=0x7fcff015ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:51.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.994+0000 7fcff5000640 1 --2- 172.21.15.114:0/1615305310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcff01547d0 0x7fcff015ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:51.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.994+0000 7fcff5000640 1 -- 172.21.15.114:0/1615305310 >> v1:172.21.15.114:6789/0 conn(0x7fcff0074840 legacy=0x7fcff006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:51.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.994+0000 7fcff5000640 1 -- 172.21.15.114:0/1615305310 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff00700f0 con 0x7fcff00741f0 2024-05-27T09:37:51.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.994+0000 7fcfefe00640 1 --2- 172.21.15.114:0/1615305310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcff01547d0 0x7fcff015ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:51.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.995+0000 7fcff5000640 1 --2- 172.21.15.114:0/1615305310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcff00741f0 0x7fcff00c63a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009870 tx=0x7fcfe402a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c3e02aac54b5996 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:51.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.996+0000 7fcfef400640 1 -- 172.21.15.114:0/1615305310 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4037070 con 0x7fcff00741f0 2024-05-27T09:37:51.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.996+0000 7fcfef400640 1 -- 172.21.15.114:0/1615305310 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe402ae50 con 0x7fcff00741f0 2024-05-27T09:37:51.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.996+0000 7fcfef400640 1 -- 172.21.15.114:0/1615305310 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4032860 con 0x7fcff00741f0 2024-05-27T09:37:51.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.997+0000 7fcff6400640 1 -- 172.21.15.114:0/1615305310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcff00741f0 msgr2=0x7fcff00c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:51.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.997+0000 7fcff6400640 1 --2- 172.21.15.114:0/1615305310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcff00741f0 0x7fcff00c63a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009870 tx=0x7fcfe402a300 comp rx=0 tx=0).stop 2024-05-27T09:37:51.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.997+0000 7fcff6400640 1 -- 172.21.15.114:0/1615305310 shutdown_connections 2024-05-27T09:37:51.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.997+0000 7fcff6400640 1 --2- 172.21.15.114:0/1615305310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcff01547d0 0x7fcff015ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:51.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.997+0000 7fcff6400640 1 --2- 172.21.15.114:0/1615305310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcff00741f0 0x7fcff00c63a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:51.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.997+0000 7fcff6400640 1 -- 172.21.15.114:0/1615305310 >> 172.21.15.114:0/1615305310 conn(0x7fcff00ca320 msgr2=0x7fcff00ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:51.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.997+0000 7fcff6400640 1 -- 172.21.15.114:0/1615305310 shutdown_connections 2024-05-27T09:37:51.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:50.997+0000 7fcff6400640 1 -- 172.21.15.114:0/1615305310 wait complete. 2024-05-27T09:37:51.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.000+0000 7fcff6400640 1 Processor -- start 2024-05-27T09:37:51.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.001+0000 7fcff6400640 1 -- start start 2024-05-27T09:37:51.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.003+0000 7fcff6400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcff00741f0 0x7fcff01ee940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:51.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.003+0000 7fcff6400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcff0074840 0x7fcff0152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:51.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.003+0000 7fcff6400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcff01547d0 0x7fcff015dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:51.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.003+0000 7fcff6400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcff0073790 con 0x7fcff0074840 2024-05-27T09:37:51.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.003+0000 7fcff6400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcff00734b0 con 0x7fcff01547d0 2024-05-27T09:37:51.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.003+0000 7fcff6400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fcff006f500 con 0x7fcff00741f0 2024-05-27T09:37:51.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.003+0000 7fcff5000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcff00741f0 0x7fcff01ee940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:51.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.003+0000 7fcff5000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcff00741f0 0x7fcff01ee940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:43896/0 (socket says 172.21.15.114:43896) 2024-05-27T09:37:51.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.003+0000 7fcff5000640 1 -- 172.21.15.114:0/4067848330 learned_addr learned my addr 172.21.15.114:0/4067848330 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:51.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.003+0000 7fcfefe00640 1 --2- 172.21.15.114:0/4067848330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcff0074840 0x7fcff0152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:51.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.004+0000 7fcff5a00640 1 --2- 172.21.15.114:0/4067848330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcff01547d0 0x7fcff015dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:51.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.004+0000 7fcff5000640 1 -- 172.21.15.114:0/4067848330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcff01547d0 msgr2=0x7fcff015dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:51.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.004+0000 7fcff5000640 1 --2- 172.21.15.114:0/4067848330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcff01547d0 0x7fcff015dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:51.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.004+0000 7fcff5000640 1 -- 172.21.15.114:0/4067848330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcff0074840 msgr2=0x7fcff0152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:51.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.004+0000 7fcff5000640 1 --2- 172.21.15.114:0/4067848330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcff0074840 0x7fcff0152e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:51.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.004+0000 7fcff5000640 1 -- 172.21.15.114:0/4067848330 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff01f2d10 con 0x7fcff00741f0 2024-05-27T09:37:51.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.004+0000 7fcff5000640 1 --2- 172.21.15.114:0/4067848330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcff00741f0 0x7fcff01ee940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4000c00 tx=0x7fcfe40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:51.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.005+0000 7fcfecc00640 1 -- 172.21.15.114:0/4067848330 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4041070 con 0x7fcff00741f0 2024-05-27T09:37:51.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.005+0000 7fcfecc00640 1 -- 172.21.15.114:0/4067848330 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe403c660 con 0x7fcff00741f0 2024-05-27T09:37:51.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.005+0000 7fcfecc00640 1 -- 172.21.15.114:0/4067848330 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4037040 con 0x7fcff00741f0 2024-05-27T09:37:51.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.005+0000 7fcff6400640 1 -- 172.21.15.114:0/4067848330 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff01472d0 con 0x7fcff00741f0 2024-05-27T09:37:51.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.005+0000 7fcff6400640 1 -- 172.21.15.114:0/4067848330 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff01f2230 con 0x7fcff00741f0 2024-05-27T09:37:51.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.007+0000 7fcff6400640 1 -- 172.21.15.114:0/4067848330 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff006f290 con 0x7fcff00741f0 2024-05-27T09:37:51.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.009+0000 7fcfecc00640 1 -- 172.21.15.114:0/4067848330 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcfe4044070 con 0x7fcff00741f0 2024-05-27T09:37:51.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.010+0000 7fcfecc00640 1 --2- 172.21.15.114:0/4067848330 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcfd00762d0 0x7fcfd00787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:51.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.010+0000 7fcfecc00640 1 -- 172.21.15.114:0/4067848330 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7fcfe40b7f00 con 0x7fcff00741f0 2024-05-27T09:37:51.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.010+0000 7fcfefe00640 1 --2- 172.21.15.114:0/4067848330 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcfd00762d0 0x7fcfd00787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:51.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.011+0000 7fcfefe00640 1 --2- 172.21.15.114:0/4067848330 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcfd00762d0 0x7fcfd00787e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcff014e880 tx=0x7fcfdc00f040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:51.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.015+0000 7fcfecc00640 1 -- 172.21.15.114:0/4067848330 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcfe40816d0 con 0x7fcff00741f0 2024-05-27T09:37:51.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.239+0000 7fcff6400640 1 -- 172.21.15.114:0/4067848330 --> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fcff00b3230 con 0x7fcfd00762d0 2024-05-27T09:37:51.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.239+0000 7fcfecc00640 1 -- 172.21.15.114:0/4067848330 <== mgr.4108 v2:172.21.15.163:6832/1122018204 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcff00b3230 con 0x7fcfd00762d0 2024-05-27T09:37:51.241 INFO:teuthology.orchestra.run.smithi114.stderr:instructed osd(s) 6 to deep-scrub 2024-05-27T09:37:51.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.243+0000 7fcff6400640 1 -- 172.21.15.114:0/4067848330 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcfd00762d0 msgr2=0x7fcfd00787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:51.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.243+0000 7fcff6400640 1 --2- 172.21.15.114:0/4067848330 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcfd00762d0 0x7fcfd00787e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcff014e880 tx=0x7fcfdc00f040 comp rx=0 tx=0).stop 2024-05-27T09:37:51.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.243+0000 7fcff6400640 1 -- 172.21.15.114:0/4067848330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcff00741f0 msgr2=0x7fcff01ee940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:51.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.243+0000 7fcff6400640 1 --2- 172.21.15.114:0/4067848330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcff00741f0 0x7fcff01ee940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4000c00 tx=0x7fcfe40046c0 comp rx=0 tx=0).stop 2024-05-27T09:37:51.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.243+0000 7fcff6400640 1 -- 172.21.15.114:0/4067848330 shutdown_connections 2024-05-27T09:37:51.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.243+0000 7fcff6400640 1 --2- 172.21.15.114:0/4067848330 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fcfd00762d0 0x7fcfd00787e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:51.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.243+0000 7fcff6400640 1 --2- 172.21.15.114:0/4067848330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcff01547d0 0x7fcff015dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:51.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.243+0000 7fcff6400640 1 --2- 172.21.15.114:0/4067848330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcff0074840 0x7fcff0152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:51.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.243+0000 7fcff6400640 1 --2- 172.21.15.114:0/4067848330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcff00741f0 0x7fcff01ee940 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:51.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.243+0000 7fcff6400640 1 -- 172.21.15.114:0/4067848330 >> 172.21.15.114:0/4067848330 conn(0x7fcff00ca320 msgr2=0x7fcff0158110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:51.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.243+0000 7fcff6400640 1 -- 172.21.15.114:0/4067848330 shutdown_connections 2024-05-27T09:37:51.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.243+0000 7fcff6400640 1 -- 172.21.15.114:0/4067848330 wait complete. 2024-05-27T09:37:51.288 INFO:tasks.ceph:Scrubbing osd.7 2024-05-27T09:37:51.288 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-27T09:37:51.415 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:37:51.421 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:37:51.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.588+0000 7fcaa7400640 1 Processor -- start 2024-05-27T09:37:51.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.588+0000 7fcaa7400640 1 -- start start 2024-05-27T09:37:51.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.590+0000 7fcaa7400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaa0073260 0x7fcaa015a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:51.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.590+0000 7fcaa7400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcaa015ad70 0x7fcaa0164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:51.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.590+0000 7fcaa7400640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaa0074cb0 con 0x7fcaa00c9830 2024-05-27T09:37:51.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.590+0000 7fcaa7400640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fcaa0073ee0 con 0x7fcaa015ad70 2024-05-27T09:37:51.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.590+0000 7fcaa7400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fcaa0074130 con 0x7fcaa0073260 2024-05-27T09:37:51.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.590+0000 7fcaa5400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcaa015ad70 0x7fcaa0164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:51.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.590+0000 7fca9fe00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaa0073260 0x7fcaa015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:51.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.590+0000 7fcaa4a00640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fcaa00c9830 0x7fcaa006c690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:52904/0 (socket says 172.21.15.114:52904) 2024-05-27T09:37:51.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.590+0000 7fcaa4a00640 1 -- 172.21.15.114:0/712607699 learned_addr learned my addr 172.21.15.114:0/712607699 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:51.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.591+0000 7fca9f400640 1 -- 172.21.15.114:0/712607699 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1566593396 0 0) 0x7fcaa0074cb0 con 0x7fcaa00c9830 2024-05-27T09:37:51.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.591+0000 7fcaa5400640 1 -- 172.21.15.114:0/712607699 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaa0073260 msgr2=0x7fcaa015a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:51.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.591+0000 7fcaa5400640 1 --2- 172.21.15.114:0/712607699 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaa0073260 0x7fcaa015a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:51.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.591+0000 7fcaa5400640 1 -- 172.21.15.114:0/712607699 >> v1:172.21.15.163:6789/0 conn(0x7fcaa00c9830 legacy=0x7fcaa006c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:51.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.591+0000 7fcaa5400640 1 -- 172.21.15.114:0/712607699 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa0074410 con 0x7fcaa015ad70 2024-05-27T09:37:51.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.591+0000 7fca9fe00640 1 --2- 172.21.15.114:0/712607699 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaa0073260 0x7fcaa015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:37:51.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.592+0000 7fcaa5400640 1 --2- 172.21.15.114:0/712607699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaa015ad70 0x7fcaa0164dd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fca98009d90 tx=0x7fca9802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a255d58ad93f063 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:51.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.592+0000 7fca9f400640 1 -- 172.21.15.114:0/712607699 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98007740 con 0x7fcaa015ad70 2024-05-27T09:37:51.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.592+0000 7fca9f400640 1 -- 172.21.15.114:0/712607699 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca980078e0 con 0x7fcaa015ad70 2024-05-27T09:37:51.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.592+0000 7fca9f400640 1 -- 172.21.15.114:0/712607699 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98007cb0 con 0x7fcaa015ad70 2024-05-27T09:37:51.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.593+0000 7fcaa7400640 1 -- 172.21.15.114:0/712607699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaa015ad70 msgr2=0x7fcaa0164dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:51.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.593+0000 7fcaa7400640 1 --2- 172.21.15.114:0/712607699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaa015ad70 0x7fcaa0164dd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fca98009d90 tx=0x7fca9802a6f0 comp rx=0 tx=0).stop 2024-05-27T09:37:51.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.593+0000 7fcaa7400640 1 -- 172.21.15.114:0/712607699 shutdown_connections 2024-05-27T09:37:51.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.593+0000 7fcaa7400640 1 --2- 172.21.15.114:0/712607699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaa015ad70 0x7fcaa0164dd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:51.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.593+0000 7fcaa7400640 1 --2- 172.21.15.114:0/712607699 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaa0073260 0x7fcaa015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:51.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.593+0000 7fcaa7400640 1 -- 172.21.15.114:0/712607699 >> 172.21.15.114:0/712607699 conn(0x7fcaa00cd9e0 msgr2=0x7fcaa00cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:51.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.593+0000 7fcaa7400640 1 -- 172.21.15.114:0/712607699 shutdown_connections 2024-05-27T09:37:51.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.593+0000 7fcaa7400640 1 -- 172.21.15.114:0/712607699 wait complete. 2024-05-27T09:37:51.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.595+0000 7fcaa7400640 1 Processor -- start 2024-05-27T09:37:51.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.595+0000 7fcaa7400640 1 -- start start 2024-05-27T09:37:51.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.597+0000 7fcaa7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcaa0073260 0x7fcaa00c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:51.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.597+0000 7fcaa7400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaa00c9830 0x7fcaa00ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:51.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.597+0000 7fcaa7400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaa015ad70 0x7fcaa0159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:51.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.597+0000 7fcaa7400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa015b180 con 0x7fcaa0073260 2024-05-27T09:37:51.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.597+0000 7fcaa7400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa00749d0 con 0x7fcaa00c9830 2024-05-27T09:37:51.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.597+0000 7fcaa7400640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fcaa00706c0 con 0x7fcaa015ad70 2024-05-27T09:37:51.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.597+0000 7fcaa4a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcaa0073260 0x7fcaa00c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:51.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.597+0000 7fcaa4a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcaa0073260 0x7fcaa00c67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37444/0 (socket says 172.21.15.114:37444) 2024-05-27T09:37:51.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.597+0000 7fcaa4a00640 1 -- 172.21.15.114:0/3104194132 learned_addr learned my addr 172.21.15.114:0/3104194132 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:51.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.597+0000 7fca9fe00640 1 --2- 172.21.15.114:0/3104194132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaa00c9830 0x7fcaa00ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:51.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.597+0000 7fcaa5400640 1 --2- 172.21.15.114:0/3104194132 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaa015ad70 0x7fcaa0159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:51.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.597+0000 7fcaa4a00640 1 -- 172.21.15.114:0/3104194132 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaa015ad70 msgr2=0x7fcaa0159590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:51.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.597+0000 7fcaa4a00640 1 --2- 172.21.15.114:0/3104194132 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaa015ad70 0x7fcaa0159590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:51.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.597+0000 7fcaa4a00640 1 -- 172.21.15.114:0/3104194132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaa00c9830 msgr2=0x7fcaa00ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:51.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.597+0000 7fcaa4a00640 1 --2- 172.21.15.114:0/3104194132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaa00c9830 0x7fcaa00ca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:51.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.597+0000 7fcaa4a00640 1 -- 172.21.15.114:0/3104194132 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa00c6d30 con 0x7fcaa0073260 2024-05-27T09:37:51.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.598+0000 7fcaa4a00640 1 --2- 172.21.15.114:0/3104194132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcaa0073260 0x7fcaa00c67f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fca9400c410 tx=0x7fca9400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:51.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.598+0000 7fca9cc00640 1 -- 172.21.15.114:0/3104194132 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca940074e0 con 0x7fcaa0073260 2024-05-27T09:37:51.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.598+0000 7fca9cc00640 1 -- 172.21.15.114:0/3104194132 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca940047e0 con 0x7fcaa0073260 2024-05-27T09:37:51.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.598+0000 7fcaa7400640 1 -- 172.21.15.114:0/3104194132 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa01f0c60 con 0x7fcaa0073260 2024-05-27T09:37:51.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.598+0000 7fca9cc00640 1 -- 172.21.15.114:0/3104194132 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca94004bd0 con 0x7fcaa0073260 2024-05-27T09:37:51.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.598+0000 7fcaa7400640 1 -- 172.21.15.114:0/3104194132 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa00cd2a0 con 0x7fcaa0073260 2024-05-27T09:37:51.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.599+0000 7fcaa7400640 1 -- 172.21.15.114:0/3104194132 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcaa00c5dc0 con 0x7fcaa0073260 2024-05-27T09:37:51.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.600+0000 7fca9cc00640 1 -- 172.21.15.114:0/3104194132 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fca94011070 con 0x7fcaa0073260 2024-05-27T09:37:51.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.601+0000 7fca9cc00640 1 --2- 172.21.15.114:0/3104194132 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fca80076300 0x7fca80078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:51.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.602+0000 7fca9cc00640 1 -- 172.21.15.114:0/3104194132 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7fca940972c0 con 0x7fcaa0073260 2024-05-27T09:37:51.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.602+0000 7fca9cc00640 1 --2- 172.21.15.114:0/3104194132 >> [v2:172.21.15.163:6824/2539451311,v1:172.21.15.163:6825/2539451311] conn(0x7fca8007e970 0x7fca80080de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:51.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.602+0000 7fca9cc00640 1 -- 172.21.15.114:0/3104194132 --> [v2:172.21.15.163:6824/2539451311,v1:172.21.15.163:6825/2539451311] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fca800814b0 con 0x7fca8007e970 2024-05-27T09:37:51.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.602+0000 7fca9cc00640 1 -- 172.21.15.114:0/3104194132 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fca9409e050 con 0x7fcaa0073260 2024-05-27T09:37:51.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.602+0000 7fca9fe00640 1 --2- 172.21.15.114:0/3104194132 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fca80076300 0x7fca80078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:51.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.602+0000 7fcaa5400640 1 --2- 172.21.15.114:0/3104194132 >> [v2:172.21.15.163:6824/2539451311,v1:172.21.15.163:6825/2539451311] conn(0x7fca8007e970 0x7fca80080de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:51.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.603+0000 7fca9fe00640 1 --2- 172.21.15.114:0/3104194132 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fca80076300 0x7fca80078810 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00c8140 tx=0x7fca8c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:51.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.603+0000 7fcaa5400640 1 --2- 172.21.15.114:0/3104194132 >> [v2:172.21.15.163:6824/2539451311,v1:172.21.15.163:6825/2539451311] conn(0x7fca8007e970 0x7fca80080de0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:51.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.604+0000 7fca9cc00640 1 -- 172.21.15.114:0/3104194132 <== osd.7 v2:172.21.15.163:6824/2539451311 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fca800814b0 con 0x7fca8007e970 2024-05-27T09:37:51.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.621+0000 7fcaa7400640 1 -- 172.21.15.114:0/3104194132 --> [v2:172.21.15.163:6824/2539451311,v1:172.21.15.163:6825/2539451311] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fcaa006a380 con 0x7fca8007e970 2024-05-27T09:37:51.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.621+0000 7fca9cc00640 1 -- 172.21.15.114:0/3104194132 <== osd.7 v2:172.21.15.163:6824/2539451311 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fcaa006a380 con 0x7fca8007e970 2024-05-27T09:37:51.623 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-05-27T09:37:51.623 INFO:teuthology.orchestra.run.smithi114.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-27T09:37:51.623 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-05-27T09:37:51.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.622+0000 7fcaa7400640 1 -- 172.21.15.114:0/3104194132 >> [v2:172.21.15.163:6824/2539451311,v1:172.21.15.163:6825/2539451311] conn(0x7fca8007e970 msgr2=0x7fca80080de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:51.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.622+0000 7fcaa7400640 1 --2- 172.21.15.114:0/3104194132 >> [v2:172.21.15.163:6824/2539451311,v1:172.21.15.163:6825/2539451311] conn(0x7fca8007e970 0x7fca80080de0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:51.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.622+0000 7fcaa7400640 1 -- 172.21.15.114:0/3104194132 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fca80076300 msgr2=0x7fca80078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:51.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.622+0000 7fcaa7400640 1 --2- 172.21.15.114:0/3104194132 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fca80076300 0x7fca80078810 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00c8140 tx=0x7fca8c009190 comp rx=0 tx=0).stop 2024-05-27T09:37:51.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.622+0000 7fcaa7400640 1 -- 172.21.15.114:0/3104194132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcaa0073260 msgr2=0x7fcaa00c67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:51.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.622+0000 7fcaa7400640 1 --2- 172.21.15.114:0/3104194132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcaa0073260 0x7fcaa00c67f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fca9400c410 tx=0x7fca9400c8d0 comp rx=0 tx=0).stop 2024-05-27T09:37:51.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.622+0000 7fcaa5400640 1 -- 172.21.15.114:0/3104194132 reap_dead start 2024-05-27T09:37:51.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.622+0000 7fcaa7400640 1 -- 172.21.15.114:0/3104194132 shutdown_connections 2024-05-27T09:37:51.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.622+0000 7fcaa7400640 1 -- 172.21.15.114:0/3104194132 >> 172.21.15.114:0/3104194132 conn(0x7fcaa00cd9e0 msgr2=0x7fcaa014f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:51.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.622+0000 7fcaa7400640 1 -- 172.21.15.114:0/3104194132 shutdown_connections 2024-05-27T09:37:51.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.622+0000 7fcaa7400640 1 -- 172.21.15.114:0/3104194132 wait complete. 2024-05-27T09:37:51.649 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-27T09:37:51.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.905+0000 7fb0e0e00640 1 Processor -- start 2024-05-27T09:37:51.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.906+0000 7fb0e0e00640 1 -- start start 2024-05-27T09:37:51.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.908+0000 7fb0e0e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0dc074a80 0x7fb0dc15cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:51.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.908+0000 7fb0e0e00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0dc0c8470 con 0x7fb0dc070060 2024-05-27T09:37:51.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.908+0000 7fb0e0e00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0dc0c8dd0 con 0x7fb0dc156880 2024-05-27T09:37:51.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.908+0000 7fb0e0e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fb0dc0c8220 con 0x7fb0dc074a80 2024-05-27T09:37:51.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.908+0000 7fb0dbe00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fb0dc070060 0x7fb0dc166de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:55954/0 (socket says 172.21.15.114:55954) 2024-05-27T09:37:51.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.908+0000 7fb0dbe00640 1 -- 172.21.15.114:0/1442228966 learned_addr learned my addr 172.21.15.114:0/1442228966 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:37:51.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.908+0000 7fb0db400640 1 --2- 172.21.15.114:0/1442228966 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0dc074a80 0x7fb0dc15cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:51.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.909+0000 7fb0db400640 1 -- 172.21.15.114:0/1442228966 >> v1:172.21.15.163:6789/0 conn(0x7fb0dc156880 legacy=0x7fb0dc06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:51.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.909+0000 7fb0daa00640 1 -- 172.21.15.114:0/1442228966 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1219180110 0 0) 0x7fb0dc0c8470 con 0x7fb0dc070060 2024-05-27T09:37:51.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.909+0000 7fb0db400640 1 -- 172.21.15.114:0/1442228966 >> v1:172.21.15.114:6789/0 conn(0x7fb0dc070060 legacy=0x7fb0dc166de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:51.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.909+0000 7fb0db400640 1 -- 172.21.15.114:0/1442228966 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0dc15c570 con 0x7fb0dc074a80 2024-05-27T09:37:51.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.910+0000 7fb0db400640 1 --2- 172.21.15.114:0/1442228966 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0dc074a80 0x7fb0dc15cd70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009750 tx=0x7fb0cc02a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=870bde2f9c22de4c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:51.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.910+0000 7fb0daa00640 1 -- 172.21.15.114:0/1442228966 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc037070 con 0x7fb0dc074a80 2024-05-27T09:37:51.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.910+0000 7fb0daa00640 1 -- 172.21.15.114:0/1442228966 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0cc02adb0 con 0x7fb0dc074a80 2024-05-27T09:37:51.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.910+0000 7fb0daa00640 1 -- 172.21.15.114:0/1442228966 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc032660 con 0x7fb0dc074a80 2024-05-27T09:37:51.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.911+0000 7fb0e0e00640 1 -- 172.21.15.114:0/1442228966 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0dc074a80 msgr2=0x7fb0dc15cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:51.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.911+0000 7fb0e0e00640 1 --2- 172.21.15.114:0/1442228966 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0dc074a80 0x7fb0dc15cd70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009750 tx=0x7fb0cc02a0a0 comp rx=0 tx=0).stop 2024-05-27T09:37:51.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.911+0000 7fb0e0e00640 1 -- 172.21.15.114:0/1442228966 shutdown_connections 2024-05-27T09:37:51.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.911+0000 7fb0e0e00640 1 --2- 172.21.15.114:0/1442228966 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0dc074a80 0x7fb0dc15cd70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:51.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.911+0000 7fb0e0e00640 1 -- 172.21.15.114:0/1442228966 >> 172.21.15.114:0/1442228966 conn(0x7fb0dc0ca320 msgr2=0x7fb0dc0ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:51.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.911+0000 7fb0e0e00640 1 -- 172.21.15.114:0/1442228966 shutdown_connections 2024-05-27T09:37:51.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.911+0000 7fb0e0e00640 1 -- 172.21.15.114:0/1442228966 wait complete. 2024-05-27T09:37:51.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.913+0000 7fb0e0e00640 1 Processor -- start 2024-05-27T09:37:51.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.913+0000 7fb0e0e00640 1 -- start start 2024-05-27T09:37:51.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.915+0000 7fb0e0e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0dc070060 0x7fb0dc14c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:51.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.915+0000 7fb0e0e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0dc074a80 0x7fb0dc1f5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:51.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.915+0000 7fb0e0e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0dc156880 0x7fb0dc151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:51.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.915+0000 7fb0e0e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc15c8e0 con 0x7fb0dc070060 2024-05-27T09:37:51.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.915+0000 7fb0e0e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc0c8750 con 0x7fb0dc074a80 2024-05-27T09:37:51.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.915+0000 7fb0e0e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fb0dc15fa50 con 0x7fb0dc156880 2024-05-27T09:37:51.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.915+0000 7fb0db400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0dc070060 0x7fb0dc14c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:51.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.915+0000 7fb0db400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0dc070060 0x7fb0dc14c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37446/0 (socket says 172.21.15.114:37446) 2024-05-27T09:37:51.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.915+0000 7fb0db400640 1 -- 172.21.15.114:0/152878767 learned_addr learned my addr 172.21.15.114:0/152878767 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:51.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.915+0000 7fb0dbe00640 1 --2- 172.21.15.114:0/152878767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0dc156880 0x7fb0dc151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:51.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.915+0000 7fb0d2a00640 1 --2- 172.21.15.114:0/152878767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0dc074a80 0x7fb0dc1f5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:51.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.915+0000 7fb0db400640 1 -- 172.21.15.114:0/152878767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0dc156880 msgr2=0x7fb0dc151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:51.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.915+0000 7fb0db400640 1 --2- 172.21.15.114:0/152878767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0dc156880 0x7fb0dc151c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:51.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.915+0000 7fb0db400640 1 -- 172.21.15.114:0/152878767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0dc074a80 msgr2=0x7fb0dc1f5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:51.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.915+0000 7fb0db400640 1 --2- 172.21.15.114:0/152878767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0dc074a80 0x7fb0dc1f5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:51.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.915+0000 7fb0db400640 1 -- 172.21.15.114:0/152878767 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0dc14ca20 con 0x7fb0dc070060 2024-05-27T09:37:51.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.916+0000 7fb0db400640 1 --2- 172.21.15.114:0/152878767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0dc070060 0x7fb0dc14c4e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc000c00 tx=0x7fb0cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:51.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.916+0000 7fb0d3e00640 1 -- 172.21.15.114:0/152878767 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc037040 con 0x7fb0dc070060 2024-05-27T09:37:51.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.916+0000 7fb0d3e00640 1 -- 172.21.15.114:0/152878767 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0cc004030 con 0x7fb0dc070060 2024-05-27T09:37:51.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.916+0000 7fb0d3e00640 1 -- 172.21.15.114:0/152878767 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc006e10 con 0x7fb0dc070060 2024-05-27T09:37:51.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.916+0000 7fb0e0e00640 1 -- 172.21.15.114:0/152878767 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0dc1f6340 con 0x7fb0dc070060 2024-05-27T09:37:51.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.916+0000 7fb0e0e00640 1 -- 172.21.15.114:0/152878767 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0dc1f5920 con 0x7fb0dc070060 2024-05-27T09:37:51.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.918+0000 7fb0e0e00640 1 -- 172.21.15.114:0/152878767 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0dc1f5730 con 0x7fb0dc070060 2024-05-27T09:37:51.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.918+0000 7fb0d3e00640 1 -- 172.21.15.114:0/152878767 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb0cc03a070 con 0x7fb0dc070060 2024-05-27T09:37:51.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.919+0000 7fb0d3e00640 1 --2- 172.21.15.114:0/152878767 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb0b80762d0 0x7fb0b80787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:51.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.920+0000 7fb0d3e00640 1 -- 172.21.15.114:0/152878767 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7fb0cc0b7e50 con 0x7fb0dc070060 2024-05-27T09:37:51.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.920+0000 7fb0d2a00640 1 --2- 172.21.15.114:0/152878767 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb0b80762d0 0x7fb0b80787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:51.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.921+0000 7fb0d2a00640 1 --2- 172.21.15.114:0/152878767 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb0b80762d0 0x7fb0b80787e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc155060 tx=0x7fb0c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:51.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:51.924+0000 7fb0d3e00640 1 -- 172.21.15.114:0/152878767 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb0dc14ca20 con 0x7fb0dc070060 2024-05-27T09:37:52.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.208+0000 7fb0e0e00640 1 -- 172.21.15.114:0/152878767 --> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fb0dc050860 con 0x7fb0b80762d0 2024-05-27T09:37:52.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.208+0000 7fb0d3e00640 1 -- 172.21.15.114:0/152878767 <== mgr.4108 v2:172.21.15.163:6832/1122018204 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb0dc050860 con 0x7fb0b80762d0 2024-05-27T09:37:52.210 INFO:teuthology.orchestra.run.smithi114.stderr:instructed osd(s) 7 to deep-scrub 2024-05-27T09:37:52.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.211+0000 7fb0e0e00640 1 -- 172.21.15.114:0/152878767 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb0b80762d0 msgr2=0x7fb0b80787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:52.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.211+0000 7fb0e0e00640 1 --2- 172.21.15.114:0/152878767 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb0b80762d0 0x7fb0b80787e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc155060 tx=0x7fb0c4009040 comp rx=0 tx=0).stop 2024-05-27T09:37:52.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.211+0000 7fb0e0e00640 1 -- 172.21.15.114:0/152878767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0dc070060 msgr2=0x7fb0dc14c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:52.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.211+0000 7fb0e0e00640 1 --2- 172.21.15.114:0/152878767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0dc070060 0x7fb0dc14c4e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc000c00 tx=0x7fb0cc0046c0 comp rx=0 tx=0).stop 2024-05-27T09:37:52.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.211+0000 7fb0e0e00640 1 -- 172.21.15.114:0/152878767 shutdown_connections 2024-05-27T09:37:52.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.211+0000 7fb0e0e00640 1 --2- 172.21.15.114:0/152878767 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fb0b80762d0 0x7fb0b80787e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:52.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.211+0000 7fb0e0e00640 1 --2- 172.21.15.114:0/152878767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0dc156880 0x7fb0dc151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:52.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.211+0000 7fb0e0e00640 1 --2- 172.21.15.114:0/152878767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0dc074a80 0x7fb0dc1f5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:52.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.211+0000 7fb0e0e00640 1 --2- 172.21.15.114:0/152878767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0dc070060 0x7fb0dc14c4e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:52.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.211+0000 7fb0e0e00640 1 -- 172.21.15.114:0/152878767 >> 172.21.15.114:0/152878767 conn(0x7fb0dc0ca320 msgr2=0x7fb0dc14f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:52.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.211+0000 7fb0e0e00640 1 -- 172.21.15.114:0/152878767 shutdown_connections 2024-05-27T09:37:52.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.211+0000 7fb0e0e00640 1 -- 172.21.15.114:0/152878767 wait complete. 2024-05-27T09:37:52.235 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T09:37:52.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.523+0000 7f2440a00640 1 Processor -- start 2024-05-27T09:37:52.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.523+0000 7f2440a00640 1 -- start start 2024-05-27T09:37:52.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.525+0000 7f2440a00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f243c0b75b0 0x7f243c0d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:52.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.525+0000 7f2440a00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7f243c0b46e0 0x7f243c0d2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:52.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.525+0000 7f2440a00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f243c0cd1e0 con 0x7f243c0c8780 2024-05-27T09:37:52.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.525+0000 7f2440a00640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7f243c0cd4c0 con 0x7f243c0b46e0 2024-05-27T09:37:52.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.525+0000 7f2440a00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f243c0c7f80 con 0x7f243c0b75b0 2024-05-27T09:37:52.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.526+0000 7f243aa00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f243c0b75b0 0x7f243c0d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:52.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.526+0000 7f243aa00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f243c0b75b0 0x7f243c0d1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:43094/0 (socket says 172.21.15.114:43094) 2024-05-27T09:37:52.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.526+0000 7f243aa00640 1 -- 172.21.15.114:0/1017827328 learned_addr learned my addr 172.21.15.114:0/1017827328 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:52.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.526+0000 7f243be00640 1 --2- 172.21.15.114:0/1017827328 >> v2:172.21.15.114:3300/0 conn(0x7f243c0b46e0 0x7f243c0d2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:52.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.526+0000 7f243aa00640 1 -- 172.21.15.114:0/1017827328 >> v2:172.21.15.114:3300/0 conn(0x7f243c0b46e0 msgr2=0x7f243c0d2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:37:52.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.526+0000 7f243aa00640 1 --2- 172.21.15.114:0/1017827328 >> v2:172.21.15.114:3300/0 conn(0x7f243c0b46e0 0x7f243c0d2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:52.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.526+0000 7f243aa00640 1 -- 172.21.15.114:0/1017827328 >> v1:172.21.15.114:6789/0 conn(0x7f243c0c8780 legacy=0x7f243c15fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:52.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.526+0000 7f243aa00640 1 -- 172.21.15.114:0/1017827328 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243c0c6bb0 con 0x7f243c0b75b0 2024-05-27T09:37:52.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.527+0000 7f243aa00640 1 --2- 172.21.15.114:0/1017827328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f243c0b75b0 0x7f243c0d1e50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f243c0d2930 tx=0x7f242802a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3e139c97fb0ff9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:52.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.527+0000 7f243a000640 1 -- 172.21.15.114:0/1017827328 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2428037070 con 0x7f243c0b75b0 2024-05-27T09:37:52.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.528+0000 7f243a000640 1 -- 172.21.15.114:0/1017827328 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f242802ae50 con 0x7f243c0b75b0 2024-05-27T09:37:52.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.528+0000 7f243a000640 1 -- 172.21.15.114:0/1017827328 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2428032860 con 0x7f243c0b75b0 2024-05-27T09:37:52.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.528+0000 7f2440a00640 1 -- 172.21.15.114:0/1017827328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f243c0b75b0 msgr2=0x7f243c0d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:52.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.528+0000 7f2440a00640 1 --2- 172.21.15.114:0/1017827328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f243c0b75b0 0x7f243c0d1e50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f243c0d2930 tx=0x7f242802a300 comp rx=0 tx=0).stop 2024-05-27T09:37:52.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.529+0000 7f2440a00640 1 -- 172.21.15.114:0/1017827328 shutdown_connections 2024-05-27T09:37:52.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.529+0000 7f2440a00640 1 --2- 172.21.15.114:0/1017827328 >> v2:172.21.15.114:3300/0 conn(0x7f243c0b46e0 0x7f243c0d2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:52.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.529+0000 7f2440a00640 1 --2- 172.21.15.114:0/1017827328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f243c0b75b0 0x7f243c0d1e50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:52.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.529+0000 7f2440a00640 1 -- 172.21.15.114:0/1017827328 >> 172.21.15.114:0/1017827328 conn(0x7f243c06e040 msgr2=0x7f243c06e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:52.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.529+0000 7f2440a00640 1 -- 172.21.15.114:0/1017827328 shutdown_connections 2024-05-27T09:37:52.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.529+0000 7f2440a00640 1 -- 172.21.15.114:0/1017827328 wait complete. 2024-05-27T09:37:52.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.531+0000 7f2440a00640 1 Processor -- start 2024-05-27T09:37:52.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.531+0000 7f2440a00640 1 -- start start 2024-05-27T09:37:52.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.533+0000 7f2440a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f243c0b46e0 0x7f243c14bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:52.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.533+0000 7f2440a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f243c0b75b0 0x7f243c1496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:52.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.533+0000 7f2440a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f243c0c8780 0x7f243c1f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:52.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.533+0000 7f2440a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f243c0ce0b0 con 0x7f243c0b75b0 2024-05-27T09:37:52.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.533+0000 7f2440a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f243c0cdf30 con 0x7f243c0c8780 2024-05-27T09:37:52.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.533+0000 7f2440a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f243c156c40 con 0x7f243c0b46e0 2024-05-27T09:37:52.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.533+0000 7f243be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f243c0c8780 0x7f243c1f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:52.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.533+0000 7f243b400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f243c0b46e0 0x7f243c14bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:52.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.533+0000 7f243be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f243c0c8780 0x7f243c1f5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:43098/0 (socket says 172.21.15.114:43098) 2024-05-27T09:37:52.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.533+0000 7f243be00640 1 -- 172.21.15.114:0/3390270737 learned_addr learned my addr 172.21.15.114:0/3390270737 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:37:52.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.533+0000 7f243aa00640 1 --2- 172.21.15.114:0/3390270737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f243c0b75b0 0x7f243c1496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:52.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.534+0000 7f243be00640 1 -- 172.21.15.114:0/3390270737 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f243c0b46e0 msgr2=0x7f243c14bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:52.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.534+0000 7f243be00640 1 --2- 172.21.15.114:0/3390270737 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f243c0b46e0 0x7f243c14bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:52.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.534+0000 7f243be00640 1 -- 172.21.15.114:0/3390270737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f243c0b75b0 msgr2=0x7f243c1496b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:52.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.534+0000 7f243be00640 1 --2- 172.21.15.114:0/3390270737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f243c0b75b0 0x7f243c1496b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:52.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.534+0000 7f243be00640 1 -- 172.21.15.114:0/3390270737 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243c14c2b0 con 0x7f243c0c8780 2024-05-27T09:37:52.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.534+0000 7f243be00640 1 --2- 172.21.15.114:0/3390270737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f243c0c8780 0x7f243c1f5ff0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2434007d40 tx=0x7f243400e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:52.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.534+0000 7f2433400640 1 -- 172.21.15.114:0/3390270737 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2434014420 con 0x7f243c0c8780 2024-05-27T09:37:52.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.534+0000 7f2433400640 1 -- 172.21.15.114:0/3390270737 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2434019070 con 0x7f243c0c8780 2024-05-27T09:37:52.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.534+0000 7f2433400640 1 -- 172.21.15.114:0/3390270737 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2434014820 con 0x7f243c0c8780 2024-05-27T09:37:52.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.534+0000 7f2440a00640 1 -- 172.21.15.114:0/3390270737 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f243c1f6ed0 con 0x7f243c0c8780 2024-05-27T09:37:52.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.535+0000 7f2440a00640 1 -- 172.21.15.114:0/3390270737 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f243c14b290 con 0x7f243c0c8780 2024-05-27T09:37:52.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.536+0000 7f2440a00640 1 -- 172.21.15.114:0/3390270737 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f243c050860 con 0x7f243c0c8780 2024-05-27T09:37:52.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.537+0000 7f2433400640 1 -- 172.21.15.114:0/3390270737 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f24340149c0 con 0x7f243c0c8780 2024-05-27T09:37:52.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.538+0000 7f2433400640 1 --2- 172.21.15.114:0/3390270737 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f241c0762d0 0x7f241c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:52.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.538+0000 7f2433400640 1 -- 172.21.15.114:0/3390270737 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7f2434097ee0 con 0x7f243c0c8780 2024-05-27T09:37:52.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.538+0000 7f243b400640 1 --2- 172.21.15.114:0/3390270737 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f241c0762d0 0x7f241c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:52.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.539+0000 7f243b400640 1 --2- 172.21.15.114:0/3390270737 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f241c0762d0 0x7f241c0787e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f243c08f8b0 tx=0x7f242c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:52.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.543+0000 7f2433400640 1 -- 172.21.15.114:0/3390270737 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2434060fc0 con 0x7f243c0c8780 2024-05-27T09:37:52.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.809+0000 7f2440a00640 1 -- 172.21.15.114:0/3390270737 --> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f243c0cdd10 con 0x7f241c0762d0 2024-05-27T09:37:52.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.816+0000 7f2433400640 1 -- 172.21.15.114:0/3390270737 <== mgr.4108 v2:172.21.15.163:6832/1122018204 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+255881 (secure 0 0 0) 0x7f243c0cdd10 con 0x7f241c0762d0 2024-05-27T09:37:52.820 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:37:52.823 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-05-27T09:37:52.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.823+0000 7f2440a00640 1 -- 172.21.15.114:0/3390270737 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f241c0762d0 msgr2=0x7f241c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:52.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.823+0000 7f2440a00640 1 --2- 172.21.15.114:0/3390270737 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f241c0762d0 0x7f241c0787e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f243c08f8b0 tx=0x7f242c009040 comp rx=0 tx=0).stop 2024-05-27T09:37:52.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.823+0000 7f2440a00640 1 -- 172.21.15.114:0/3390270737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f243c0c8780 msgr2=0x7f243c1f5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:52.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.823+0000 7f2440a00640 1 --2- 172.21.15.114:0/3390270737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f243c0c8780 0x7f243c1f5ff0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2434007d40 tx=0x7f243400e4b0 comp rx=0 tx=0).stop 2024-05-27T09:37:52.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.823+0000 7f2440a00640 1 -- 172.21.15.114:0/3390270737 shutdown_connections 2024-05-27T09:37:52.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.823+0000 7f2440a00640 1 --2- 172.21.15.114:0/3390270737 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f241c0762d0 0x7f241c0787e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:52.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.823+0000 7f2440a00640 1 --2- 172.21.15.114:0/3390270737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f243c0c8780 0x7f243c1f5ff0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:52.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.823+0000 7f2440a00640 1 --2- 172.21.15.114:0/3390270737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f243c0b75b0 0x7f243c1496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:52.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.823+0000 7f2440a00640 1 --2- 172.21.15.114:0/3390270737 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f243c0b46e0 0x7f243c14bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:52.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.823+0000 7f2440a00640 1 -- 172.21.15.114:0/3390270737 >> 172.21.15.114:0/3390270737 conn(0x7f243c06e040 msgr2=0x7f243c0764d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:52.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.823+0000 7f2440a00640 1 -- 172.21.15.114:0/3390270737 shutdown_connections 2024-05-27T09:37:52.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:37:52.824+0000 7f2440a00640 1 -- 172.21.15.114:0/3390270737 wait complete. 2024-05-27T09:37:52.850 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-05-27T09:37:50.943245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1141680653,"num_objects":10877,"num_object_clones":0,"num_object_copies":21754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10877,"num_whiteouts":0,"num_read":1873,"num_read_kb":502425,"num_write":25980,"num_write_kb":1570068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":105,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19729,"num_omap_keys":33,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26185,"ondisk_log_size":26185,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2200320,"kb_used_data":1780160,"kb_used_omap":12,"kb_used_meta":420083,"kb_avail":752774400,"statfs":{"total":773094113280,"available":770840985600,"internally_reserved":0,"allocated":1822883840,"data_stored":2283888386,"data_compressed":218552946,"data_compressed_allocated":249012224,"data_compressed_original":768172032,"omap_allocated":12750,"internal_metadata":430165554},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":201438,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":67,"num_write":34,"num_write_kb":209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19729,"num_omap_keys":33,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003541"},"pg_stats":[{"pgid":"3.3f","version":"50'328","reported_seq":2450,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:45.815550+0000","last_change":"2024-05-27T09:33:11.447418+0000","last_active":"2024-05-27T09:37:45.815550+0000","last_peered":"2024-05-27T09:37:45.815550+0000","last_clean":"2024-05-27T09:37:45.815550+0000","last_became_active":"2024-05-27T09:27:18.585169+0000","last_became_peered":"2024-05-27T09:27:18.585169+0000","last_unstale":"2024-05-27T09:37:45.815550+0000","last_undegraded":"2024-05-27T09:37:45.815550+0000","last_fullsized":"2024-05-27T09:37:45.815550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01102325,"stat_sum":{"num_bytes":23698106,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":17,"num_read_kb":4111,"num_write":320,"num_write_kb":27403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"50'317","reported_seq":2437,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.352291+0000","last_change":"2024-05-27T09:33:11.731804+0000","last_active":"2024-05-27T09:37:37.352291+0000","last_peered":"2024-05-27T09:37:37.352291+0000","last_clean":"2024-05-27T09:37:37.352291+0000","last_became_active":"2024-05-27T09:27:18.566269+0000","last_became_peered":"2024-05-27T09:27:18.566269+0000","last_unstale":"2024-05-27T09:37:37.352291+0000","last_undegraded":"2024-05-27T09:37:37.352291+0000","last_fullsized":"2024-05-27T09:37:37.352291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:30:51.472078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061074720000000001,"stat_sum":{"num_bytes":5460707,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":313,"num_write_kb":13693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"50'349","reported_seq":2477,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.350070+0000","last_change":"2024-05-27T09:33:11.731326+0000","last_active":"2024-05-27T09:37:37.350070+0000","last_peered":"2024-05-27T09:37:37.350070+0000","last_clean":"2024-05-27T09:37:37.350070+0000","last_became_active":"2024-05-27T09:27:18.560620+0000","last_became_peered":"2024-05-27T09:27:18.560620+0000","last_unstale":"2024-05-27T09:37:37.350070+0000","last_undegraded":"2024-05-27T09:37:37.350070+0000","last_fullsized":"2024-05-27T09:37:37.350070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:02:04.553273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005599354,"stat_sum":{"num_bytes":19266359,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":24,"num_read_kb":17715,"num_write":342,"num_write_kb":39437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"50'307","reported_seq":2421,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.378615+0000","last_change":"2024-05-27T09:33:11.450825+0000","last_active":"2024-05-27T09:37:37.378615+0000","last_peered":"2024-05-27T09:37:37.378615+0000","last_clean":"2024-05-27T09:37:37.378615+0000","last_became_active":"2024-05-27T09:27:18.569481+0000","last_became_peered":"2024-05-27T09:27:18.569481+0000","last_unstale":"2024-05-27T09:37:37.378615+0000","last_undegraded":"2024-05-27T09:37:37.378615+0000","last_fullsized":"2024-05-27T09:37:37.378615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:19:26.159843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007726882,"stat_sum":{"num_bytes":6546535,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":298,"num_write_kb":6559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"50'375","reported_seq":2497,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.378806+0000","last_change":"2024-05-27T09:33:11.440268+0000","last_active":"2024-05-27T09:37:37.378806+0000","last_peered":"2024-05-27T09:37:37.378806+0000","last_clean":"2024-05-27T09:37:37.378806+0000","last_became_active":"2024-05-27T09:27:18.567795+0000","last_became_peered":"2024-05-27T09:27:18.567795+0000","last_unstale":"2024-05-27T09:37:37.378806+0000","last_undegraded":"2024-05-27T09:37:37.378806+0000","last_fullsized":"2024-05-27T09:37:37.378806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049759740000000002,"stat_sum":{"num_bytes":10629514,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":371,"num_write_kb":12093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"50'350","reported_seq":2475,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.371402+0000","last_change":"2024-05-27T09:33:11.445483+0000","last_active":"2024-05-27T09:37:37.371402+0000","last_peered":"2024-05-27T09:37:37.371402+0000","last_clean":"2024-05-27T09:37:37.371402+0000","last_became_active":"2024-05-27T09:27:18.573576+0000","last_became_peered":"2024-05-27T09:27:18.573576+0000","last_unstale":"2024-05-27T09:37:37.371402+0000","last_undegraded":"2024-05-27T09:37:37.371402+0000","last_fullsized":"2024-05-27T09:37:37.371402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:45:04.738799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0074740400000000004,"stat_sum":{"num_bytes":30579450,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":22,"num_read_kb":8211,"num_write":344,"num_write_kb":38238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"50'360","reported_seq":2482,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.370792+0000","last_change":"2024-05-27T09:33:11.448897+0000","last_active":"2024-05-27T09:37:37.370792+0000","last_peered":"2024-05-27T09:37:37.370792+0000","last_clean":"2024-05-27T09:37:37.370792+0000","last_became_active":"2024-05-27T09:27:18.542593+0000","last_became_peered":"2024-05-27T09:27:18.542593+0000","last_unstale":"2024-05-27T09:37:37.370792+0000","last_undegraded":"2024-05-27T09:37:37.370792+0000","last_fullsized":"2024-05-27T09:37:37.370792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014070156,"stat_sum":{"num_bytes":5299988,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":18,"num_read_kb":4113,"num_write":355,"num_write_kb":9461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"50'329","reported_seq":2445,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.384663+0000","last_change":"2024-05-27T09:33:11.732725+0000","last_active":"2024-05-27T09:37:37.384663+0000","last_peered":"2024-05-27T09:37:37.384663+0000","last_clean":"2024-05-27T09:37:37.384663+0000","last_became_active":"2024-05-27T09:27:18.561788+0000","last_became_peered":"2024-05-27T09:27:18.561788+0000","last_unstale":"2024-05-27T09:37:37.384663+0000","last_undegraded":"2024-05-27T09:37:37.384663+0000","last_fullsized":"2024-05-27T09:37:37.384663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:51:29.653992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065132080000000004,"stat_sum":{"num_bytes":27492830,"num_objects":191,"num_object_clones":0,"num_object_copies":382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":191,"num_whiteouts":0,"num_read":13,"num_read_kb":4107,"num_write":319,"num_write_kb":27008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"50'338","reported_seq":2464,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.369818+0000","last_change":"2024-05-27T09:33:11.739498+0000","last_active":"2024-05-27T09:37:37.369818+0000","last_peered":"2024-05-27T09:37:37.369818+0000","last_clean":"2024-05-27T09:37:37.369818+0000","last_became_active":"2024-05-27T09:27:18.582041+0000","last_became_peered":"2024-05-27T09:27:18.582041+0000","last_unstale":"2024-05-27T09:37:37.369818+0000","last_undegraded":"2024-05-27T09:37:37.369818+0000","last_fullsized":"2024-05-27T09:37:37.369818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0083382379999999996,"stat_sum":{"num_bytes":9733898,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":22,"num_read_kb":8210,"num_write":328,"num_write_kb":30110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"50'321","reported_seq":2442,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.392330+0000","last_change":"2024-05-27T09:33:11.446829+0000","last_active":"2024-05-27T09:37:37.392330+0000","last_peered":"2024-05-27T09:37:37.392330+0000","last_clean":"2024-05-27T09:37:37.392330+0000","last_became_active":"2024-05-27T09:27:18.574409+0000","last_became_peered":"2024-05-27T09:27:18.574409+0000","last_unstale":"2024-05-27T09:37:37.392330+0000","last_undegraded":"2024-05-27T09:37:37.392330+0000","last_fullsized":"2024-05-27T09:37:37.392330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010493662000000001,"stat_sum":{"num_bytes":13833965,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":17,"num_read_kb":4112,"num_write":306,"num_write_kb":13622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"50'319","reported_seq":2445,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.384860+0000","last_change":"2024-05-27T09:33:11.443445+0000","last_active":"2024-05-27T09:37:37.384860+0000","last_peered":"2024-05-27T09:37:37.384860+0000","last_clean":"2024-05-27T09:37:37.384860+0000","last_became_active":"2024-05-27T09:27:18.578679+0000","last_became_peered":"2024-05-27T09:27:18.578679+0000","last_unstale":"2024-05-27T09:37:37.384860+0000","last_undegraded":"2024-05-27T09:37:37.384860+0000","last_fullsized":"2024-05-27T09:37:37.384860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080365679999999991,"stat_sum":{"num_bytes":26002241,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":22,"num_read_kb":7043,"num_write":312,"num_write_kb":25555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"50'301","reported_seq":2410,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.358428+0000","last_change":"2024-05-27T09:33:11.625670+0000","last_active":"2024-05-27T09:37:37.358428+0000","last_peered":"2024-05-27T09:37:37.358428+0000","last_clean":"2024-05-27T09:37:37.358428+0000","last_became_active":"2024-05-27T09:28:06.207562+0000","last_became_peered":"2024-05-27T09:28:06.207562+0000","last_unstale":"2024-05-27T09:37:37.358428+0000","last_undegraded":"2024-05-27T09:37:37.358428+0000","last_fullsized":"2024-05-27T09:37:37.358428+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:52:32.773329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001018333,"stat_sum":{"num_bytes":11019660,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":12,"num_read_kb":8201,"num_write":293,"num_write_kb":23198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"50'308","reported_seq":2432,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.385407+0000","last_change":"2024-05-27T09:33:11.728591+0000","last_active":"2024-05-27T09:37:37.385407+0000","last_peered":"2024-05-27T09:37:37.385407+0000","last_clean":"2024-05-27T09:37:37.385407+0000","last_became_active":"2024-05-27T09:27:18.561493+0000","last_became_peered":"2024-05-27T09:27:18.561493+0000","last_unstale":"2024-05-27T09:37:37.385407+0000","last_undegraded":"2024-05-27T09:37:37.385407+0000","last_fullsized":"2024-05-27T09:37:37.385407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:22.877083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024650470000000002,"stat_sum":{"num_bytes":14891482,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":21,"num_read_kb":4116,"num_write":298,"num_write_kb":31046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"50'342","reported_seq":2463,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.356879+0000","last_change":"2024-05-27T09:33:11.727658+0000","last_active":"2024-05-27T09:37:37.356879+0000","last_peered":"2024-05-27T09:37:37.356879+0000","last_clean":"2024-05-27T09:37:37.356879+0000","last_became_active":"2024-05-27T09:27:18.562843+0000","last_became_peered":"2024-05-27T09:27:18.562843+0000","last_unstale":"2024-05-27T09:37:37.356879+0000","last_undegraded":"2024-05-27T09:37:37.356879+0000","last_fullsized":"2024-05-27T09:37:37.356879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:31.147919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025802960000000002,"stat_sum":{"num_bytes":6434369,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":335,"num_write_kb":10518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"50'364","reported_seq":2489,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.384329+0000","last_change":"2024-05-27T09:33:11.729192+0000","last_active":"2024-05-27T09:37:37.384329+0000","last_peered":"2024-05-27T09:37:37.384329+0000","last_clean":"2024-05-27T09:37:37.384329+0000","last_became_active":"2024-05-27T09:27:18.566108+0000","last_became_peered":"2024-05-27T09:27:18.566108+0000","last_unstale":"2024-05-27T09:37:37.384329+0000","last_undegraded":"2024-05-27T09:37:37.384329+0000","last_fullsized":"2024-05-27T09:37:37.384329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:39:23.102944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041189030000000001,"stat_sum":{"num_bytes":20283045,"num_objects":201,"num_object_clones":0,"num_object_copies":402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":201,"num_whiteouts":0,"num_read":22,"num_read_kb":6627,"num_write":357,"num_write_kb":19995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"50'369","reported_seq":2493,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.386640+0000","last_change":"2024-05-27T09:33:11.457079+0000","last_active":"2024-05-27T09:37:37.386640+0000","last_peered":"2024-05-27T09:37:37.386640+0000","last_clean":"2024-05-27T09:37:37.386640+0000","last_became_active":"2024-05-27T09:27:18.581719+0000","last_became_peered":"2024-05-27T09:27:18.581719+0000","last_unstale":"2024-05-27T09:37:37.386640+0000","last_undegraded":"2024-05-27T09:37:37.386640+0000","last_fullsized":"2024-05-27T09:37:37.386640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:59:21.546098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.008579415,"stat_sum":{"num_bytes":24362280,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":22,"num_read_kb":5287,"num_write":359,"num_write_kb":31349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"50'366","reported_seq":2491,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.387571+0000","last_change":"2024-05-27T09:33:11.458057+0000","last_active":"2024-05-27T09:37:37.387571+0000","last_peered":"2024-05-27T09:37:37.387571+0000","last_clean":"2024-05-27T09:37:37.387571+0000","last_became_active":"2024-05-27T09:27:18.577193+0000","last_became_peered":"2024-05-27T09:27:18.577193+0000","last_unstale":"2024-05-27T09:37:37.387571+0000","last_undegraded":"2024-05-27T09:37:37.387571+0000","last_fullsized":"2024-05-27T09:37:37.387571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:41:12.903002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079143630000000006,"stat_sum":{"num_bytes":23132412,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":23,"num_read_kb":8093,"num_write":357,"num_write_kb":30967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"50'307","reported_seq":2438,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.372050+0000","last_change":"2024-05-27T09:33:11.626051+0000","last_active":"2024-05-27T09:37:37.372050+0000","last_peered":"2024-05-27T09:37:37.372050+0000","last_clean":"2024-05-27T09:37:37.372050+0000","last_became_active":"2024-05-27T09:27:18.570712+0000","last_became_peered":"2024-05-27T09:27:18.570712+0000","last_unstale":"2024-05-27T09:37:37.372050+0000","last_undegraded":"2024-05-27T09:37:37.372050+0000","last_fullsized":"2024-05-27T09:37:37.372050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:32:58.266174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011767629999999999,"stat_sum":{"num_bytes":13973992,"num_objects":138,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":0,"num_read":28,"num_read_kb":8218,"num_write":296,"num_write_kb":21985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"50'323","reported_seq":2443,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.367564+0000","last_change":"2024-05-27T09:33:11.456909+0000","last_active":"2024-05-27T09:37:37.367564+0000","last_peered":"2024-05-27T09:37:37.367564+0000","last_clean":"2024-05-27T09:37:37.367564+0000","last_became_active":"2024-05-27T09:27:18.564643+0000","last_became_peered":"2024-05-27T09:27:18.564643+0000","last_unstale":"2024-05-27T09:37:37.367564+0000","last_undegraded":"2024-05-27T09:37:37.367564+0000","last_fullsized":"2024-05-27T09:37:37.367564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:14:07.864139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0091687629999999999,"stat_sum":{"num_bytes":2873482,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":18,"num_read_kb":4113,"num_write":315,"num_write_kb":11167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"50'345","reported_seq":2470,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.376533+0000","last_change":"2024-05-27T09:33:11.452205+0000","last_active":"2024-05-27T09:37:37.376533+0000","last_peered":"2024-05-27T09:37:37.376533+0000","last_clean":"2024-05-27T09:37:37.376533+0000","last_became_active":"2024-05-27T09:27:18.543777+0000","last_became_peered":"2024-05-27T09:27:18.543777+0000","last_unstale":"2024-05-27T09:37:37.376533+0000","last_undegraded":"2024-05-27T09:37:37.376533+0000","last_fullsized":"2024-05-27T09:37:37.376533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012534072,"stat_sum":{"num_bytes":9859903,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":21,"num_read_kb":4116,"num_write":337,"num_write_kb":17999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"50'337","reported_seq":2462,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.367546+0000","last_change":"2024-05-27T09:33:11.732683+0000","last_active":"2024-05-27T09:37:37.367546+0000","last_peered":"2024-05-27T09:37:37.367546+0000","last_clean":"2024-05-27T09:37:37.367546+0000","last_became_active":"2024-05-27T09:27:18.570948+0000","last_became_peered":"2024-05-27T09:27:18.570948+0000","last_unstale":"2024-05-27T09:37:37.367546+0000","last_undegraded":"2024-05-27T09:37:37.367546+0000","last_fullsized":"2024-05-27T09:37:37.367546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:03:03.895209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040840829999999996,"stat_sum":{"num_bytes":5923849,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":325,"num_write_kb":5958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"50'343","reported_seq":2467,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.383348+0000","last_change":"2024-05-27T09:33:11.737682+0000","last_active":"2024-05-27T09:37:37.383348+0000","last_peered":"2024-05-27T09:37:37.383348+0000","last_clean":"2024-05-27T09:37:37.383348+0000","last_became_active":"2024-05-27T09:27:18.543921+0000","last_became_peered":"2024-05-27T09:27:18.543921+0000","last_unstale":"2024-05-27T09:37:37.383348+0000","last_undegraded":"2024-05-27T09:37:37.383348+0000","last_fullsized":"2024-05-27T09:37:37.383348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0094259180000000001,"stat_sum":{"num_bytes":14450069,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":20,"num_read_kb":8209,"num_write":332,"num_write_kb":18385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"50'337","reported_seq":2471,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.387356+0000","last_change":"2024-05-27T09:33:11.731031+0000","last_active":"2024-05-27T09:37:37.387356+0000","last_peered":"2024-05-27T09:37:37.387356+0000","last_clean":"2024-05-27T09:37:37.387356+0000","last_became_active":"2024-05-27T09:27:18.575115+0000","last_became_peered":"2024-05-27T09:27:18.575115+0000","last_unstale":"2024-05-27T09:37:37.387356+0000","last_undegraded":"2024-05-27T09:37:37.387356+0000","last_fullsized":"2024-05-27T09:37:37.387356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:56:57.284871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054063330000000001,"stat_sum":{"num_bytes":18656408,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":31,"num_read_kb":19450,"num_write":329,"num_write_kb":38861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"50'353","reported_seq":2474,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.367488+0000","last_change":"2024-05-27T09:33:11.451714+0000","last_active":"2024-05-27T09:37:37.367488+0000","last_peered":"2024-05-27T09:37:37.367488+0000","last_clean":"2024-05-27T09:37:37.367488+0000","last_became_active":"2024-05-27T09:27:18.563991+0000","last_became_peered":"2024-05-27T09:27:18.563991+0000","last_unstale":"2024-05-27T09:37:37.367488+0000","last_undegraded":"2024-05-27T09:37:37.367488+0000","last_fullsized":"2024-05-27T09:37:37.367488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:41:37.963962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009841753,"stat_sum":{"num_bytes":13963475,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":19,"num_read_kb":8208,"num_write":347,"num_write_kb":22013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"50'332","reported_seq":2456,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.371538+0000","last_change":"2024-05-27T09:33:11.732272+0000","last_active":"2024-05-27T09:37:37.371538+0000","last_peered":"2024-05-27T09:37:37.371538+0000","last_clean":"2024-05-27T09:37:37.371538+0000","last_became_active":"2024-05-27T09:27:18.562803+0000","last_became_peered":"2024-05-27T09:27:18.562803+0000","last_unstale":"2024-05-27T09:37:37.371538+0000","last_undegraded":"2024-05-27T09:37:37.371538+0000","last_fullsized":"2024-05-27T09:37:37.371538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:09:56.174365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067443290000000003,"stat_sum":{"num_bytes":23071782,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":21,"num_read_kb":4115,"num_write":328,"num_write_kb":30886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"50'384","reported_seq":2507,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.393993+0000","last_change":"2024-05-27T09:33:11.453888+0000","last_active":"2024-05-27T09:37:37.393993+0000","last_peered":"2024-05-27T09:37:37.393993+0000","last_clean":"2024-05-27T09:37:37.393993+0000","last_became_active":"2024-05-27T09:27:18.590083+0000","last_became_peered":"2024-05-27T09:27:18.590083+0000","last_unstale":"2024-05-27T09:37:37.393993+0000","last_undegraded":"2024-05-27T09:37:37.393993+0000","last_fullsized":"2024-05-27T09:37:37.393993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:00.006136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.008181008,"stat_sum":{"num_bytes":14547405,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":20,"num_read_kb":4115,"num_write":369,"num_write_kb":14397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"50'332","reported_seq":2451,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.354888+0000","last_change":"2024-05-27T09:33:11.728099+0000","last_active":"2024-05-27T09:37:37.354888+0000","last_peered":"2024-05-27T09:37:37.354888+0000","last_clean":"2024-05-27T09:37:37.354888+0000","last_became_active":"2024-05-27T09:27:18.557953+0000","last_became_peered":"2024-05-27T09:27:18.557953+0000","last_unstale":"2024-05-27T09:37:37.354888+0000","last_undegraded":"2024-05-27T09:37:37.354888+0000","last_fullsized":"2024-05-27T09:37:37.354888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:03:03.895209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027117650000000001,"stat_sum":{"num_bytes":18627167,"num_objects":187,"num_object_clones":0,"num_object_copies":374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":187,"num_whiteouts":0,"num_read":16,"num_read_kb":4110,"num_write":325,"num_write_kb":26562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"50'307","reported_seq":2427,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.379634+0000","last_change":"2024-05-27T09:33:11.729430+0000","last_active":"2024-05-27T09:37:37.379634+0000","last_peered":"2024-05-27T09:37:37.379634+0000","last_clean":"2024-05-27T09:37:37.379634+0000","last_became_active":"2024-05-27T09:27:18.562839+0000","last_became_peered":"2024-05-27T09:27:18.562839+0000","last_unstale":"2024-05-27T09:37:37.379634+0000","last_undegraded":"2024-05-27T09:37:37.379634+0000","last_fullsized":"2024-05-27T09:37:37.379634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:21:17.734561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032647399999999999,"stat_sum":{"num_bytes":14533754,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":17,"num_read_kb":11479,"num_write":303,"num_write_kb":21724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"50'351","reported_seq":2475,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.344235+0000","last_change":"2024-05-27T09:33:11.730679+0000","last_active":"2024-05-27T09:37:37.344235+0000","last_peered":"2024-05-27T09:37:37.344235+0000","last_clean":"2024-05-27T09:37:37.344235+0000","last_became_active":"2024-05-27T09:27:18.561475+0000","last_became_peered":"2024-05-27T09:27:18.561475+0000","last_unstale":"2024-05-27T09:37:37.344235+0000","last_undegraded":"2024-05-27T09:37:37.344235+0000","last_fullsized":"2024-05-27T09:37:37.344235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:32:06.826783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053708779999999999,"stat_sum":{"num_bytes":11382190,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":20,"num_read_kb":4114,"num_write":347,"num_write_kb":19366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"50'377","reported_seq":2508,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.362385+0000","last_change":"2024-05-27T09:33:11.454566+0000","last_active":"2024-05-27T09:37:37.362385+0000","last_peered":"2024-05-27T09:37:37.362385+0000","last_clean":"2024-05-27T09:37:37.362385+0000","last_became_active":"2024-05-27T09:27:18.563103+0000","last_became_peered":"2024-05-27T09:27:18.563103+0000","last_unstale":"2024-05-27T09:37:37.362385+0000","last_undegraded":"2024-05-27T09:37:37.362385+0000","last_fullsized":"2024-05-27T09:37:37.362385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:25.776404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011066041,"stat_sum":{"num_bytes":30187091,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":28,"num_read_kb":16405,"num_write":369,"num_write_kb":50152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"50'401","reported_seq":2524,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.358562+0000","last_change":"2024-05-27T09:33:11.625969+0000","last_active":"2024-05-27T09:37:37.358562+0000","last_peered":"2024-05-27T09:37:37.358562+0000","last_clean":"2024-05-27T09:37:37.358562+0000","last_became_active":"2024-05-27T09:27:18.560244+0000","last_became_peered":"2024-05-27T09:27:18.560244+0000","last_unstale":"2024-05-27T09:37:37.358562+0000","last_undegraded":"2024-05-27T09:37:37.358562+0000","last_fullsized":"2024-05-27T09:37:37.358562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:14:56.979490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012131959999999999,"stat_sum":{"num_bytes":8541274,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":19,"num_read_kb":2631,"num_write":389,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"50'363","reported_seq":2494,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.377471+0000","last_change":"2024-05-27T09:33:11.447687+0000","last_active":"2024-05-27T09:37:37.377471+0000","last_peered":"2024-05-27T09:37:37.377471+0000","last_clean":"2024-05-27T09:37:37.377471+0000","last_became_active":"2024-05-27T09:27:18.567845+0000","last_became_peered":"2024-05-27T09:27:18.567845+0000","last_unstale":"2024-05-27T09:37:37.377471+0000","last_undegraded":"2024-05-27T09:37:37.377471+0000","last_fullsized":"2024-05-27T09:37:37.377471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:48:36.494196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082936180000000009,"stat_sum":{"num_bytes":1444930,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":352,"num_write_kb":9783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"50'336","reported_seq":2459,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.371552+0000","last_change":"2024-05-27T09:33:11.738447+0000","last_active":"2024-05-27T09:37:37.371552+0000","last_peered":"2024-05-27T09:37:37.371552+0000","last_clean":"2024-05-27T09:37:37.371552+0000","last_became_active":"2024-05-27T09:27:18.556156+0000","last_became_peered":"2024-05-27T09:27:18.556156+0000","last_unstale":"2024-05-27T09:37:37.371552+0000","last_undegraded":"2024-05-27T09:37:37.371552+0000","last_fullsized":"2024-05-27T09:37:37.371552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0094378269999999993,"stat_sum":{"num_bytes":19241078,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":19,"num_read_kb":8208,"num_write":329,"num_write_kb":23062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"50'74","reported_seq":157,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:36.346406+0000","last_change":"2024-05-27T09:27:17.548038+0000","last_active":"2024-05-27T09:37:36.346406+0000","last_peered":"2024-05-27T09:37:36.346406+0000","last_clean":"2024-05-27T09:37:36.346406+0000","last_became_active":"2024-05-27T09:27:17.547653+0000","last_became_peered":"2024-05-27T09:27:17.547653+0000","last_unstale":"2024-05-27T09:37:36.346406+0000","last_undegraded":"2024-05-27T09:37:36.346406+0000","last_fullsized":"2024-05-27T09:37:36.346406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195553,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":4100,"num_write":72,"num_write_kb":4221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"50'328","reported_seq":2449,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.384098+0000","last_change":"2024-05-27T09:33:11.733348+0000","last_active":"2024-05-27T09:37:37.384098+0000","last_peered":"2024-05-27T09:37:37.384098+0000","last_clean":"2024-05-27T09:37:37.384098+0000","last_became_active":"2024-05-27T09:27:18.542715+0000","last_became_peered":"2024-05-27T09:27:18.542715+0000","last_unstale":"2024-05-27T09:37:37.384098+0000","last_undegraded":"2024-05-27T09:37:37.384098+0000","last_fullsized":"2024-05-27T09:37:37.384098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:45:47.000935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004594323,"stat_sum":{"num_bytes":10080804,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":19,"num_read_kb":4113,"num_write":319,"num_write_kb":14102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"49'299","reported_seq":384,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.547628+0000","last_change":"2024-05-27T09:27:17.570014+0000","last_active":"2024-05-27T09:37:15.547628+0000","last_peered":"2024-05-27T09:37:15.547628+0000","last_clean":"2024-05-27T09:37:15.547628+0000","last_became_active":"2024-05-27T09:27:17.569673+0000","last_became_peered":"2024-05-27T09:27:17.569673+0000","last_unstale":"2024-05-27T09:37:15.547628+0000","last_undegraded":"2024-05-27T09:37:15.547628+0000","last_fullsized":"2024-05-27T09:37:15.547628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:57:26.210041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":22,"num_read_kb":12200,"num_write":300,"num_write_kb":12509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"49'46","reported_seq":129,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.952341+0000","last_change":"2024-05-27T09:27:17.565489+0000","last_active":"2024-05-27T09:37:15.952341+0000","last_peered":"2024-05-27T09:37:15.952341+0000","last_clean":"2024-05-27T09:37:15.952341+0000","last_became_active":"2024-05-27T09:27:17.565137+0000","last_became_peered":"2024-05-27T09:27:17.565137+0000","last_unstale":"2024-05-27T09:37:15.952341+0000","last_undegraded":"2024-05-27T09:37:15.952341+0000","last_fullsized":"2024-05-27T09:37:15.952341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":19,"num_read_kb":4114,"num_write":50,"num_write_kb":4167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"50'397","reported_seq":2523,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.353248+0000","last_change":"2024-05-27T09:33:11.733228+0000","last_active":"2024-05-27T09:37:37.353248+0000","last_peered":"2024-05-27T09:37:37.353248+0000","last_clean":"2024-05-27T09:37:37.353248+0000","last_became_active":"2024-05-27T09:27:18.564115+0000","last_became_peered":"2024-05-27T09:27:18.564115+0000","last_unstale":"2024-05-27T09:37:37.353248+0000","last_undegraded":"2024-05-27T09:37:37.353248+0000","last_fullsized":"2024-05-27T09:37:37.353248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:06:44.772830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066511199999999999,"stat_sum":{"num_bytes":10503109,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":23,"num_read_kb":4118,"num_write":389,"num_write_kb":18659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"50'294","reported_seq":371,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.547032+0000","last_change":"2024-05-27T09:27:17.561204+0000","last_active":"2024-05-27T09:37:15.547032+0000","last_peered":"2024-05-27T09:37:15.547032+0000","last_clean":"2024-05-27T09:37:15.547032+0000","last_became_active":"2024-05-27T09:27:17.560149+0000","last_became_peered":"2024-05-27T09:27:17.560149+0000","last_unstale":"2024-05-27T09:37:15.547032+0000","last_undegraded":"2024-05-27T09:37:15.547032+0000","last_fullsized":"2024-05-27T09:37:15.547032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.284270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":291,"num_write_kb":12506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"50'344","reported_seq":2466,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.356787+0000","last_change":"2024-05-27T09:33:11.734046+0000","last_active":"2024-05-27T09:37:37.356787+0000","last_peered":"2024-05-27T09:37:37.356787+0000","last_clean":"2024-05-27T09:37:37.356787+0000","last_became_active":"2024-05-27T09:27:18.541946+0000","last_became_peered":"2024-05-27T09:27:18.541946+0000","last_unstale":"2024-05-27T09:37:37.356787+0000","last_undegraded":"2024-05-27T09:37:37.356787+0000","last_fullsized":"2024-05-27T09:37:37.356787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:30.087688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030229530000000001,"stat_sum":{"num_bytes":19296821,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":20,"num_read_kb":12305,"num_write":330,"num_write_kb":31302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"50'9","reported_seq":92,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:41.383842+0000","last_change":"2024-05-27T09:27:17.561714+0000","last_active":"2024-05-27T09:37:41.383842+0000","last_peered":"2024-05-27T09:37:41.383842+0000","last_clean":"2024-05-27T09:37:41.383842+0000","last_became_active":"2024-05-27T09:27:17.561355+0000","last_became_peered":"2024-05-27T09:27:17.561355+0000","last_unstale":"2024-05-27T09:37:41.383842+0000","last_undegraded":"2024-05-27T09:37:41.383842+0000","last_fullsized":"2024-05-27T09:37:41.383842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":8,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"50'354","reported_seq":2478,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.394166+0000","last_change":"2024-05-27T09:33:11.454088+0000","last_active":"2024-05-27T09:37:37.394166+0000","last_peered":"2024-05-27T09:37:37.394166+0000","last_clean":"2024-05-27T09:37:37.394166+0000","last_became_active":"2024-05-27T09:27:18.587921+0000","last_became_peered":"2024-05-27T09:27:18.587921+0000","last_unstale":"2024-05-27T09:37:37.394166+0000","last_undegraded":"2024-05-27T09:37:37.394166+0000","last_fullsized":"2024-05-27T09:37:37.394166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:17:20.248600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0078895319999999994,"stat_sum":{"num_bytes":10502055,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":21,"num_read_kb":4116,"num_write":345,"num_write_kb":10394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"50'70","reported_seq":160,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:46.660778+0000","last_change":"2024-05-27T09:37:46.660862+0000","last_active":"2024-05-27T09:37:46.660778+0000","last_peered":"2024-05-27T09:37:46.660778+0000","last_clean":"2024-05-27T09:37:46.660778+0000","last_became_active":"2024-05-27T09:27:17.565488+0000","last_became_peered":"2024-05-27T09:27:17.565488+0000","last_unstale":"2024-05-27T09:37:46.660778+0000","last_undegraded":"2024-05-27T09:37:46.660778+0000","last_fullsized":"2024-05-27T09:37:46.660778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'70","last_scrub_stamp":"2024-05-27T09:37:46.660608+0000","last_deep_scrub":"50'70","last_deep_scrub_stamp":"2024-05-27T09:37:46.660608+0000","last_clean_scrub_stamp":"2024-05-27T09:37:46.660608+0000","objects_scrubbed":5,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:07:50.518839+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":4108,"num_write":67,"num_write_kb":4232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19729,"num_omap_keys":33,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"50'338","reported_seq":2464,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.366109+0000","last_change":"2024-05-27T09:33:11.733328+0000","last_active":"2024-05-27T09:37:37.366109+0000","last_peered":"2024-05-27T09:37:37.366109+0000","last_clean":"2024-05-27T09:37:37.366109+0000","last_became_active":"2024-05-27T09:27:18.585243+0000","last_became_peered":"2024-05-27T09:27:18.585243+0000","last_unstale":"2024-05-27T09:37:37.366109+0000","last_undegraded":"2024-05-27T09:37:37.366109+0000","last_fullsized":"2024-05-27T09:37:37.366109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:18:44.814432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067133569999999997,"stat_sum":{"num_bytes":18063712,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":23,"num_read_kb":8211,"num_write":328,"num_write_kb":30099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"50'204","reported_seq":284,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.529879+0000","last_change":"2024-05-27T09:27:17.548651+0000","last_active":"2024-05-27T09:37:20.529879+0000","last_peered":"2024-05-27T09:37:20.529879+0000","last_clean":"2024-05-27T09:37:20.529879+0000","last_became_active":"2024-05-27T09:27:17.548307+0000","last_became_peered":"2024-05-27T09:27:17.548307+0000","last_unstale":"2024-05-27T09:37:20.529879+0000","last_undegraded":"2024-05-27T09:37:20.529879+0000","last_fullsized":"2024-05-27T09:37:20.529879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:48.905422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":206,"num_write_kb":4295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"50'341","reported_seq":2460,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.387494+0000","last_change":"2024-05-27T09:33:11.455764+0000","last_active":"2024-05-27T09:37:37.387494+0000","last_peered":"2024-05-27T09:37:37.387494+0000","last_clean":"2024-05-27T09:37:37.387494+0000","last_became_active":"2024-05-27T09:27:18.581742+0000","last_became_peered":"2024-05-27T09:27:18.581742+0000","last_unstale":"2024-05-27T09:37:37.387494+0000","last_undegraded":"2024-05-27T09:37:37.387494+0000","last_fullsized":"2024-05-27T09:37:37.387494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:14:45.868801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089613100000000001,"stat_sum":{"num_bytes":20607307,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":17,"num_read_kb":10914,"num_write":328,"num_write_kb":24390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"50'246","reported_seq":331,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:41.386911+0000","last_change":"2024-05-27T09:27:17.556419+0000","last_active":"2024-05-27T09:37:41.386911+0000","last_peered":"2024-05-27T09:37:41.386911+0000","last_clean":"2024-05-27T09:37:41.386911+0000","last_became_active":"2024-05-27T09:27:17.556036+0000","last_became_peered":"2024-05-27T09:27:17.556036+0000","last_unstale":"2024-05-27T09:37:41.386911+0000","last_undegraded":"2024-05-27T09:37:41.386911+0000","last_fullsized":"2024-05-27T09:37:41.386911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:48:20.718085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":19,"num_read_kb":8209,"num_write":244,"num_write_kb":12460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"50'335","reported_seq":2458,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.383123+0000","last_change":"2024-05-27T09:33:11.449451+0000","last_active":"2024-05-27T09:37:37.383123+0000","last_peered":"2024-05-27T09:37:37.383123+0000","last_clean":"2024-05-27T09:37:37.383123+0000","last_became_active":"2024-05-27T09:27:18.565500+0000","last_became_peered":"2024-05-27T09:27:18.565500+0000","last_unstale":"2024-05-27T09:37:37.383123+0000","last_undegraded":"2024-05-27T09:37:37.383123+0000","last_fullsized":"2024-05-27T09:37:37.383123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011618516000000001,"stat_sum":{"num_bytes":10741446,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":19,"num_read_kb":4114,"num_write":329,"num_write_kb":14761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"49'17","reported_seq":107,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:41.378694+0000","last_change":"2024-05-27T09:27:17.572168+0000","last_active":"2024-05-27T09:37:41.378694+0000","last_peered":"2024-05-27T09:37:41.378694+0000","last_clean":"2024-05-27T09:37:41.378694+0000","last_became_active":"2024-05-27T09:27:17.571709+0000","last_became_peered":"2024-05-27T09:27:17.571709+0000","last_unstale":"2024-05-27T09:37:41.378694+0000","last_undegraded":"2024-05-27T09:37:41.378694+0000","last_fullsized":"2024-05-27T09:37:41.378694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:44.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":8230,"num_write":18,"num_write_kb":8221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"50'343","reported_seq":2472,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.365363+0000","last_change":"2024-05-27T09:33:11.728047+0000","last_active":"2024-05-27T09:37:37.365363+0000","last_peered":"2024-05-27T09:37:37.365363+0000","last_clean":"2024-05-27T09:37:37.365363+0000","last_became_active":"2024-05-27T09:27:18.564589+0000","last_became_peered":"2024-05-27T09:27:18.564589+0000","last_unstale":"2024-05-27T09:37:37.365363+0000","last_undegraded":"2024-05-27T09:37:37.365363+0000","last_fullsized":"2024-05-27T09:37:37.365363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:09:11.832557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028467779999999999,"stat_sum":{"num_bytes":14347466,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":26,"num_read_kb":16403,"num_write":332,"num_write_kb":30566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"50'8","reported_seq":87,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:41.373266+0000","last_change":"2024-05-27T09:27:17.561344+0000","last_active":"2024-05-27T09:37:41.373266+0000","last_peered":"2024-05-27T09:37:41.373266+0000","last_clean":"2024-05-27T09:37:41.373266+0000","last_became_active":"2024-05-27T09:27:17.554860+0000","last_became_peered":"2024-05-27T09:27:17.554860+0000","last_unstale":"2024-05-27T09:37:41.373266+0000","last_undegraded":"2024-05-27T09:37:41.373266+0000","last_fullsized":"2024-05-27T09:37:41.373266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:16.880101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":9,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"50'359","reported_seq":2484,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.391039+0000","last_change":"2024-05-27T09:33:11.456274+0000","last_active":"2024-05-27T09:37:37.391039+0000","last_peered":"2024-05-27T09:37:37.391039+0000","last_clean":"2024-05-27T09:37:37.391039+0000","last_became_active":"2024-05-27T09:27:18.572024+0000","last_became_peered":"2024-05-27T09:27:18.572024+0000","last_unstale":"2024-05-27T09:37:37.391039+0000","last_undegraded":"2024-05-27T09:37:37.391039+0000","last_fullsized":"2024-05-27T09:37:37.391039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:40:00.595445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010292084,"stat_sum":{"num_bytes":12019831,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":346,"num_write_kb":11826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"50'12","reported_seq":98,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.371414+0000","last_change":"2024-05-27T09:27:17.575859+0000","last_active":"2024-05-27T09:37:37.371414+0000","last_peered":"2024-05-27T09:37:37.371414+0000","last_clean":"2024-05-27T09:37:37.371414+0000","last_became_active":"2024-05-27T09:27:17.574695+0000","last_became_peered":"2024-05-27T09:27:17.574695+0000","last_unstale":"2024-05-27T09:37:37.371414+0000","last_undegraded":"2024-05-27T09:37:37.371414+0000","last_fullsized":"2024-05-27T09:37:37.371414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1331,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":9,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"50'340","reported_seq":2463,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.354483+0000","last_change":"2024-05-27T09:33:11.736362+0000","last_active":"2024-05-27T09:37:37.354483+0000","last_peered":"2024-05-27T09:37:37.354483+0000","last_clean":"2024-05-27T09:37:37.354483+0000","last_became_active":"2024-05-27T09:27:18.554932+0000","last_became_peered":"2024-05-27T09:27:18.554932+0000","last_unstale":"2024-05-27T09:37:37.354483+0000","last_undegraded":"2024-05-27T09:37:37.354483+0000","last_fullsized":"2024-05-27T09:37:37.354483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0083893030000000007,"stat_sum":{"num_bytes":24206949,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":20,"num_read_kb":12303,"num_write":329,"num_write_kb":23805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"49'68","reported_seq":163,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:41.368008+0000","last_change":"2024-05-27T09:27:17.547895+0000","last_active":"2024-05-27T09:37:41.368008+0000","last_peered":"2024-05-27T09:37:41.368008+0000","last_clean":"2024-05-27T09:37:41.368008+0000","last_became_active":"2024-05-27T09:27:17.547523+0000","last_became_peered":"2024-05-27T09:27:17.547523+0000","last_unstale":"2024-05-27T09:37:41.368008+0000","last_undegraded":"2024-05-27T09:37:41.368008+0000","last_fullsized":"2024-05-27T09:37:41.368008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":66,"num_write_kb":8220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"50'366","reported_seq":2491,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.385150+0000","last_change":"2024-05-27T09:33:11.457430+0000","last_active":"2024-05-27T09:37:37.385150+0000","last_peered":"2024-05-27T09:37:37.385150+0000","last_clean":"2024-05-27T09:37:37.385150+0000","last_became_active":"2024-05-27T09:27:18.582360+0000","last_became_peered":"2024-05-27T09:27:18.582360+0000","last_unstale":"2024-05-27T09:37:37.385150+0000","last_undegraded":"2024-05-27T09:37:37.385150+0000","last_fullsized":"2024-05-27T09:37:37.385150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:40:26.596775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0087858729999999996,"stat_sum":{"num_bytes":23260375,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":23,"num_read_kb":13874,"num_write":361,"num_write_kb":35144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"49'42","reported_seq":124,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.543556+0000","last_change":"2024-05-27T09:27:17.571481+0000","last_active":"2024-05-27T09:37:15.543556+0000","last_peered":"2024-05-27T09:37:15.543556+0000","last_clean":"2024-05-27T09:37:15.543556+0000","last_became_active":"2024-05-27T09:27:17.571146+0000","last_became_peered":"2024-05-27T09:27:17.571146+0000","last_unstale":"2024-05-27T09:37:15.543556+0000","last_undegraded":"2024-05-27T09:37:15.543556+0000","last_fullsized":"2024-05-27T09:37:15.543556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":8,"num_read_kb":4103,"num_write":60,"num_write_kb":4132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"50'302","reported_seq":2417,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.390053+0000","last_change":"2024-05-27T09:33:11.454615+0000","last_active":"2024-05-27T09:37:37.390053+0000","last_peered":"2024-05-27T09:37:37.390053+0000","last_clean":"2024-05-27T09:37:37.390053+0000","last_became_active":"2024-05-27T09:27:18.579440+0000","last_became_peered":"2024-05-27T09:27:18.579440+0000","last_unstale":"2024-05-27T09:37:37.390053+0000","last_undegraded":"2024-05-27T09:37:37.390053+0000","last_fullsized":"2024-05-27T09:37:37.390053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:06:43.673978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079260630000000006,"stat_sum":{"num_bytes":7949059,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":12,"num_read_kb":4346,"num_write":296,"num_write_kb":16109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"49'27","reported_seq":104,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.632660+0000","last_change":"2024-05-27T09:27:17.551295+0000","last_active":"2024-05-27T09:37:15.632660+0000","last_peered":"2024-05-27T09:37:15.632660+0000","last_clean":"2024-05-27T09:37:15.632660+0000","last_became_active":"2024-05-27T09:27:17.550861+0000","last_became_peered":"2024-05-27T09:27:17.550861+0000","last_unstale":"2024-05-27T09:37:15.632660+0000","last_undegraded":"2024-05-27T09:37:15.632660+0000","last_fullsized":"2024-05-27T09:37:15.632660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:17:13.955191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":4099,"num_write":31,"num_write_kb":4134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"50'337","reported_seq":2461,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.376610+0000","last_change":"2024-05-27T09:33:11.453500+0000","last_active":"2024-05-27T09:37:37.376610+0000","last_peered":"2024-05-27T09:37:37.376610+0000","last_clean":"2024-05-27T09:37:37.376610+0000","last_became_active":"2024-05-27T09:27:18.543683+0000","last_became_peered":"2024-05-27T09:27:18.543683+0000","last_unstale":"2024-05-27T09:37:37.376610+0000","last_undegraded":"2024-05-27T09:37:37.376610+0000","last_fullsized":"2024-05-27T09:37:37.376610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012478668,"stat_sum":{"num_bytes":14963144,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":20,"num_read_kb":4115,"num_write":326,"num_write_kb":22968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"49'93","reported_seq":171,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.952185+0000","last_change":"2024-05-27T09:27:17.559255+0000","last_active":"2024-05-27T09:37:15.952185+0000","last_peered":"2024-05-27T09:37:15.952185+0000","last_clean":"2024-05-27T09:37:15.952185+0000","last_became_active":"2024-05-27T09:27:17.558973+0000","last_became_peered":"2024-05-27T09:27:17.558973+0000","last_unstale":"2024-05-27T09:37:15.952185+0000","last_undegraded":"2024-05-27T09:37:15.952185+0000","last_fullsized":"2024-05-27T09:37:15.952185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":4097,"num_write":96,"num_write_kb":4263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"50'366","reported_seq":2484,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.372191+0000","last_change":"2024-05-27T09:33:11.452657+0000","last_active":"2024-05-27T09:37:37.372191+0000","last_peered":"2024-05-27T09:37:37.372191+0000","last_clean":"2024-05-27T09:37:37.372191+0000","last_became_active":"2024-05-27T09:27:18.557076+0000","last_became_peered":"2024-05-27T09:27:18.557076+0000","last_unstale":"2024-05-27T09:37:37.372191+0000","last_undegraded":"2024-05-27T09:37:37.372191+0000","last_fullsized":"2024-05-27T09:37:37.372191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014584746000000001,"stat_sum":{"num_bytes":15385587,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":358,"num_write_kb":15182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"49'49","reported_seq":134,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.953808+0000","last_change":"2024-05-27T09:27:17.573700+0000","last_active":"2024-05-27T09:37:15.953808+0000","last_peered":"2024-05-27T09:37:15.953808+0000","last_clean":"2024-05-27T09:37:15.953808+0000","last_became_active":"2024-05-27T09:27:17.573484+0000","last_became_peered":"2024-05-27T09:27:17.573484+0000","last_unstale":"2024-05-27T09:37:15.953808+0000","last_undegraded":"2024-05-27T09:37:15.953808+0000","last_fullsized":"2024-05-27T09:37:15.953808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:48.905422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":51,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"50'336","reported_seq":2461,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.377629+0000","last_change":"2024-05-27T09:33:11.456688+0000","last_active":"2024-05-27T09:37:37.377629+0000","last_peered":"2024-05-27T09:37:37.377629+0000","last_clean":"2024-05-27T09:37:37.377629+0000","last_became_active":"2024-05-27T09:27:18.574959+0000","last_became_peered":"2024-05-27T09:27:18.574959+0000","last_unstale":"2024-05-27T09:37:37.377629+0000","last_undegraded":"2024-05-27T09:37:37.377629+0000","last_fullsized":"2024-05-27T09:37:37.377629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:53.188459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0093604480000000004,"stat_sum":{"num_bytes":20164393,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":23,"num_read_kb":12305,"num_write":329,"num_write_kb":28020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"50'74","reported_seq":184,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:46.937525+0000","last_change":"2024-05-27T09:37:46.937605+0000","last_active":"2024-05-27T09:37:46.937525+0000","last_peered":"2024-05-27T09:37:46.937525+0000","last_clean":"2024-05-27T09:37:46.937525+0000","last_became_active":"2024-05-27T09:27:12.690301+0000","last_became_peered":"2024-05-27T09:27:12.690301+0000","last_unstale":"2024-05-27T09:37:46.937525+0000","last_undegraded":"2024-05-27T09:37:46.937525+0000","last_fullsized":"2024-05-27T09:37:46.937525+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'74","last_scrub_stamp":"2024-05-27T09:37:46.937317+0000","last_deep_scrub":"50'74","last_deep_scrub_stamp":"2024-05-27T09:37:46.937317+0000","last_clean_scrub_stamp":"2024-05-27T09:37:46.937317+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.317649+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'12","reported_seq":95,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:41.365431+0000","last_change":"2024-05-27T09:27:17.567670+0000","last_active":"2024-05-27T09:37:41.365431+0000","last_peered":"2024-05-27T09:37:41.365431+0000","last_clean":"2024-05-27T09:37:41.365431+0000","last_became_active":"2024-05-27T09:27:17.566292+0000","last_became_peered":"2024-05-27T09:27:17.566292+0000","last_unstale":"2024-05-27T09:37:41.365431+0000","last_undegraded":"2024-05-27T09:37:41.365431+0000","last_fullsized":"2024-05-27T09:37:41.365431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:18.038221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194338,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":21,"num_write":14,"num_write_kb":4206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"50'341","reported_seq":2457,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.367949+0000","last_change":"2024-05-27T09:33:11.733467+0000","last_active":"2024-05-27T09:37:37.367949+0000","last_peered":"2024-05-27T09:37:37.367949+0000","last_clean":"2024-05-27T09:37:37.367949+0000","last_became_active":"2024-05-27T09:27:18.570782+0000","last_became_peered":"2024-05-27T09:27:18.570782+0000","last_unstale":"2024-05-27T09:37:37.367949+0000","last_undegraded":"2024-05-27T09:37:37.367949+0000","last_fullsized":"2024-05-27T09:37:37.367949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:22.877083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044685039999999999,"stat_sum":{"num_bytes":10376859,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":333,"num_write_kb":18496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"50'344","reported_seq":428,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:17.550853+0000","last_change":"2024-05-27T09:27:17.563259+0000","last_active":"2024-05-27T09:37:17.550853+0000","last_peered":"2024-05-27T09:37:17.550853+0000","last_clean":"2024-05-27T09:37:17.550853+0000","last_became_active":"2024-05-27T09:27:17.563010+0000","last_became_peered":"2024-05-27T09:27:17.563010+0000","last_unstale":"2024-05-27T09:37:17.550853+0000","last_undegraded":"2024-05-27T09:37:17.550853+0000","last_fullsized":"2024-05-27T09:37:17.550853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:37:53.711508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":8210,"num_write":339,"num_write_kb":12482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"50'323","reported_seq":2437,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.367708+0000","last_change":"2024-05-27T09:33:11.730552+0000","last_active":"2024-05-27T09:37:37.367708+0000","last_peered":"2024-05-27T09:37:37.367708+0000","last_clean":"2024-05-27T09:37:37.367708+0000","last_became_active":"2024-05-27T09:27:18.574090+0000","last_became_peered":"2024-05-27T09:27:18.574090+0000","last_unstale":"2024-05-27T09:37:37.367708+0000","last_undegraded":"2024-05-27T09:37:37.367708+0000","last_fullsized":"2024-05-27T09:37:37.367708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:29:43.530819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002904064,"stat_sum":{"num_bytes":5516261,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":316,"num_write_kb":17833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"50'123","reported_seq":318,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.312751+0000","last_change":"2024-05-27T09:27:17.551310+0000","last_active":"2024-05-27T09:37:37.312751+0000","last_peered":"2024-05-27T09:37:37.312751+0000","last_clean":"2024-05-27T09:37:37.312751+0000","last_became_active":"2024-05-27T09:27:17.550982+0000","last_became_peered":"2024-05-27T09:27:17.550982+0000","last_unstale":"2024-05-27T09:37:37.312751+0000","last_undegraded":"2024-05-27T09:37:37.312751+0000","last_fullsized":"2024-05-27T09:37:37.312751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":534380,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":130,"num_read_kb":615,"num_write":121,"num_write_kb":621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"50'348","reported_seq":2473,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.349125+0000","last_change":"2024-05-27T09:33:11.732933+0000","last_active":"2024-05-27T09:37:37.349125+0000","last_peered":"2024-05-27T09:37:37.349125+0000","last_clean":"2024-05-27T09:37:37.349125+0000","last_became_active":"2024-05-27T09:27:18.559017+0000","last_became_peered":"2024-05-27T09:27:18.559017+0000","last_unstale":"2024-05-27T09:37:37.349125+0000","last_undegraded":"2024-05-27T09:37:37.349125+0000","last_fullsized":"2024-05-27T09:37:37.349125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:06:28.204044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065168789999999997,"stat_sum":{"num_bytes":26414737,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":21,"num_read_kb":8209,"num_write":340,"num_write_kb":34165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"49'48","reported_seq":126,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.547944+0000","last_change":"2024-05-27T09:27:17.563947+0000","last_active":"2024-05-27T09:37:15.547944+0000","last_peered":"2024-05-27T09:37:15.547944+0000","last_clean":"2024-05-27T09:37:15.547944+0000","last_became_active":"2024-05-27T09:27:17.563640+0000","last_became_peered":"2024-05-27T09:27:17.563640+0000","last_unstale":"2024-05-27T09:37:15.547944+0000","last_undegraded":"2024-05-27T09:37:15.547944+0000","last_fullsized":"2024-05-27T09:37:15.547944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:02.747708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":43,"num_write_kb":8223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"50'359","reported_seq":2481,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:45.815038+0000","last_change":"2024-05-27T09:33:11.442514+0000","last_active":"2024-05-27T09:37:45.815038+0000","last_peered":"2024-05-27T09:37:45.815038+0000","last_clean":"2024-05-27T09:37:45.815038+0000","last_became_active":"2024-05-27T09:27:18.574710+0000","last_became_peered":"2024-05-27T09:27:18.574710+0000","last_unstale":"2024-05-27T09:37:45.815038+0000","last_undegraded":"2024-05-27T09:37:45.815038+0000","last_fullsized":"2024-05-27T09:37:45.815038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075561990000000004,"stat_sum":{"num_bytes":9767075,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":17,"num_read_kb":8206,"num_write":352,"num_write_kb":9721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"49'81","reported_seq":158,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.952151+0000","last_change":"2024-05-27T09:27:17.550749+0000","last_active":"2024-05-27T09:37:15.952151+0000","last_peered":"2024-05-27T09:37:15.952151+0000","last_clean":"2024-05-27T09:37:15.952151+0000","last_became_active":"2024-05-27T09:27:17.550378+0000","last_became_peered":"2024-05-27T09:27:17.550378+0000","last_unstale":"2024-05-27T09:37:15.952151+0000","last_undegraded":"2024-05-27T09:37:15.952151+0000","last_fullsized":"2024-05-27T09:37:15.952151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:20.167629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83,"num_write_kb":4165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"50'360","reported_seq":2484,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.377163+0000","last_change":"2024-05-27T09:33:11.451931+0000","last_active":"2024-05-27T09:37:37.377163+0000","last_peered":"2024-05-27T09:37:37.377163+0000","last_clean":"2024-05-27T09:37:37.377163+0000","last_became_active":"2024-05-27T09:27:18.566326+0000","last_became_peered":"2024-05-27T09:27:18.566326+0000","last_unstale":"2024-05-27T09:37:37.377163+0000","last_undegraded":"2024-05-27T09:37:37.377163+0000","last_fullsized":"2024-05-27T09:37:37.377163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:04:50.848181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075094710000000002,"stat_sum":{"num_bytes":14788118,"num_objects":187,"num_object_clones":0,"num_object_copies":374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":187,"num_whiteouts":0,"num_read":20,"num_read_kb":8209,"num_write":356,"num_write_kb":35101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'352","reported_seq":431,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:20.528059+0000","last_change":"2024-05-27T09:27:17.557912+0000","last_active":"2024-05-27T09:37:20.528059+0000","last_peered":"2024-05-27T09:37:20.528059+0000","last_clean":"2024-05-27T09:37:20.528059+0000","last_became_active":"2024-05-27T09:27:17.557216+0000","last_became_peered":"2024-05-27T09:27:17.557216+0000","last_unstale":"2024-05-27T09:37:20.528059+0000","last_undegraded":"2024-05-27T09:37:20.528059+0000","last_fullsized":"2024-05-27T09:37:20.528059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:21:22.020724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":4098,"num_write":353,"num_write_kb":8433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"50'347","reported_seq":2464,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.354084+0000","last_change":"2024-05-27T09:33:11.728508+0000","last_active":"2024-05-27T09:37:37.354084+0000","last_peered":"2024-05-27T09:37:37.354084+0000","last_clean":"2024-05-27T09:37:37.354084+0000","last_became_active":"2024-05-27T09:27:18.562565+0000","last_became_peered":"2024-05-27T09:27:18.562565+0000","last_unstale":"2024-05-27T09:37:37.354084+0000","last_undegraded":"2024-05-27T09:37:37.354084+0000","last_fullsized":"2024-05-27T09:37:37.354084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:45:47.000935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030336680000000002,"stat_sum":{"num_bytes":10461277,"num_objects":176,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":176,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":338,"num_write_kb":10392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"50'198","reported_seq":280,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:35.644068+0000","last_change":"2024-05-27T09:27:17.571911+0000","last_active":"2024-05-27T09:37:35.644068+0000","last_peered":"2024-05-27T09:37:35.644068+0000","last_clean":"2024-05-27T09:37:35.644068+0000","last_became_active":"2024-05-27T09:27:17.571551+0000","last_became_peered":"2024-05-27T09:27:17.571551+0000","last_unstale":"2024-05-27T09:37:35.644068+0000","last_undegraded":"2024-05-27T09:37:35.644068+0000","last_fullsized":"2024-05-27T09:37:35.644068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:20.167629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":455,"num_write_kb":2056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"50'329","reported_seq":2447,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.348511+0000","last_change":"2024-05-27T09:33:11.734048+0000","last_active":"2024-05-27T09:37:37.348511+0000","last_peered":"2024-05-27T09:37:37.348511+0000","last_clean":"2024-05-27T09:37:37.348511+0000","last_became_active":"2024-05-27T09:27:18.558463+0000","last_became_peered":"2024-05-27T09:27:18.558463+0000","last_unstale":"2024-05-27T09:37:37.348511+0000","last_undegraded":"2024-05-27T09:37:37.348511+0000","last_fullsized":"2024-05-27T09:37:37.348511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:29:43.530819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041648589999999999,"stat_sum":{"num_bytes":14892598,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":326,"num_write_kb":22861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"49'171","reported_seq":251,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:41.368959+0000","last_change":"2024-05-27T09:27:17.567973+0000","last_active":"2024-05-27T09:37:41.368959+0000","last_peered":"2024-05-27T09:37:41.368959+0000","last_clean":"2024-05-27T09:37:41.368959+0000","last_became_active":"2024-05-27T09:27:17.567650+0000","last_became_peered":"2024-05-27T09:27:17.567650+0000","last_unstale":"2024-05-27T09:37:41.368959+0000","last_undegraded":"2024-05-27T09:37:41.368959+0000","last_fullsized":"2024-05-27T09:37:41.368959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:25:52.924185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":176,"num_write_kb":4290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"50'347","reported_seq":2476,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.350628+0000","last_change":"2024-05-27T09:33:11.733662+0000","last_active":"2024-05-27T09:37:37.350628+0000","last_peered":"2024-05-27T09:37:37.350628+0000","last_clean":"2024-05-27T09:37:37.350628+0000","last_became_active":"2024-05-27T09:27:18.554462+0000","last_became_peered":"2024-05-27T09:27:18.554462+0000","last_unstale":"2024-05-27T09:37:37.350628+0000","last_undegraded":"2024-05-27T09:37:37.350628+0000","last_fullsized":"2024-05-27T09:37:37.350628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:21:17.734561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042444429999999997,"stat_sum":{"num_bytes":6471160,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":27,"num_read_kb":4122,"num_write":339,"num_write_kb":6465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"49'126","reported_seq":208,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:41.388821+0000","last_change":"2024-05-27T09:27:17.570300+0000","last_active":"2024-05-27T09:37:41.388821+0000","last_peered":"2024-05-27T09:37:41.388821+0000","last_clean":"2024-05-27T09:37:41.388821+0000","last_became_active":"2024-05-27T09:27:17.569685+0000","last_became_peered":"2024-05-27T09:27:17.569685+0000","last_unstale":"2024-05-27T09:37:41.388821+0000","last_undegraded":"2024-05-27T09:37:41.388821+0000","last_fullsized":"2024-05-27T09:37:41.388821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":4108,"num_write":127,"num_write_kb":4196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"50'355","reported_seq":2484,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.352377+0000","last_change":"2024-05-27T09:33:11.450475+0000","last_active":"2024-05-27T09:37:37.352377+0000","last_peered":"2024-05-27T09:37:37.352377+0000","last_clean":"2024-05-27T09:37:37.352377+0000","last_became_active":"2024-05-27T09:27:18.562455+0000","last_became_peered":"2024-05-27T09:27:18.562455+0000","last_unstale":"2024-05-27T09:37:37.352377+0000","last_undegraded":"2024-05-27T09:37:37.352377+0000","last_fullsized":"2024-05-27T09:37:37.352377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:31.423566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082016410000000008,"stat_sum":{"num_bytes":31948882,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":27,"num_read_kb":17799,"num_write":353,"num_write_kb":35479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"49'171","reported_seq":252,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:41.379258+0000","last_change":"2024-05-27T09:27:17.563172+0000","last_active":"2024-05-27T09:37:41.379258+0000","last_peered":"2024-05-27T09:37:41.379258+0000","last_clean":"2024-05-27T09:37:41.379258+0000","last_became_active":"2024-05-27T09:27:17.562941+0000","last_became_peered":"2024-05-27T09:27:17.562941+0000","last_unstale":"2024-05-27T09:37:41.379258+0000","last_undegraded":"2024-05-27T09:37:41.379258+0000","last_fullsized":"2024-05-27T09:37:41.379258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:51.742078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":171,"num_write_kb":4214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"50'349","reported_seq":2470,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.349831+0000","last_change":"2024-05-27T09:33:11.731891+0000","last_active":"2024-05-27T09:37:37.349831+0000","last_peered":"2024-05-27T09:37:37.349831+0000","last_clean":"2024-05-27T09:37:37.349831+0000","last_became_active":"2024-05-27T09:27:18.555280+0000","last_became_peered":"2024-05-27T09:27:18.555280+0000","last_unstale":"2024-05-27T09:37:37.349831+0000","last_undegraded":"2024-05-27T09:37:37.349831+0000","last_fullsized":"2024-05-27T09:37:37.349831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:31.147919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041574209999999997,"stat_sum":{"num_bytes":13440874,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":19,"num_read_kb":8208,"num_write":339,"num_write_kb":17394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"50'280","reported_seq":359,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:45.544659+0000","last_change":"2024-05-27T09:27:17.548266+0000","last_active":"2024-05-27T09:37:45.544659+0000","last_peered":"2024-05-27T09:37:45.544659+0000","last_clean":"2024-05-27T09:37:45.544659+0000","last_became_active":"2024-05-27T09:27:17.547938+0000","last_became_peered":"2024-05-27T09:27:17.547938+0000","last_unstale":"2024-05-27T09:37:45.544659+0000","last_undegraded":"2024-05-27T09:37:45.544659+0000","last_fullsized":"2024-05-27T09:37:45.544659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:59.686417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7314063,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":4100,"num_write":309,"num_write_kb":7303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"50'342","reported_seq":2460,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.352342+0000","last_change":"2024-05-27T09:33:11.625744+0000","last_active":"2024-05-27T09:37:37.352342+0000","last_peered":"2024-05-27T09:37:37.352342+0000","last_clean":"2024-05-27T09:37:37.352342+0000","last_became_active":"2024-05-27T09:27:18.563227+0000","last_became_peered":"2024-05-27T09:27:18.563227+0000","last_unstale":"2024-05-27T09:37:37.352342+0000","last_undegraded":"2024-05-27T09:37:37.352342+0000","last_fullsized":"2024-05-27T09:37:37.352342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:30:49.326058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010490250000000001,"stat_sum":{"num_bytes":10717337,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":15,"num_read_kb":2569,"num_write":334,"num_write_kb":14740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"50'342","reported_seq":424,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:41.370164+0000","last_change":"2024-05-27T09:27:17.562200+0000","last_active":"2024-05-27T09:37:41.370164+0000","last_peered":"2024-05-27T09:37:41.370164+0000","last_clean":"2024-05-27T09:37:41.370164+0000","last_became_active":"2024-05-27T09:27:17.561857+0000","last_became_peered":"2024-05-27T09:27:17.561857+0000","last_unstale":"2024-05-27T09:37:41.370164+0000","last_undegraded":"2024-05-27T09:37:41.370164+0000","last_fullsized":"2024-05-27T09:37:41.370164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:05.209528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":16,"num_read_kb":8206,"num_write":340,"num_write_kb":12481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"50'360","reported_seq":2475,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.372550+0000","last_change":"2024-05-27T09:33:11.455285+0000","last_active":"2024-05-27T09:37:37.372550+0000","last_peered":"2024-05-27T09:37:37.372550+0000","last_clean":"2024-05-27T09:37:37.372550+0000","last_became_active":"2024-05-27T09:27:18.572339+0000","last_became_peered":"2024-05-27T09:27:18.572339+0000","last_unstale":"2024-05-27T09:37:37.372550+0000","last_undegraded":"2024-05-27T09:37:37.372550+0000","last_fullsized":"2024-05-27T09:37:37.372550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:11:51.773780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011410574,"stat_sum":{"num_bytes":1288358,"num_objects":185,"num_object_clones":0,"num_object_copies":370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":185,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":355,"num_write_kb":9639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"50'330","reported_seq":2455,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.388768+0000","last_change":"2024-05-27T09:33:11.456154+0000","last_active":"2024-05-27T09:37:37.388768+0000","last_peered":"2024-05-27T09:37:37.388768+0000","last_clean":"2024-05-27T09:37:37.388768+0000","last_became_active":"2024-05-27T09:27:18.572307+0000","last_became_peered":"2024-05-27T09:27:18.572307+0000","last_unstale":"2024-05-27T09:37:37.388768+0000","last_undegraded":"2024-05-27T09:37:37.388768+0000","last_fullsized":"2024-05-27T09:37:37.388768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:38:38.928939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089418529999999996,"stat_sum":{"num_bytes":7551421,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":315,"num_write_kb":11635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"49'32","reported_seq":111,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:17.594035+0000","last_change":"2024-05-27T09:27:17.570634+0000","last_active":"2024-05-27T09:37:17.594035+0000","last_peered":"2024-05-27T09:37:17.594035+0000","last_clean":"2024-05-27T09:37:17.594035+0000","last_became_active":"2024-05-27T09:27:17.569866+0000","last_became_peered":"2024-05-27T09:27:17.569866+0000","last_unstale":"2024-05-27T09:37:17.594035+0000","last_undegraded":"2024-05-27T09:37:17.594035+0000","last_fullsized":"2024-05-27T09:37:17.594035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4099,"num_write":35,"num_write_kb":4131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"50'359","reported_seq":2477,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.381088+0000","last_change":"2024-05-27T09:33:11.737809+0000","last_active":"2024-05-27T09:37:37.381088+0000","last_peered":"2024-05-27T09:37:37.381088+0000","last_clean":"2024-05-27T09:37:37.381088+0000","last_became_active":"2024-05-27T09:27:18.549035+0000","last_became_peered":"2024-05-27T09:27:18.549035+0000","last_unstale":"2024-05-27T09:37:37.381088+0000","last_undegraded":"2024-05-27T09:37:37.381088+0000","last_fullsized":"2024-05-27T09:37:37.381088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0090041500000000007,"stat_sum":{"num_bytes":20118472,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":14,"num_read_kb":13,"num_write":350,"num_write_kb":27987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"49'143","reported_seq":227,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:17.594087+0000","last_change":"2024-05-27T09:27:17.568143+0000","last_active":"2024-05-27T09:37:17.594087+0000","last_peered":"2024-05-27T09:37:17.594087+0000","last_clean":"2024-05-27T09:37:17.594087+0000","last_became_active":"2024-05-27T09:27:17.567235+0000","last_became_peered":"2024-05-27T09:27:17.567235+0000","last_unstale":"2024-05-27T09:37:17.594087+0000","last_undegraded":"2024-05-27T09:37:17.594087+0000","last_fullsized":"2024-05-27T09:37:17.594087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":15,"num_read_kb":12312,"num_write":136,"num_write_kb":12381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"50'319","reported_seq":2444,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.373051+0000","last_change":"2024-05-27T09:33:11.732530+0000","last_active":"2024-05-27T09:37:37.373051+0000","last_peered":"2024-05-27T09:37:37.373051+0000","last_clean":"2024-05-27T09:37:37.373051+0000","last_became_active":"2024-05-27T09:27:18.562919+0000","last_became_peered":"2024-05-27T09:27:18.562919+0000","last_unstale":"2024-05-27T09:37:37.373051+0000","last_undegraded":"2024-05-27T09:37:37.373051+0000","last_fullsized":"2024-05-27T09:37:37.373051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:09:22.339045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0063913879999999996,"stat_sum":{"num_bytes":22598856,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":21,"num_read_kb":8211,"num_write":313,"num_write_kb":22222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"49'87","reported_seq":165,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.953251+0000","last_change":"2024-05-27T09:27:17.551193+0000","last_active":"2024-05-27T09:37:15.953251+0000","last_peered":"2024-05-27T09:37:15.953251+0000","last_clean":"2024-05-27T09:37:15.953251+0000","last_became_active":"2024-05-27T09:27:17.550849+0000","last_became_peered":"2024-05-27T09:27:17.550849+0000","last_unstale":"2024-05-27T09:37:15.953251+0000","last_undegraded":"2024-05-27T09:37:15.953251+0000","last_fullsized":"2024-05-27T09:37:15.953251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:51.742078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":81,"num_write_kb":4153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"50'336","reported_seq":2454,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:37.385192+0000","last_change":"2024-05-27T09:33:11.732054+0000","last_active":"2024-05-27T09:37:37.385192+0000","last_peered":"2024-05-27T09:37:37.385192+0000","last_clean":"2024-05-27T09:37:37.385192+0000","last_became_active":"2024-05-27T09:27:18.579574+0000","last_became_peered":"2024-05-27T09:27:18.579574+0000","last_unstale":"2024-05-27T09:37:37.385192+0000","last_undegraded":"2024-05-27T09:37:37.385192+0000","last_fullsized":"2024-05-27T09:37:37.385192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:17.511301+0000","last_clean_scrub_stamp":"2024-05-27T09:27:17.511301+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:56:05.185246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062209780000000003,"stat_sum":{"num_bytes":14492175,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":14,"num_read_kb":4108,"num_write":330,"num_write_kb":14321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"49'82","reported_seq":173,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:15.632796+0000","last_change":"2024-05-27T09:27:17.549711+0000","last_active":"2024-05-27T09:37:15.632796+0000","last_peered":"2024-05-27T09:37:15.632796+0000","last_clean":"2024-05-27T09:37:15.632796+0000","last_became_active":"2024-05-27T09:27:17.549291+0000","last_became_peered":"2024-05-27T09:27:17.549291+0000","last_unstale":"2024-05-27T09:37:15.632796+0000","last_undegraded":"2024-05-27T09:37:15.632796+0000","last_fullsized":"2024-05-27T09:37:15.632796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:27:16.496192+0000","last_clean_scrub_stamp":"2024-05-27T09:27:16.496192+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:16:22.057803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":41,"num_read_kb":4136,"num_write":85,"num_write_kb":8333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":952881208,"num_objects":10697,"num_object_clones":0,"num_object_copies":21394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10697,"num_whiteouts":0,"num_read":1250,"num_read_kb":378565,"num_write":21433,"num_write_kb":1379166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1734066176,"data_stored":1904222320,"data_compressed":168711384,"data_compressed_allocated":186425344,"data_compressed_original":412123136,"omap_allocated":0,"internal_metadata":0},"log_size":21961,"ondisk_log_size":21961,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":188209077,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":557,"num_read_kb":123805,"num_write":4446,"num_write_kb":189274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":105,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19729,"num_omap_keys":33,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":84164608,"data_stored":376418154,"data_compressed":49776484,"data_compressed_allocated":62496768,"data_compressed_original":354902016,"omap_allocated":0,"internal_metadata":0},"log_size":4150,"ondisk_log_size":4150,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65078,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509571,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":381232,"kb_used_data":309800,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":93990608,"statfs":{"total":96636764160,"available":96246382592,"internally_reserved":0,"allocated":317235200,"data_stored":412122108,"data_compressed":41350796,"data_compressed_allocated":47357952,"data_compressed_original":150941696,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509571,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":282276,"kb_used_data":233052,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94089564,"statfs":{"total":96636764160,"available":96347713536,"internally_reserved":0,"allocated":238645248,"data_stored":280794177,"data_compressed":23230909,"data_compressed_allocated":26537984,"data_compressed_original":76705792,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509571,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246488,"kb_used_data":201424,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94125352,"statfs":{"total":96636764160,"available":96384360448,"internally_reserved":0,"allocated":206258176,"data_stored":257021246,"data_compressed":23229218,"data_compressed_allocated":26480640,"data_compressed_original":83582976,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509571,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356864,"kb_used_data":285432,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94014976,"statfs":{"total":96636764160,"available":96271335424,"internally_reserved":0,"allocated":292282368,"data_stored":379742172,"data_compressed":39531620,"data_compressed_allocated":45047808,"data_compressed_original":141885440,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542274,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":157168,"kb_used_data":130152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94214672,"statfs":{"total":96636764160,"available":96475824128,"internally_reserved":0,"allocated":133275648,"data_stored":168154774,"data_compressed":18202841,"data_compressed_allocated":20492288,"data_compressed_original":60579840,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542274,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":196160,"kb_used_data":146936,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94175680,"statfs":{"total":96636764160,"available":96435896320,"internally_reserved":0,"allocated":150462464,"data_stored":185985448,"data_compressed":18856887,"data_compressed_allocated":21581824,"data_compressed_original":63496192,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542274,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":358888,"kb_used_data":305504,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94012952,"statfs":{"total":96636764160,"available":96269262848,"internally_reserved":0,"allocated":312836096,"data_stored":372593203,"data_compressed":31452299,"data_compressed_allocated":35487744,"data_compressed_original":103534592,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542274,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":221244,"kb_used_data":167860,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94150596,"statfs":{"total":96636764160,"available":96410210304,"internally_reserved":0,"allocated":171888640,"data_stored":227475258,"data_compressed":22698376,"data_compressed_allocated":26025984,"data_compressed_original":87445504,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13996032,"data_stored":67110923,"data_compressed":8258411,"data_compressed_allocated":10555392,"data_compressed_original":63688704,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7446528,"data_stored":37748896,"data_compressed":4390710,"data_compressed_allocated":5623808,"data_compressed_original":35934208,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7921664,"data_stored":33555138,"data_compressed":4517094,"data_compressed_allocated":5791744,"data_compressed_original":31436800,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19890176,"data_stored":83888049,"data_compressed":11332765,"data_compressed_allocated":14295040,"data_compressed_original":78307328,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12206080,"data_stored":50333888,"data_compressed":7575831,"data_compressed_allocated":9129984,"data_compressed_original":47296512,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10080256,"data_stored":45598520,"data_compressed":5669179,"data_compressed_allocated":7151616,"data_compressed_original":42692608,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7192576,"data_stored":32481718,"data_compressed":4486880,"data_compressed_allocated":5623808,"data_compressed_original":30924800,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5431296,"data_stored":25701022,"data_compressed":3545614,"data_compressed_allocated":4325376,"data_compressed_original":24621056,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":277721088,"data_stored":312372852,"data_compressed":31273209,"data_compressed_allocated":34492416,"data_compressed_original":78196736,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":198180864,"data_stored":218423585,"data_compressed":18805969,"data_compressed_allocated":20811776,"data_compressed_original":47075328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":230158336,"data_stored":246980642,"data_compressed":18713815,"data_compressed_allocated":20746240,"data_compressed_original":45268992,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":296779776,"data_stored":327975662,"data_compressed":30018031,"data_compressed_allocated":33062912,"data_compressed_original":72634368,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":159117312,"data_stored":176882973,"data_compressed":15122545,"data_compressed_allocated":16896000,"data_compressed_original":40148992,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":302125056,"data_stored":326145918,"data_compressed":25750581,"data_compressed_allocated":28291072,"data_compressed_original":60268544,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":142704640,"data_stored":153245333,"data_compressed":14370007,"data_compressed_allocated":15958016,"data_compressed_original":32571392,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":127279104,"data_stored":142195355,"data_compressed":14657227,"data_compressed_allocated":16166912,"data_compressed_original":35958784,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T09:37:52.857 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.858 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.858 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.858 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.858 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.859 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.859 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.859 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.859 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.859 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.859 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.860 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.860 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.860 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.860 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.860 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.860 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.861 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.861 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.861 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.861 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.861 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.861 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.862 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.862 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.862 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.862 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.862 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.863 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.863 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.863 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.863 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.863 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.863 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.864 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.864 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.864 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.864 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.864 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.864 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.865 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.865 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.865 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.865 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.865 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.865 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.866 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.866 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.866 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.866 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.866 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.866 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.867 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.867 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.867 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.867 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.867 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.867 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.868 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.868 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.868 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.868 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.868 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.868 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.869 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.869 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.869 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.869 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.869 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.869 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.869 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.870 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.870 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.870 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.870 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.870 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.871 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.871 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.871 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.871 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.871 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.872 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.872 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.872 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.872 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.873 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.873 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.873 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.873 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.873 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.874 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.874 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.874 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.874 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-05-27T09:27:17.511301+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=17, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.875 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-05-27T09:27:16.496192+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=27, tm_sec=16, tm_wday=0, tm_yday=148, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=27, tm_hour=9, tm_min=37, tm_sec=43, tm_wday=0, tm_yday=148, tm_isdst=0) 2024-05-27T09:37:52.875 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-27T09:38:12.876 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T09:38:13.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.159+0000 7fe843c00640 1 Processor -- start 2024-05-27T09:38:13.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.159+0000 7fe843c00640 1 -- start start 2024-05-27T09:38:13.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.161+0000 7fe843c00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe83c1593b0 0x7fe83c142720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:13.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.161+0000 7fe843c00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe83c0739a0 con 0x7fe83c0c2990 2024-05-27T09:38:13.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.162+0000 7fe843c00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe83c073c80 con 0x7fe83c073260 2024-05-27T09:38:13.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.162+0000 7fe843c00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fe83c074d20 con 0x7fe83c1593b0 2024-05-27T09:38:13.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.162+0000 7fe841200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe83c1593b0 0x7fe83c142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:13.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.162+0000 7fe841200640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe83c1593b0 0x7fe83c142720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:39402/0 (socket says 172.21.15.114:39402) 2024-05-27T09:38:13.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.162+0000 7fe841200640 1 -- 172.21.15.114:0/500116076 learned_addr learned my addr 172.21.15.114:0/500116076 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:38:13.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.162+0000 7fe841200640 1 -- 172.21.15.114:0/500116076 >> v1:172.21.15.163:6789/0 conn(0x7fe83c073260 legacy=0x7fe83c15a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:13.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.162+0000 7fe841200640 1 -- 172.21.15.114:0/500116076 >> v1:172.21.15.114:6789/0 conn(0x7fe83c0c2990 legacy=0x7fe83c163d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:13.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.162+0000 7fe841200640 1 -- 172.21.15.114:0/500116076 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83c073f60 con 0x7fe83c1593b0 2024-05-27T09:38:13.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.162+0000 7fe83b400640 1 -- 172.21.15.114:0/500116076 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3461032540 0 0) 0x7fe83c0739a0 con 0x7fe83c0c2990 2024-05-27T09:38:13.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.163+0000 7fe841200640 1 --2- 172.21.15.114:0/500116076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe83c1593b0 0x7fe83c142720 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe830009870 tx=0x7fe83002a3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae00287260c10845 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:13.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.164+0000 7fe83b400640 1 -- 172.21.15.114:0/500116076 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe830037070 con 0x7fe83c1593b0 2024-05-27T09:38:13.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.164+0000 7fe83b400640 1 -- 172.21.15.114:0/500116076 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe830032640 con 0x7fe83c1593b0 2024-05-27T09:38:13.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.164+0000 7fe83b400640 1 -- 172.21.15.114:0/500116076 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe830032a10 con 0x7fe83c1593b0 2024-05-27T09:38:13.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.164+0000 7fe843c00640 1 -- 172.21.15.114:0/500116076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe83c1593b0 msgr2=0x7fe83c142720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:13.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.164+0000 7fe843c00640 1 --2- 172.21.15.114:0/500116076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe83c1593b0 0x7fe83c142720 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe830009870 tx=0x7fe83002a3c0 comp rx=0 tx=0).stop 2024-05-27T09:38:13.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.164+0000 7fe843c00640 1 -- 172.21.15.114:0/500116076 shutdown_connections 2024-05-27T09:38:13.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.164+0000 7fe843c00640 1 --2- 172.21.15.114:0/500116076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe83c1593b0 0x7fe83c142720 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:13.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.164+0000 7fe843c00640 1 -- 172.21.15.114:0/500116076 >> 172.21.15.114:0/500116076 conn(0x7fe83c06f6e0 msgr2=0x7fe83c070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:13.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.165+0000 7fe843c00640 1 -- 172.21.15.114:0/500116076 shutdown_connections 2024-05-27T09:38:13.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.165+0000 7fe843c00640 1 -- 172.21.15.114:0/500116076 wait complete. 2024-05-27T09:38:13.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.167+0000 7fe843c00640 1 Processor -- start 2024-05-27T09:38:13.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.167+0000 7fe843c00640 1 -- start start 2024-05-27T09:38:13.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.169+0000 7fe843c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe83c073260 0x7fe83c0b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:13.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.169+0000 7fe843c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe83c0c2990 0x7fe83c141420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:13.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.169+0000 7fe843c00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe83c1593b0 0x7fe83c13a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:13.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.169+0000 7fe843c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c074800 con 0x7fe83c073260 2024-05-27T09:38:13.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.169+0000 7fe843c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c074520 con 0x7fe83c0c2990 2024-05-27T09:38:13.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.169+0000 7fe843c00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fe83c073670 con 0x7fe83c1593b0 2024-05-27T09:38:13.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.169+0000 7fe841200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe83c073260 0x7fe83c0b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:13.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.169+0000 7fe841200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe83c073260 0x7fe83c0b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59546/0 (socket says 172.21.15.114:59546) 2024-05-27T09:38:13.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.169+0000 7fe841200640 1 -- 172.21.15.114:0/445095612 learned_addr learned my addr 172.21.15.114:0/445095612 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:38:13.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.169+0000 7fe841c00640 1 --2- 172.21.15.114:0/445095612 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe83c1593b0 0x7fe83c13a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:13.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.169+0000 7fe83be00640 1 --2- 172.21.15.114:0/445095612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe83c0c2990 0x7fe83c141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:13.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.169+0000 7fe841200640 1 -- 172.21.15.114:0/445095612 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe83c1593b0 msgr2=0x7fe83c13a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:13.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.169+0000 7fe841200640 1 --2- 172.21.15.114:0/445095612 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe83c1593b0 0x7fe83c13a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:13.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.169+0000 7fe841200640 1 -- 172.21.15.114:0/445095612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe83c0c2990 msgr2=0x7fe83c141420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:13.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.169+0000 7fe841200640 1 --2- 172.21.15.114:0/445095612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe83c0c2990 0x7fe83c141420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:13.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.169+0000 7fe841200640 1 -- 172.21.15.114:0/445095612 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83c0cab30 con 0x7fe83c073260 2024-05-27T09:38:13.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.170+0000 7fe841200640 1 --2- 172.21.15.114:0/445095612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe83c073260 0x7fe83c0b1a20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe8300090b0 tx=0x7fe8300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:13.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.170+0000 7fe838c00640 1 -- 172.21.15.114:0/445095612 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe830037040 con 0x7fe83c073260 2024-05-27T09:38:13.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.170+0000 7fe838c00640 1 -- 172.21.15.114:0/445095612 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe830004230 con 0x7fe83c073260 2024-05-27T09:38:13.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.170+0000 7fe843c00640 1 -- 172.21.15.114:0/445095612 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83c0cadd0 con 0x7fe83c073260 2024-05-27T09:38:13.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.170+0000 7fe838c00640 1 -- 172.21.15.114:0/445095612 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8300306f0 con 0x7fe83c073260 2024-05-27T09:38:13.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.170+0000 7fe843c00640 1 -- 172.21.15.114:0/445095612 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe83c0c8d30 con 0x7fe83c073260 2024-05-27T09:38:13.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.171+0000 7fe843c00640 1 -- 172.21.15.114:0/445095612 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe83c06a380 con 0x7fe83c073260 2024-05-27T09:38:13.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.173+0000 7fe838c00640 1 -- 172.21.15.114:0/445095612 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe830032500 con 0x7fe83c073260 2024-05-27T09:38:13.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.174+0000 7fe838c00640 1 --2- 172.21.15.114:0/445095612 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe820076300 0x7fe820078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:13.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.174+0000 7fe838c00640 1 -- 172.21.15.114:0/445095612 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7fe8300b8640 con 0x7fe83c073260 2024-05-27T09:38:13.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.174+0000 7fe83be00640 1 --2- 172.21.15.114:0/445095612 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe820076300 0x7fe820078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:13.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.175+0000 7fe83be00640 1 --2- 172.21.15.114:0/445095612 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe820076300 0x7fe820078810 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0b25c0 tx=0x7fe82c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:13.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.179+0000 7fe838c00640 1 -- 172.21.15.114:0/445095612 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe830081690 con 0x7fe83c073260 2024-05-27T09:38:13.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.386+0000 7fe843c00640 1 -- 172.21.15.114:0/445095612 --> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe83c0c41f0 con 0x7fe820076300 2024-05-27T09:38:13.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.395+0000 7fe838c00640 1 -- 172.21.15.114:0/445095612 <== mgr.4108 v2:172.21.15.163:6832/1122018204 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257627 (secure 0 0 0) 0x7fe83c0c41f0 con 0x7fe820076300 2024-05-27T09:38:13.400 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:38:13.402 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-05-27T09:38:13.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.399+0000 7fe843c00640 1 -- 172.21.15.114:0/445095612 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe820076300 msgr2=0x7fe820078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:13.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.399+0000 7fe843c00640 1 --2- 172.21.15.114:0/445095612 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe820076300 0x7fe820078810 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0b25c0 tx=0x7fe82c009040 comp rx=0 tx=0).stop 2024-05-27T09:38:13.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.399+0000 7fe843c00640 1 -- 172.21.15.114:0/445095612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe83c073260 msgr2=0x7fe83c0b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:13.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.399+0000 7fe843c00640 1 --2- 172.21.15.114:0/445095612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe83c073260 0x7fe83c0b1a20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe8300090b0 tx=0x7fe8300046c0 comp rx=0 tx=0).stop 2024-05-27T09:38:13.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.400+0000 7fe843c00640 1 -- 172.21.15.114:0/445095612 shutdown_connections 2024-05-27T09:38:13.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.400+0000 7fe843c00640 1 --2- 172.21.15.114:0/445095612 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe820076300 0x7fe820078810 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:13.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.400+0000 7fe843c00640 1 --2- 172.21.15.114:0/445095612 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe83c1593b0 0x7fe83c13a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:13.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.400+0000 7fe843c00640 1 --2- 172.21.15.114:0/445095612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe83c0c2990 0x7fe83c141420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:13.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.400+0000 7fe843c00640 1 --2- 172.21.15.114:0/445095612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe83c073260 0x7fe83c0b1a20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:13.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.400+0000 7fe843c00640 1 -- 172.21.15.114:0/445095612 >> 172.21.15.114:0/445095612 conn(0x7fe83c06f6e0 msgr2=0x7fe83c0b2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:13.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.400+0000 7fe843c00640 1 -- 172.21.15.114:0/445095612 shutdown_connections 2024-05-27T09:38:13.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.400+0000 7fe843c00640 1 -- 172.21.15.114:0/445095612 wait complete. 2024-05-27T09:38:13.427 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-05-27T09:38:12.948776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1141882990,"num_objects":10877,"num_object_clones":0,"num_object_copies":21754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10877,"num_whiteouts":0,"num_read":1873,"num_read_kb":502425,"num_write":25992,"num_write_kb":1570272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":105,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":668876,"num_omap_keys":1129,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36984,"ondisk_log_size":36984,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2232408,"kb_used_data":1780312,"kb_used_omap":12,"kb_used_meta":452019,"kb_avail":752742312,"statfs":{"total":773094113280,"available":770808127488,"internally_reserved":0,"allocated":1823039488,"data_stored":2284293060,"data_compressed":218592430,"data_compressed_allocated":249077760,"data_compressed_original":768491520,"omap_allocated":12750,"internal_metadata":462868018},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":69254,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":71155,"num_omap_keys":121,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003240"},"pg_stats":[{"pgid":"3.3f","version":"50'488","reported_seq":2624,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:01.699801+0000","last_change":"2024-05-27T09:38:01.699848+0000","last_active":"2024-05-27T09:38:01.699801+0000","last_peered":"2024-05-27T09:38:01.699801+0000","last_clean":"2024-05-27T09:38:01.699801+0000","last_became_active":"2024-05-27T09:27:18.585169+0000","last_became_peered":"2024-05-27T09:27:18.585169+0000","last_unstale":"2024-05-27T09:38:01.699801+0000","last_undegraded":"2024-05-27T09:38:01.699801+0000","last_fullsized":"2024-05-27T09:38:01.699801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'488","last_scrub_stamp":"2024-05-27T09:38:01.699702+0000","last_deep_scrub":"50'488","last_deep_scrub_stamp":"2024-05-27T09:38:01.699702+0000","last_clean_scrub_stamp":"2024-05-27T09:38:01.699702+0000","objects_scrubbed":160,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:07.450608+0000","scrub_duration":286,"objects_trimmed":0,"snaptrim_duration":0.01102325,"stat_sum":{"num_bytes":23698106,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":17,"num_read_kb":4111,"num_write":320,"num_write_kb":27403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"50'484","reported_seq":2619,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:02.813864+0000","last_change":"2024-05-27T09:38:02.813928+0000","last_active":"2024-05-27T09:38:02.813864+0000","last_peered":"2024-05-27T09:38:02.813864+0000","last_clean":"2024-05-27T09:38:02.813864+0000","last_became_active":"2024-05-27T09:27:18.566269+0000","last_became_peered":"2024-05-27T09:27:18.566269+0000","last_unstale":"2024-05-27T09:38:02.813864+0000","last_undegraded":"2024-05-27T09:38:02.813864+0000","last_fullsized":"2024-05-27T09:38:02.813864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'484","last_scrub_stamp":"2024-05-27T09:38:02.813745+0000","last_deep_scrub":"50'484","last_deep_scrub_stamp":"2024-05-27T09:38:02.813745+0000","last_clean_scrub_stamp":"2024-05-27T09:38:02.813745+0000","objects_scrubbed":167,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:10:07.654544+0000","scrub_duration":144,"objects_trimmed":0,"snaptrim_duration":0.0061074720000000001,"stat_sum":{"num_bytes":5460707,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":313,"num_write_kb":13693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"50'516","reported_seq":2659,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:55.905542+0000","last_change":"2024-05-27T09:37:55.905608+0000","last_active":"2024-05-27T09:37:55.905542+0000","last_peered":"2024-05-27T09:37:55.905542+0000","last_clean":"2024-05-27T09:37:55.905542+0000","last_became_active":"2024-05-27T09:27:18.560620+0000","last_became_peered":"2024-05-27T09:27:18.560620+0000","last_unstale":"2024-05-27T09:37:55.905542+0000","last_undegraded":"2024-05-27T09:37:55.905542+0000","last_fullsized":"2024-05-27T09:37:55.905542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'516","last_scrub_stamp":"2024-05-27T09:37:55.905450+0000","last_deep_scrub":"50'516","last_deep_scrub_stamp":"2024-05-27T09:37:55.905450+0000","last_clean_scrub_stamp":"2024-05-27T09:37:55.905450+0000","objects_scrubbed":167,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:05:42.995452+0000","scrub_duration":160,"objects_trimmed":0,"snaptrim_duration":0.005599354,"stat_sum":{"num_bytes":19266359,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":24,"num_read_kb":17715,"num_write":342,"num_write_kb":39437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"50'471","reported_seq":2600,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:51.537852+0000","last_change":"2024-05-27T09:37:51.537916+0000","last_active":"2024-05-27T09:37:51.537852+0000","last_peered":"2024-05-27T09:37:51.537852+0000","last_clean":"2024-05-27T09:37:51.537852+0000","last_became_active":"2024-05-27T09:27:18.569481+0000","last_became_peered":"2024-05-27T09:27:18.569481+0000","last_unstale":"2024-05-27T09:37:51.537852+0000","last_undegraded":"2024-05-27T09:37:51.537852+0000","last_fullsized":"2024-05-27T09:37:51.537852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'471","last_scrub_stamp":"2024-05-27T09:37:51.537779+0000","last_deep_scrub":"50'471","last_deep_scrub_stamp":"2024-05-27T09:37:51.537779+0000","last_clean_scrub_stamp":"2024-05-27T09:37:51.537779+0000","objects_scrubbed":164,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:11:32.761189+0000","scrub_duration":156,"objects_trimmed":0,"snaptrim_duration":0.007726882,"stat_sum":{"num_bytes":6546535,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":298,"num_write_kb":6559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"50'547","reported_seq":2683,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:56.866496+0000","last_change":"2024-05-27T09:37:56.866606+0000","last_active":"2024-05-27T09:37:56.866496+0000","last_peered":"2024-05-27T09:37:56.866496+0000","last_clean":"2024-05-27T09:37:56.866496+0000","last_became_active":"2024-05-27T09:27:18.567795+0000","last_became_peered":"2024-05-27T09:27:18.567795+0000","last_unstale":"2024-05-27T09:37:56.866496+0000","last_undegraded":"2024-05-27T09:37:56.866496+0000","last_fullsized":"2024-05-27T09:37:56.866496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'547","last_scrub_stamp":"2024-05-27T09:37:56.866258+0000","last_deep_scrub":"50'547","last_deep_scrub_stamp":"2024-05-27T09:37:56.866258+0000","last_clean_scrub_stamp":"2024-05-27T09:37:56.866258+0000","objects_scrubbed":172,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:52:44.900078+0000","scrub_duration":333,"objects_trimmed":0,"snaptrim_duration":0.0049759740000000002,"stat_sum":{"num_bytes":10629514,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":371,"num_write_kb":12093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"50'518","reported_seq":2659,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:52.623749+0000","last_change":"2024-05-27T09:37:52.623830+0000","last_active":"2024-05-27T09:37:52.623749+0000","last_peered":"2024-05-27T09:37:52.623749+0000","last_clean":"2024-05-27T09:37:52.623749+0000","last_became_active":"2024-05-27T09:27:18.573576+0000","last_became_peered":"2024-05-27T09:27:18.573576+0000","last_unstale":"2024-05-27T09:37:52.623749+0000","last_undegraded":"2024-05-27T09:37:52.623749+0000","last_fullsized":"2024-05-27T09:37:52.623749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'518","last_scrub_stamp":"2024-05-27T09:37:52.623620+0000","last_deep_scrub":"50'518","last_deep_scrub_stamp":"2024-05-27T09:37:52.623620+0000","last_clean_scrub_stamp":"2024-05-27T09:37:52.623620+0000","objects_scrubbed":168,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:46:05.384368+0000","scrub_duration":233,"objects_trimmed":0,"snaptrim_duration":0.0074740400000000004,"stat_sum":{"num_bytes":30579450,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":22,"num_read_kb":8211,"num_write":344,"num_write_kb":38238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"50'515","reported_seq":2652,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:52.743239+0000","last_change":"2024-05-27T09:37:52.743329+0000","last_active":"2024-05-27T09:37:52.743239+0000","last_peered":"2024-05-27T09:37:52.743239+0000","last_clean":"2024-05-27T09:37:52.743239+0000","last_became_active":"2024-05-27T09:27:18.542593+0000","last_became_peered":"2024-05-27T09:27:18.542593+0000","last_unstale":"2024-05-27T09:37:52.743239+0000","last_undegraded":"2024-05-27T09:37:52.743239+0000","last_fullsized":"2024-05-27T09:37:52.743239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'515","last_scrub_stamp":"2024-05-27T09:37:52.743103+0000","last_deep_scrub":"50'515","last_deep_scrub_stamp":"2024-05-27T09:37:52.743103+0000","last_clean_scrub_stamp":"2024-05-27T09:37:52.743103+0000","objects_scrubbed":155,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:20:34.804085+0000","scrub_duration":162,"objects_trimmed":0,"snaptrim_duration":0.014070156,"stat_sum":{"num_bytes":5299988,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":18,"num_read_kb":4113,"num_write":355,"num_write_kb":9461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"50'520","reported_seq":2652,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:00.065500+0000","last_change":"2024-05-27T09:38:00.065566+0000","last_active":"2024-05-27T09:38:00.065500+0000","last_peered":"2024-05-27T09:38:00.065500+0000","last_clean":"2024-05-27T09:38:00.065500+0000","last_became_active":"2024-05-27T09:27:18.561788+0000","last_became_peered":"2024-05-27T09:27:18.561788+0000","last_unstale":"2024-05-27T09:38:00.065500+0000","last_undegraded":"2024-05-27T09:38:00.065500+0000","last_fullsized":"2024-05-27T09:38:00.065500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'520","last_scrub_stamp":"2024-05-27T09:38:00.065383+0000","last_deep_scrub":"50'520","last_deep_scrub_stamp":"2024-05-27T09:38:00.065383+0000","last_clean_scrub_stamp":"2024-05-27T09:38:00.065383+0000","objects_scrubbed":191,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:18:00.020736+0000","scrub_duration":331,"objects_trimmed":0,"snaptrim_duration":0.0065132080000000004,"stat_sum":{"num_bytes":27492830,"num_objects":191,"num_object_clones":0,"num_object_copies":382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":191,"num_whiteouts":0,"num_read":13,"num_read_kb":4107,"num_write":319,"num_write_kb":27008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"50'509","reported_seq":2649,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:53.285274+0000","last_change":"2024-05-27T09:37:53.285328+0000","last_active":"2024-05-27T09:37:53.285274+0000","last_peered":"2024-05-27T09:37:53.285274+0000","last_clean":"2024-05-27T09:37:53.285274+0000","last_became_active":"2024-05-27T09:27:18.582041+0000","last_became_peered":"2024-05-27T09:27:18.582041+0000","last_unstale":"2024-05-27T09:37:53.285274+0000","last_undegraded":"2024-05-27T09:37:53.285274+0000","last_fullsized":"2024-05-27T09:37:53.285274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'509","last_scrub_stamp":"2024-05-27T09:37:53.285180+0000","last_deep_scrub":"50'509","last_deep_scrub_stamp":"2024-05-27T09:37:53.285180+0000","last_clean_scrub_stamp":"2024-05-27T09:37:53.285180+0000","objects_scrubbed":171,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:11:01.550283+0000","scrub_duration":310,"objects_trimmed":0,"snaptrim_duration":0.0083382379999999996,"stat_sum":{"num_bytes":9733898,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":22,"num_read_kb":8210,"num_write":328,"num_write_kb":30110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"50'457","reported_seq":2591,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:58.726744+0000","last_change":"2024-05-27T09:37:58.726822+0000","last_active":"2024-05-27T09:37:58.726744+0000","last_peered":"2024-05-27T09:37:58.726744+0000","last_clean":"2024-05-27T09:37:58.726744+0000","last_became_active":"2024-05-27T09:27:18.574409+0000","last_became_peered":"2024-05-27T09:27:18.574409+0000","last_unstale":"2024-05-27T09:37:58.726744+0000","last_undegraded":"2024-05-27T09:37:58.726744+0000","last_fullsized":"2024-05-27T09:37:58.726744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'457","last_scrub_stamp":"2024-05-27T09:37:58.726635+0000","last_deep_scrub":"50'457","last_deep_scrub_stamp":"2024-05-27T09:37:58.726635+0000","last_clean_scrub_stamp":"2024-05-27T09:37:58.726635+0000","objects_scrubbed":136,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:12:15.968770+0000","scrub_duration":252,"objects_trimmed":0,"snaptrim_duration":0.010493662000000001,"stat_sum":{"num_bytes":13833965,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":17,"num_read_kb":4112,"num_write":306,"num_write_kb":13622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"50'461","reported_seq":2600,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:00.673296+0000","last_change":"2024-05-27T09:38:00.673446+0000","last_active":"2024-05-27T09:38:00.673296+0000","last_peered":"2024-05-27T09:38:00.673296+0000","last_clean":"2024-05-27T09:38:00.673296+0000","last_became_active":"2024-05-27T09:27:18.578679+0000","last_became_peered":"2024-05-27T09:27:18.578679+0000","last_unstale":"2024-05-27T09:38:00.673296+0000","last_undegraded":"2024-05-27T09:38:00.673296+0000","last_fullsized":"2024-05-27T09:38:00.673296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'461","last_scrub_stamp":"2024-05-27T09:38:00.673136+0000","last_deep_scrub":"50'461","last_deep_scrub_stamp":"2024-05-27T09:38:00.673136+0000","last_clean_scrub_stamp":"2024-05-27T09:38:00.673136+0000","objects_scrubbed":142,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:27.697755+0000","scrub_duration":218,"objects_trimmed":0,"snaptrim_duration":0.0080365679999999991,"stat_sum":{"num_bytes":26002241,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":22,"num_read_kb":7043,"num_write":312,"num_write_kb":25555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"50'452","reported_seq":2576,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:58.423387+0000","last_change":"2024-05-27T09:37:58.423436+0000","last_active":"2024-05-27T09:37:58.423387+0000","last_peered":"2024-05-27T09:37:58.423387+0000","last_clean":"2024-05-27T09:37:58.423387+0000","last_became_active":"2024-05-27T09:28:06.207562+0000","last_became_peered":"2024-05-27T09:28:06.207562+0000","last_unstale":"2024-05-27T09:37:58.423387+0000","last_undegraded":"2024-05-27T09:37:58.423387+0000","last_fullsized":"2024-05-27T09:37:58.423387+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'452","last_scrub_stamp":"2024-05-27T09:37:58.423263+0000","last_deep_scrub":"50'452","last_deep_scrub_stamp":"2024-05-27T09:37:58.423263+0000","last_clean_scrub_stamp":"2024-05-27T09:37:58.423263+0000","objects_scrubbed":151,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:20:18.138295+0000","scrub_duration":196,"objects_trimmed":0,"snaptrim_duration":0.001018333,"stat_sum":{"num_bytes":11019660,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":12,"num_read_kb":8201,"num_write":293,"num_write_kb":23198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"50'457","reported_seq":2595,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:56.392221+0000","last_change":"2024-05-27T09:37:56.392266+0000","last_active":"2024-05-27T09:37:56.392221+0000","last_peered":"2024-05-27T09:37:56.392221+0000","last_clean":"2024-05-27T09:37:56.392221+0000","last_became_active":"2024-05-27T09:27:18.561493+0000","last_became_peered":"2024-05-27T09:27:18.561493+0000","last_unstale":"2024-05-27T09:37:56.392221+0000","last_undegraded":"2024-05-27T09:37:56.392221+0000","last_fullsized":"2024-05-27T09:37:56.392221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'457","last_scrub_stamp":"2024-05-27T09:37:56.392146+0000","last_deep_scrub":"50'457","last_deep_scrub_stamp":"2024-05-27T09:37:56.392146+0000","last_clean_scrub_stamp":"2024-05-27T09:37:56.392146+0000","objects_scrubbed":149,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:49:29.868426+0000","scrub_duration":253,"objects_trimmed":0,"snaptrim_duration":0.0024650470000000002,"stat_sum":{"num_bytes":14891482,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":21,"num_read_kb":4116,"num_write":298,"num_write_kb":31046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"50'506","reported_seq":2642,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:57.296338+0000","last_change":"2024-05-27T09:37:57.296402+0000","last_active":"2024-05-27T09:37:57.296338+0000","last_peered":"2024-05-27T09:37:57.296338+0000","last_clean":"2024-05-27T09:37:57.296338+0000","last_became_active":"2024-05-27T09:27:18.562843+0000","last_became_peered":"2024-05-27T09:27:18.562843+0000","last_unstale":"2024-05-27T09:37:57.296338+0000","last_undegraded":"2024-05-27T09:37:57.296338+0000","last_fullsized":"2024-05-27T09:37:57.296338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'506","last_scrub_stamp":"2024-05-27T09:37:57.296241+0000","last_deep_scrub":"50'506","last_deep_scrub_stamp":"2024-05-27T09:37:57.296241+0000","last_clean_scrub_stamp":"2024-05-27T09:37:57.296241+0000","objects_scrubbed":164,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:27:39.067620+0000","scrub_duration":150,"objects_trimmed":0,"snaptrim_duration":0.0025802960000000002,"stat_sum":{"num_bytes":6434369,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":335,"num_write_kb":10518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"50'565","reported_seq":2707,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:01.996190+0000","last_change":"2024-05-27T09:38:01.996217+0000","last_active":"2024-05-27T09:38:01.996190+0000","last_peered":"2024-05-27T09:38:01.996190+0000","last_clean":"2024-05-27T09:38:01.996190+0000","last_became_active":"2024-05-27T09:27:18.566108+0000","last_became_peered":"2024-05-27T09:27:18.566108+0000","last_unstale":"2024-05-27T09:38:01.996190+0000","last_undegraded":"2024-05-27T09:38:01.996190+0000","last_fullsized":"2024-05-27T09:38:01.996190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'565","last_scrub_stamp":"2024-05-27T09:38:01.996112+0000","last_deep_scrub":"50'565","last_deep_scrub_stamp":"2024-05-27T09:38:01.996112+0000","last_clean_scrub_stamp":"2024-05-27T09:38:01.996112+0000","objects_scrubbed":201,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:37:59.910683+0000","scrub_duration":321,"objects_trimmed":0,"snaptrim_duration":0.0041189030000000001,"stat_sum":{"num_bytes":20283045,"num_objects":201,"num_object_clones":0,"num_object_copies":402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":201,"num_whiteouts":0,"num_read":22,"num_read_kb":6627,"num_write":357,"num_write_kb":19995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"50'548","reported_seq":2688,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:06.685771+0000","last_change":"2024-05-27T09:38:06.685813+0000","last_active":"2024-05-27T09:38:06.685771+0000","last_peered":"2024-05-27T09:38:06.685771+0000","last_clean":"2024-05-27T09:38:06.685771+0000","last_became_active":"2024-05-27T09:27:18.581719+0000","last_became_peered":"2024-05-27T09:27:18.581719+0000","last_unstale":"2024-05-27T09:38:06.685771+0000","last_undegraded":"2024-05-27T09:38:06.685771+0000","last_fullsized":"2024-05-27T09:38:06.685771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'548","last_scrub_stamp":"2024-05-27T09:38:06.685654+0000","last_deep_scrub":"50'548","last_deep_scrub_stamp":"2024-05-27T09:38:06.685654+0000","last_clean_scrub_stamp":"2024-05-27T09:38:06.685654+0000","objects_scrubbed":179,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:32:21.505138+0000","scrub_duration":369,"objects_trimmed":0,"snaptrim_duration":0.008579415,"stat_sum":{"num_bytes":24362280,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":22,"num_read_kb":5287,"num_write":359,"num_write_kb":31349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"50'549","reported_seq":2690,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:04.669911+0000","last_change":"2024-05-27T09:38:04.669983+0000","last_active":"2024-05-27T09:38:04.669911+0000","last_peered":"2024-05-27T09:38:04.669911+0000","last_clean":"2024-05-27T09:38:04.669911+0000","last_became_active":"2024-05-27T09:27:18.577193+0000","last_became_peered":"2024-05-27T09:27:18.577193+0000","last_unstale":"2024-05-27T09:38:04.669911+0000","last_undegraded":"2024-05-27T09:38:04.669911+0000","last_fullsized":"2024-05-27T09:38:04.669911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'549","last_scrub_stamp":"2024-05-27T09:38:04.669761+0000","last_deep_scrub":"50'549","last_deep_scrub_stamp":"2024-05-27T09:38:04.669761+0000","last_clean_scrub_stamp":"2024-05-27T09:38:04.669761+0000","objects_scrubbed":183,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:53:23.954379+0000","scrub_duration":388,"objects_trimmed":0,"snaptrim_duration":0.0079143630000000006,"stat_sum":{"num_bytes":23132412,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":23,"num_read_kb":8093,"num_write":357,"num_write_kb":30967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"50'445","reported_seq":2590,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:57.438601+0000","last_change":"2024-05-27T09:37:57.438669+0000","last_active":"2024-05-27T09:37:57.438601+0000","last_peered":"2024-05-27T09:37:57.438601+0000","last_clean":"2024-05-27T09:37:57.438601+0000","last_became_active":"2024-05-27T09:27:18.570712+0000","last_became_peered":"2024-05-27T09:27:18.570712+0000","last_unstale":"2024-05-27T09:37:57.438601+0000","last_undegraded":"2024-05-27T09:37:57.438601+0000","last_fullsized":"2024-05-27T09:37:57.438601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'445","last_scrub_stamp":"2024-05-27T09:37:57.438472+0000","last_deep_scrub":"50'445","last_deep_scrub_stamp":"2024-05-27T09:37:57.438472+0000","last_clean_scrub_stamp":"2024-05-27T09:37:57.438472+0000","objects_scrubbed":138,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:24:15.292236+0000","scrub_duration":225,"objects_trimmed":0,"snaptrim_duration":0.0011767629999999999,"stat_sum":{"num_bytes":13973992,"num_objects":138,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":0,"num_read":28,"num_read_kb":8218,"num_write":296,"num_write_kb":21985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"50'478","reported_seq":2613,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:52.481425+0000","last_change":"2024-05-27T09:37:52.481487+0000","last_active":"2024-05-27T09:37:52.481425+0000","last_peered":"2024-05-27T09:37:52.481425+0000","last_clean":"2024-05-27T09:37:52.481425+0000","last_became_active":"2024-05-27T09:27:18.564643+0000","last_became_peered":"2024-05-27T09:27:18.564643+0000","last_unstale":"2024-05-27T09:37:52.481425+0000","last_undegraded":"2024-05-27T09:37:52.481425+0000","last_fullsized":"2024-05-27T09:37:52.481425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'478","last_scrub_stamp":"2024-05-27T09:37:52.481262+0000","last_deep_scrub":"50'478","last_deep_scrub_stamp":"2024-05-27T09:37:52.481262+0000","last_clean_scrub_stamp":"2024-05-27T09:37:52.481262+0000","objects_scrubbed":155,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:19:54.271204+0000","scrub_duration":227,"objects_trimmed":0,"snaptrim_duration":0.0091687629999999999,"stat_sum":{"num_bytes":2873482,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":18,"num_read_kb":4113,"num_write":315,"num_write_kb":11167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"50'498","reported_seq":2637,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:53.733669+0000","last_change":"2024-05-27T09:37:53.733721+0000","last_active":"2024-05-27T09:37:53.733669+0000","last_peered":"2024-05-27T09:37:53.733669+0000","last_clean":"2024-05-27T09:37:53.733669+0000","last_became_active":"2024-05-27T09:27:18.543777+0000","last_became_peered":"2024-05-27T09:27:18.543777+0000","last_unstale":"2024-05-27T09:37:53.733669+0000","last_undegraded":"2024-05-27T09:37:53.733669+0000","last_fullsized":"2024-05-27T09:37:53.733669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'498","last_scrub_stamp":"2024-05-27T09:37:53.733563+0000","last_deep_scrub":"50'498","last_deep_scrub_stamp":"2024-05-27T09:37:53.733563+0000","last_clean_scrub_stamp":"2024-05-27T09:37:53.733563+0000","objects_scrubbed":153,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:51:40.711375+0000","scrub_duration":182,"objects_trimmed":0,"snaptrim_duration":0.012534072,"stat_sum":{"num_bytes":9859903,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":21,"num_read_kb":4116,"num_write":337,"num_write_kb":17999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"50'506","reported_seq":2646,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:00.565877+0000","last_change":"2024-05-27T09:38:00.565914+0000","last_active":"2024-05-27T09:38:00.565877+0000","last_peered":"2024-05-27T09:38:00.565877+0000","last_clean":"2024-05-27T09:38:00.565877+0000","last_became_active":"2024-05-27T09:27:18.570948+0000","last_became_peered":"2024-05-27T09:27:18.570948+0000","last_unstale":"2024-05-27T09:38:00.565877+0000","last_undegraded":"2024-05-27T09:38:00.565877+0000","last_fullsized":"2024-05-27T09:38:00.565877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'506","last_scrub_stamp":"2024-05-27T09:38:00.565799+0000","last_deep_scrub":"50'506","last_deep_scrub_stamp":"2024-05-27T09:38:00.565799+0000","last_clean_scrub_stamp":"2024-05-27T09:38:00.565799+0000","objects_scrubbed":169,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:17:27.682172+0000","scrub_duration":214,"objects_trimmed":0,"snaptrim_duration":0.0040840829999999996,"stat_sum":{"num_bytes":5923849,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":325,"num_write_kb":5958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"50'482","reported_seq":2620,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:49.185520+0000","last_change":"2024-05-27T09:37:49.185585+0000","last_active":"2024-05-27T09:37:49.185520+0000","last_peered":"2024-05-27T09:37:49.185520+0000","last_clean":"2024-05-27T09:37:49.185520+0000","last_became_active":"2024-05-27T09:27:18.543921+0000","last_became_peered":"2024-05-27T09:27:18.543921+0000","last_unstale":"2024-05-27T09:37:49.185520+0000","last_undegraded":"2024-05-27T09:37:49.185520+0000","last_fullsized":"2024-05-27T09:37:49.185520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'482","last_scrub_stamp":"2024-05-27T09:37:49.185414+0000","last_deep_scrub":"50'482","last_deep_scrub_stamp":"2024-05-27T09:37:49.185414+0000","last_clean_scrub_stamp":"2024-05-27T09:37:49.185414+0000","objects_scrubbed":160,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:36:04.794414+0000","scrub_duration":227,"objects_trimmed":0,"snaptrim_duration":0.0094259180000000001,"stat_sum":{"num_bytes":14450069,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":20,"num_read_kb":8209,"num_write":332,"num_write_kb":18385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"50'488","reported_seq":2637,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:03.940729+0000","last_change":"2024-05-27T09:38:03.940757+0000","last_active":"2024-05-27T09:38:03.940729+0000","last_peered":"2024-05-27T09:38:03.940729+0000","last_clean":"2024-05-27T09:38:03.940729+0000","last_became_active":"2024-05-27T09:27:18.575115+0000","last_became_peered":"2024-05-27T09:27:18.575115+0000","last_unstale":"2024-05-27T09:38:03.940729+0000","last_undegraded":"2024-05-27T09:38:03.940729+0000","last_fullsized":"2024-05-27T09:38:03.940729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'488","last_scrub_stamp":"2024-05-27T09:38:03.940652+0000","last_deep_scrub":"50'488","last_deep_scrub_stamp":"2024-05-27T09:38:03.940652+0000","last_clean_scrub_stamp":"2024-05-27T09:38:03.940652+0000","objects_scrubbed":151,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:55:48.051463+0000","scrub_duration":254,"objects_trimmed":0,"snaptrim_duration":0.0054063330000000001,"stat_sum":{"num_bytes":18656408,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":31,"num_read_kb":19450,"num_write":329,"num_write_kb":38861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"50'521","reported_seq":2658,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:50.396032+0000","last_change":"2024-05-27T09:37:50.396032+0000","last_active":"2024-05-27T09:37:50.396032+0000","last_peered":"2024-05-27T09:37:50.396032+0000","last_clean":"2024-05-27T09:37:50.396032+0000","last_became_active":"2024-05-27T09:27:18.563991+0000","last_became_peered":"2024-05-27T09:27:18.563991+0000","last_unstale":"2024-05-27T09:37:50.396032+0000","last_undegraded":"2024-05-27T09:37:50.396032+0000","last_fullsized":"2024-05-27T09:37:50.396032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'521","last_scrub_stamp":"2024-05-27T09:37:50.395803+0000","last_deep_scrub":"50'521","last_deep_scrub_stamp":"2024-05-27T09:37:50.395803+0000","last_clean_scrub_stamp":"2024-05-27T09:37:50.395803+0000","objects_scrubbed":173,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:26:56.146709+0000","scrub_duration":185,"objects_trimmed":0,"snaptrim_duration":0.009841753,"stat_sum":{"num_bytes":13963475,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":19,"num_read_kb":8208,"num_write":347,"num_write_kb":22013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"50'496","reported_seq":2636,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:58.958566+0000","last_change":"2024-05-27T09:37:58.958566+0000","last_active":"2024-05-27T09:37:58.958566+0000","last_peered":"2024-05-27T09:37:58.958566+0000","last_clean":"2024-05-27T09:37:58.958566+0000","last_became_active":"2024-05-27T09:27:18.562803+0000","last_became_peered":"2024-05-27T09:27:18.562803+0000","last_unstale":"2024-05-27T09:37:58.958566+0000","last_undegraded":"2024-05-27T09:37:58.958566+0000","last_fullsized":"2024-05-27T09:37:58.958566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'496","last_scrub_stamp":"2024-05-27T09:37:58.958449+0000","last_deep_scrub":"50'496","last_deep_scrub_stamp":"2024-05-27T09:37:58.958449+0000","last_clean_scrub_stamp":"2024-05-27T09:37:58.958449+0000","objects_scrubbed":164,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:31:49.418958+0000","scrub_duration":251,"objects_trimmed":0,"snaptrim_duration":0.0067443290000000003,"stat_sum":{"num_bytes":23071782,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":21,"num_read_kb":4115,"num_write":328,"num_write_kb":30886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"50'565","reported_seq":2705,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:55.662069+0000","last_change":"2024-05-27T09:37:55.662069+0000","last_active":"2024-05-27T09:37:55.662069+0000","last_peered":"2024-05-27T09:37:55.662069+0000","last_clean":"2024-05-27T09:37:55.662069+0000","last_became_active":"2024-05-27T09:27:18.590083+0000","last_became_peered":"2024-05-27T09:27:18.590083+0000","last_unstale":"2024-05-27T09:37:55.662069+0000","last_undegraded":"2024-05-27T09:37:55.662069+0000","last_fullsized":"2024-05-27T09:37:55.662069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'565","last_scrub_stamp":"2024-05-27T09:37:55.661898+0000","last_deep_scrub":"50'565","last_deep_scrub_stamp":"2024-05-27T09:37:55.661898+0000","last_clean_scrub_stamp":"2024-05-27T09:37:55.661898+0000","objects_scrubbed":181,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:01:24.136152+0000","scrub_duration":251,"objects_trimmed":0,"snaptrim_duration":0.008181008,"stat_sum":{"num_bytes":14547405,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":20,"num_read_kb":4115,"num_write":369,"num_write_kb":14397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"50'519","reported_seq":2655,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:55.344648+0000","last_change":"2024-05-27T09:37:55.344648+0000","last_active":"2024-05-27T09:37:55.344648+0000","last_peered":"2024-05-27T09:37:55.344648+0000","last_clean":"2024-05-27T09:37:55.344648+0000","last_became_active":"2024-05-27T09:27:18.557953+0000","last_became_peered":"2024-05-27T09:27:18.557953+0000","last_unstale":"2024-05-27T09:37:55.344648+0000","last_undegraded":"2024-05-27T09:37:55.344648+0000","last_fullsized":"2024-05-27T09:37:55.344648+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'519","last_scrub_stamp":"2024-05-27T09:37:55.344536+0000","last_deep_scrub":"50'519","last_deep_scrub_stamp":"2024-05-27T09:37:55.344536+0000","last_clean_scrub_stamp":"2024-05-27T09:37:55.344536+0000","objects_scrubbed":187,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:49:19.412646+0000","scrub_duration":194,"objects_trimmed":0,"snaptrim_duration":0.0027117650000000001,"stat_sum":{"num_bytes":18627167,"num_objects":187,"num_object_clones":0,"num_object_copies":374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":187,"num_whiteouts":0,"num_read":16,"num_read_kb":4110,"num_write":325,"num_write_kb":26562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"50'456","reported_seq":2591,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:58.392534+0000","last_change":"2024-05-27T09:37:58.392534+0000","last_active":"2024-05-27T09:37:58.392534+0000","last_peered":"2024-05-27T09:37:58.392534+0000","last_clean":"2024-05-27T09:37:58.392534+0000","last_became_active":"2024-05-27T09:27:18.562839+0000","last_became_peered":"2024-05-27T09:27:18.562839+0000","last_unstale":"2024-05-27T09:37:58.392534+0000","last_undegraded":"2024-05-27T09:37:58.392534+0000","last_fullsized":"2024-05-27T09:37:58.392534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'456","last_scrub_stamp":"2024-05-27T09:37:58.392324+0000","last_deep_scrub":"50'456","last_deep_scrub_stamp":"2024-05-27T09:37:58.392324+0000","last_clean_scrub_stamp":"2024-05-27T09:37:58.392324+0000","objects_scrubbed":149,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:02:27.205655+0000","scrub_duration":217,"objects_trimmed":0,"snaptrim_duration":0.0032647399999999999,"stat_sum":{"num_bytes":14533754,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":17,"num_read_kb":11479,"num_write":303,"num_write_kb":21724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"50'506","reported_seq":2645,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:56.871146+0000","last_change":"2024-05-27T09:37:56.871180+0000","last_active":"2024-05-27T09:37:56.871146+0000","last_peered":"2024-05-27T09:37:56.871146+0000","last_clean":"2024-05-27T09:37:56.871146+0000","last_became_active":"2024-05-27T09:27:18.561475+0000","last_became_peered":"2024-05-27T09:27:18.561475+0000","last_unstale":"2024-05-27T09:37:56.871146+0000","last_undegraded":"2024-05-27T09:37:56.871146+0000","last_fullsized":"2024-05-27T09:37:56.871146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'506","last_scrub_stamp":"2024-05-27T09:37:56.871039+0000","last_deep_scrub":"50'506","last_deep_scrub_stamp":"2024-05-27T09:37:56.871039+0000","last_clean_scrub_stamp":"2024-05-27T09:37:56.871039+0000","objects_scrubbed":155,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:04:48.043012+0000","scrub_duration":154,"objects_trimmed":0,"snaptrim_duration":0.0053708779999999999,"stat_sum":{"num_bytes":11382190,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":20,"num_read_kb":4114,"num_write":347,"num_write_kb":19366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"50'530","reported_seq":2676,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:49.523008+0000","last_change":"2024-05-27T09:37:49.523096+0000","last_active":"2024-05-27T09:37:49.523008+0000","last_peered":"2024-05-27T09:37:49.523008+0000","last_clean":"2024-05-27T09:37:49.523008+0000","last_became_active":"2024-05-27T09:27:18.563103+0000","last_became_peered":"2024-05-27T09:27:18.563103+0000","last_unstale":"2024-05-27T09:37:49.523008+0000","last_undegraded":"2024-05-27T09:37:49.523008+0000","last_fullsized":"2024-05-27T09:37:49.523008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'530","last_scrub_stamp":"2024-05-27T09:37:49.522897+0000","last_deep_scrub":"50'530","last_deep_scrub_stamp":"2024-05-27T09:37:49.522897+0000","last_clean_scrub_stamp":"2024-05-27T09:37:49.522897+0000","objects_scrubbed":171,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:16.547516+0000","scrub_duration":295,"objects_trimmed":0,"snaptrim_duration":0.011066041,"stat_sum":{"num_bytes":30187091,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":28,"num_read_kb":16405,"num_write":369,"num_write_kb":50152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"50'587","reported_seq":2726,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:55.490281+0000","last_change":"2024-05-27T09:37:55.490348+0000","last_active":"2024-05-27T09:37:55.490281+0000","last_peered":"2024-05-27T09:37:55.490281+0000","last_clean":"2024-05-27T09:37:55.490281+0000","last_became_active":"2024-05-27T09:27:18.560244+0000","last_became_peered":"2024-05-27T09:27:18.560244+0000","last_unstale":"2024-05-27T09:37:55.490281+0000","last_undegraded":"2024-05-27T09:37:55.490281+0000","last_fullsized":"2024-05-27T09:37:55.490281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'587","last_scrub_stamp":"2024-05-27T09:37:55.490178+0000","last_deep_scrub":"50'587","last_deep_scrub_stamp":"2024-05-27T09:37:55.490178+0000","last_clean_scrub_stamp":"2024-05-27T09:37:55.490178+0000","objects_scrubbed":186,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:40:25.007945+0000","scrub_duration":272,"objects_trimmed":0,"snaptrim_duration":0.0012131959999999999,"stat_sum":{"num_bytes":8541274,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":19,"num_read_kb":2631,"num_write":389,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"50'519","reported_seq":2665,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:50.514418+0000","last_change":"2024-05-27T09:37:50.514456+0000","last_active":"2024-05-27T09:37:50.514418+0000","last_peered":"2024-05-27T09:37:50.514418+0000","last_clean":"2024-05-27T09:37:50.514418+0000","last_became_active":"2024-05-27T09:27:18.567845+0000","last_became_peered":"2024-05-27T09:27:18.567845+0000","last_unstale":"2024-05-27T09:37:50.514418+0000","last_undegraded":"2024-05-27T09:37:50.514418+0000","last_fullsized":"2024-05-27T09:37:50.514418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'519","last_scrub_stamp":"2024-05-27T09:37:50.514315+0000","last_deep_scrub":"50'519","last_deep_scrub_stamp":"2024-05-27T09:37:50.514315+0000","last_clean_scrub_stamp":"2024-05-27T09:37:50.514315+0000","objects_scrubbed":156,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:18:08.822242+0000","scrub_duration":164,"objects_trimmed":0,"snaptrim_duration":0.0082936180000000009,"stat_sum":{"num_bytes":1444930,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":352,"num_write_kb":9783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"50'507","reported_seq":2644,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:52.180847+0000","last_change":"2024-05-27T09:37:52.180922+0000","last_active":"2024-05-27T09:37:52.180847+0000","last_peered":"2024-05-27T09:37:52.180847+0000","last_clean":"2024-05-27T09:37:52.180847+0000","last_became_active":"2024-05-27T09:27:18.556156+0000","last_became_peered":"2024-05-27T09:27:18.556156+0000","last_unstale":"2024-05-27T09:37:52.180847+0000","last_undegraded":"2024-05-27T09:37:52.180847+0000","last_fullsized":"2024-05-27T09:37:52.180847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'507","last_scrub_stamp":"2024-05-27T09:37:52.180732+0000","last_deep_scrub":"50'507","last_deep_scrub_stamp":"2024-05-27T09:37:52.180732+0000","last_clean_scrub_stamp":"2024-05-27T09:37:52.180732+0000","objects_scrubbed":171,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:52:12.633393+0000","scrub_duration":229,"objects_trimmed":0,"snaptrim_duration":0.0094378269999999993,"stat_sum":{"num_bytes":19241078,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":19,"num_read_kb":8208,"num_write":329,"num_write_kb":23062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"50'78","reported_seq":170,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:50.858015+0000","last_change":"2024-05-27T09:37:50.858047+0000","last_active":"2024-05-27T09:37:50.858015+0000","last_peered":"2024-05-27T09:37:50.858015+0000","last_clean":"2024-05-27T09:37:50.858015+0000","last_became_active":"2024-05-27T09:27:17.547653+0000","last_became_peered":"2024-05-27T09:27:17.547653+0000","last_unstale":"2024-05-27T09:37:50.858015+0000","last_undegraded":"2024-05-27T09:37:50.858015+0000","last_fullsized":"2024-05-27T09:37:50.858015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'78","last_scrub_stamp":"2024-05-27T09:37:50.857933+0000","last_deep_scrub":"50'78","last_deep_scrub_stamp":"2024-05-27T09:37:50.857933+0000","last_clean_scrub_stamp":"2024-05-27T09:37:50.857933+0000","objects_scrubbed":5,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:52:50.740163+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195553,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":4100,"num_write":72,"num_write_kb":4221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15497,"num_omap_keys":25,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"50'481","reported_seq":2617,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:55.600956+0000","last_change":"2024-05-27T09:37:55.601008+0000","last_active":"2024-05-27T09:37:55.600956+0000","last_peered":"2024-05-27T09:37:55.600956+0000","last_clean":"2024-05-27T09:37:55.600956+0000","last_became_active":"2024-05-27T09:27:18.542715+0000","last_became_peered":"2024-05-27T09:27:18.542715+0000","last_unstale":"2024-05-27T09:37:55.600956+0000","last_undegraded":"2024-05-27T09:37:55.600956+0000","last_fullsized":"2024-05-27T09:37:55.600956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'481","last_scrub_stamp":"2024-05-27T09:37:55.600873+0000","last_deep_scrub":"50'481","last_deep_scrub_stamp":"2024-05-27T09:37:55.600873+0000","last_clean_scrub_stamp":"2024-05-27T09:37:55.600873+0000","objects_scrubbed":153,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:19:08.850742+0000","scrub_duration":174,"objects_trimmed":0,"snaptrim_duration":0.004594323,"stat_sum":{"num_bytes":10080804,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":19,"num_read_kb":4113,"num_write":319,"num_write_kb":14102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"50'306","reported_seq":400,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:53.265123+0000","last_change":"2024-05-27T09:37:53.265184+0000","last_active":"2024-05-27T09:37:53.265123+0000","last_peered":"2024-05-27T09:37:53.265123+0000","last_clean":"2024-05-27T09:37:53.265123+0000","last_became_active":"2024-05-27T09:27:17.569673+0000","last_became_peered":"2024-05-27T09:27:17.569673+0000","last_unstale":"2024-05-27T09:37:53.265123+0000","last_undegraded":"2024-05-27T09:37:53.265123+0000","last_fullsized":"2024-05-27T09:37:53.265123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'306","last_scrub_stamp":"2024-05-27T09:37:53.265022+0000","last_deep_scrub":"50'306","last_deep_scrub_stamp":"2024-05-27T09:37:53.265022+0000","last_clean_scrub_stamp":"2024-05-27T09:37:53.265022+0000","objects_scrubbed":7,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:07:41.963309+0000","scrub_duration":53,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":22,"num_read_kb":12200,"num_write":300,"num_write_kb":12509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":38298,"num_omap_keys":64,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"50'53","reported_seq":145,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:53.124082+0000","last_change":"2024-05-27T09:37:53.124116+0000","last_active":"2024-05-27T09:37:53.124082+0000","last_peered":"2024-05-27T09:37:53.124082+0000","last_clean":"2024-05-27T09:37:53.124082+0000","last_became_active":"2024-05-27T09:27:17.565137+0000","last_became_peered":"2024-05-27T09:27:17.565137+0000","last_unstale":"2024-05-27T09:37:53.124082+0000","last_undegraded":"2024-05-27T09:37:53.124082+0000","last_fullsized":"2024-05-27T09:37:53.124082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'53","last_scrub_stamp":"2024-05-27T09:37:53.124011+0000","last_deep_scrub":"50'53","last_deep_scrub_stamp":"2024-05-27T09:37:53.124011+0000","last_clean_scrub_stamp":"2024-05-27T09:37:53.124011+0000","objects_scrubbed":7,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:02.975127+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":19,"num_read_kb":4114,"num_write":50,"num_write_kb":4167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":44697,"num_omap_keys":75,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"50'578","reported_seq":2720,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:00.863229+0000","last_change":"2024-05-27T09:38:00.863262+0000","last_active":"2024-05-27T09:38:00.863229+0000","last_peered":"2024-05-27T09:38:00.863229+0000","last_clean":"2024-05-27T09:38:00.863229+0000","last_became_active":"2024-05-27T09:27:18.564115+0000","last_became_peered":"2024-05-27T09:27:18.564115+0000","last_unstale":"2024-05-27T09:38:00.863229+0000","last_undegraded":"2024-05-27T09:38:00.863229+0000","last_fullsized":"2024-05-27T09:38:00.863229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'578","last_scrub_stamp":"2024-05-27T09:38:00.863153+0000","last_deep_scrub":"50'578","last_deep_scrub_stamp":"2024-05-27T09:38:00.863153+0000","last_clean_scrub_stamp":"2024-05-27T09:38:00.863153+0000","objects_scrubbed":181,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:27:40.172947+0000","scrub_duration":156,"objects_trimmed":0,"snaptrim_duration":0.0066511199999999999,"stat_sum":{"num_bytes":10503109,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":23,"num_read_kb":4118,"num_write":389,"num_write_kb":18659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"50'300","reported_seq":386,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:55.331103+0000","last_change":"2024-05-27T09:37:55.331214+0000","last_active":"2024-05-27T09:37:55.331103+0000","last_peered":"2024-05-27T09:37:55.331103+0000","last_clean":"2024-05-27T09:37:55.331103+0000","last_became_active":"2024-05-27T09:27:17.560149+0000","last_became_peered":"2024-05-27T09:27:17.560149+0000","last_unstale":"2024-05-27T09:37:55.331103+0000","last_undegraded":"2024-05-27T09:37:55.331103+0000","last_fullsized":"2024-05-27T09:37:55.331103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'300","last_scrub_stamp":"2024-05-27T09:37:55.330832+0000","last_deep_scrub":"50'300","last_deep_scrub_stamp":"2024-05-27T09:37:55.330832+0000","last_clean_scrub_stamp":"2024-05-27T09:37:55.330832+0000","objects_scrubbed":6,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:35:23.852140+0000","scrub_duration":80,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":291,"num_write_kb":12506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40124,"num_omap_keys":68,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"50'514","reported_seq":2651,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:54.679622+0000","last_change":"2024-05-27T09:37:54.679680+0000","last_active":"2024-05-27T09:37:54.679622+0000","last_peered":"2024-05-27T09:37:54.679622+0000","last_clean":"2024-05-27T09:37:54.679622+0000","last_became_active":"2024-05-27T09:27:18.541946+0000","last_became_peered":"2024-05-27T09:27:18.541946+0000","last_unstale":"2024-05-27T09:37:54.679622+0000","last_undegraded":"2024-05-27T09:37:54.679622+0000","last_fullsized":"2024-05-27T09:37:54.679622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'514","last_scrub_stamp":"2024-05-27T09:37:54.679509+0000","last_deep_scrub":"50'514","last_deep_scrub_stamp":"2024-05-27T09:37:54.679509+0000","last_clean_scrub_stamp":"2024-05-27T09:37:54.679509+0000","objects_scrubbed":170,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:08:39.945707+0000","scrub_duration":223,"objects_trimmed":0,"snaptrim_duration":0.0030229530000000001,"stat_sum":{"num_bytes":19296821,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":20,"num_read_kb":12305,"num_write":330,"num_write_kb":31302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"50'11","reported_seq":102,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:47.954497+0000","last_change":"2024-05-27T09:37:47.954539+0000","last_active":"2024-05-27T09:37:47.954497+0000","last_peered":"2024-05-27T09:37:47.954497+0000","last_clean":"2024-05-27T09:37:47.954497+0000","last_became_active":"2024-05-27T09:27:17.561355+0000","last_became_peered":"2024-05-27T09:27:17.561355+0000","last_unstale":"2024-05-27T09:37:47.954497+0000","last_undegraded":"2024-05-27T09:37:47.954497+0000","last_fullsized":"2024-05-27T09:37:47.954497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'11","last_scrub_stamp":"2024-05-27T09:37:47.954398+0000","last_deep_scrub":"50'11","last_deep_scrub_stamp":"2024-05-27T09:37:47.954398+0000","last_clean_scrub_stamp":"2024-05-27T09:37:47.954398+0000","objects_scrubbed":3,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:31:55.916234+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":8,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9773,"num_omap_keys":17,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"50'508","reported_seq":2647,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:54.623245+0000","last_change":"2024-05-27T09:37:54.623301+0000","last_active":"2024-05-27T09:37:54.623245+0000","last_peered":"2024-05-27T09:37:54.623245+0000","last_clean":"2024-05-27T09:37:54.623245+0000","last_became_active":"2024-05-27T09:27:18.587921+0000","last_became_peered":"2024-05-27T09:27:18.587921+0000","last_unstale":"2024-05-27T09:37:54.623245+0000","last_undegraded":"2024-05-27T09:37:54.623245+0000","last_fullsized":"2024-05-27T09:37:54.623245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'508","last_scrub_stamp":"2024-05-27T09:37:54.623118+0000","last_deep_scrub":"50'508","last_deep_scrub_stamp":"2024-05-27T09:37:54.623118+0000","last_clean_scrub_stamp":"2024-05-27T09:37:54.623118+0000","objects_scrubbed":154,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:09.185088+0000","scrub_duration":241,"objects_trimmed":0,"snaptrim_duration":0.0078895319999999994,"stat_sum":{"num_bytes":10502055,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":21,"num_read_kb":4116,"num_write":345,"num_write_kb":10394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"50'70","reported_seq":160,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:46.660778+0000","last_change":"2024-05-27T09:37:46.660862+0000","last_active":"2024-05-27T09:37:46.660778+0000","last_peered":"2024-05-27T09:37:46.660778+0000","last_clean":"2024-05-27T09:37:46.660778+0000","last_became_active":"2024-05-27T09:27:17.565488+0000","last_became_peered":"2024-05-27T09:27:17.565488+0000","last_unstale":"2024-05-27T09:37:46.660778+0000","last_undegraded":"2024-05-27T09:37:46.660778+0000","last_fullsized":"2024-05-27T09:37:46.660778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'70","last_scrub_stamp":"2024-05-27T09:37:46.660608+0000","last_deep_scrub":"50'70","last_deep_scrub_stamp":"2024-05-27T09:37:46.660608+0000","last_clean_scrub_stamp":"2024-05-27T09:37:46.660608+0000","objects_scrubbed":5,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:07:50.518839+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":4108,"num_write":67,"num_write_kb":4232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19729,"num_omap_keys":33,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"50'500","reported_seq":2641,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:05.950823+0000","last_change":"2024-05-27T09:38:05.950869+0000","last_active":"2024-05-27T09:38:05.950823+0000","last_peered":"2024-05-27T09:38:05.950823+0000","last_clean":"2024-05-27T09:38:05.950823+0000","last_became_active":"2024-05-27T09:27:18.585243+0000","last_became_peered":"2024-05-27T09:27:18.585243+0000","last_unstale":"2024-05-27T09:38:05.950823+0000","last_undegraded":"2024-05-27T09:38:05.950823+0000","last_fullsized":"2024-05-27T09:38:05.950823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'500","last_scrub_stamp":"2024-05-27T09:38:05.950737+0000","last_deep_scrub":"50'500","last_deep_scrub_stamp":"2024-05-27T09:38:05.950737+0000","last_clean_scrub_stamp":"2024-05-27T09:38:05.950737+0000","objects_scrubbed":162,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:27:21.111933+0000","scrub_duration":251,"objects_trimmed":0,"snaptrim_duration":0.0067133569999999997,"stat_sum":{"num_bytes":18063712,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":23,"num_read_kb":8211,"num_write":328,"num_write_kb":30099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"50'212","reported_seq":301,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:52.265400+0000","last_change":"2024-05-27T09:37:52.265458+0000","last_active":"2024-05-27T09:37:52.265400+0000","last_peered":"2024-05-27T09:37:52.265400+0000","last_clean":"2024-05-27T09:37:52.265400+0000","last_became_active":"2024-05-27T09:27:17.548307+0000","last_became_peered":"2024-05-27T09:27:17.548307+0000","last_unstale":"2024-05-27T09:37:52.265400+0000","last_undegraded":"2024-05-27T09:37:52.265400+0000","last_fullsized":"2024-05-27T09:37:52.265400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'212","last_scrub_stamp":"2024-05-27T09:37:52.265291+0000","last_deep_scrub":"50'212","last_deep_scrub_stamp":"2024-05-27T09:37:52.265291+0000","last_clean_scrub_stamp":"2024-05-27T09:37:52.265291+0000","objects_scrubbed":8,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:33:16.734057+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":206,"num_write_kb":4295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45106,"num_omap_keys":76,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"50'519","reported_seq":2654,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:08.690592+0000","last_change":"2024-05-27T09:38:08.690674+0000","last_active":"2024-05-27T09:38:08.690592+0000","last_peered":"2024-05-27T09:38:08.690592+0000","last_clean":"2024-05-27T09:38:08.690592+0000","last_became_active":"2024-05-27T09:27:18.581742+0000","last_became_peered":"2024-05-27T09:27:18.581742+0000","last_unstale":"2024-05-27T09:38:08.690592+0000","last_undegraded":"2024-05-27T09:38:08.690592+0000","last_fullsized":"2024-05-27T09:38:08.690592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'519","last_scrub_stamp":"2024-05-27T09:38:08.690402+0000","last_deep_scrub":"50'519","last_deep_scrub_stamp":"2024-05-27T09:38:08.690402+0000","last_clean_scrub_stamp":"2024-05-27T09:38:08.690402+0000","objects_scrubbed":178,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:13:38.644551+0000","scrub_duration":383,"objects_trimmed":0,"snaptrim_duration":0.0089613100000000001,"stat_sum":{"num_bytes":20607307,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":17,"num_read_kb":10914,"num_write":328,"num_write_kb":24390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"50'255","reported_seq":349,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:48.372336+0000","last_change":"2024-05-27T09:37:48.372336+0000","last_active":"2024-05-27T09:37:48.372336+0000","last_peered":"2024-05-27T09:37:48.372336+0000","last_clean":"2024-05-27T09:37:48.372336+0000","last_became_active":"2024-05-27T09:27:17.556036+0000","last_became_peered":"2024-05-27T09:27:17.556036+0000","last_unstale":"2024-05-27T09:37:48.372336+0000","last_undegraded":"2024-05-27T09:37:48.372336+0000","last_fullsized":"2024-05-27T09:37:48.372336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'254","last_scrub_stamp":"2024-05-27T09:37:48.372049+0000","last_deep_scrub":"50'254","last_deep_scrub_stamp":"2024-05-27T09:37:48.372049+0000","last_clean_scrub_stamp":"2024-05-27T09:37:48.372049+0000","objects_scrubbed":9,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:08:58.424369+0000","scrub_duration":54,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":19,"num_read_kb":8209,"num_write":245,"num_write_kb":12461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19168,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"50'508","reported_seq":2646,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:57.874500+0000","last_change":"2024-05-27T09:37:57.874679+0000","last_active":"2024-05-27T09:37:57.874500+0000","last_peered":"2024-05-27T09:37:57.874500+0000","last_clean":"2024-05-27T09:37:57.874500+0000","last_became_active":"2024-05-27T09:27:18.565500+0000","last_became_peered":"2024-05-27T09:27:18.565500+0000","last_unstale":"2024-05-27T09:37:57.874500+0000","last_undegraded":"2024-05-27T09:37:57.874500+0000","last_fullsized":"2024-05-27T09:37:57.874500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'508","last_scrub_stamp":"2024-05-27T09:37:57.874267+0000","last_deep_scrub":"50'508","last_deep_scrub_stamp":"2024-05-27T09:37:57.874267+0000","last_clean_scrub_stamp":"2024-05-27T09:37:57.874267+0000","objects_scrubbed":173,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:09:16.373282+0000","scrub_duration":364,"objects_trimmed":0,"snaptrim_duration":0.011618516000000001,"stat_sum":{"num_bytes":10741446,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":19,"num_read_kb":4114,"num_write":329,"num_write_kb":14761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"50'24","reported_seq":123,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:54.273986+0000","last_change":"2024-05-27T09:37:54.274029+0000","last_active":"2024-05-27T09:37:54.273986+0000","last_peered":"2024-05-27T09:37:54.273986+0000","last_clean":"2024-05-27T09:37:54.273986+0000","last_became_active":"2024-05-27T09:27:17.571709+0000","last_became_peered":"2024-05-27T09:27:17.571709+0000","last_unstale":"2024-05-27T09:37:54.273986+0000","last_undegraded":"2024-05-27T09:37:54.273986+0000","last_fullsized":"2024-05-27T09:37:54.273986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'24","last_scrub_stamp":"2024-05-27T09:37:54.273898+0000","last_deep_scrub":"50'24","last_deep_scrub_stamp":"2024-05-27T09:37:54.273898+0000","last_clean_scrub_stamp":"2024-05-27T09:37:54.273898+0000","objects_scrubbed":7,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:41:37.759105+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":8230,"num_write":18,"num_write_kb":8221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":21772,"num_omap_keys":38,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"50'517","reported_seq":2661,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:00.524751+0000","last_change":"2024-05-27T09:38:00.524841+0000","last_active":"2024-05-27T09:38:00.524751+0000","last_peered":"2024-05-27T09:38:00.524751+0000","last_clean":"2024-05-27T09:38:00.524751+0000","last_became_active":"2024-05-27T09:27:18.564589+0000","last_became_peered":"2024-05-27T09:27:18.564589+0000","last_unstale":"2024-05-27T09:38:00.524751+0000","last_undegraded":"2024-05-27T09:38:00.524751+0000","last_fullsized":"2024-05-27T09:38:00.524751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'517","last_scrub_stamp":"2024-05-27T09:38:00.524665+0000","last_deep_scrub":"50'517","last_deep_scrub_stamp":"2024-05-27T09:38:00.524665+0000","last_clean_scrub_stamp":"2024-05-27T09:38:00.524665+0000","objects_scrubbed":174,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:12:13.591150+0000","scrub_duration":300,"objects_trimmed":0,"snaptrim_duration":0.0028467779999999999,"stat_sum":{"num_bytes":14347466,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":26,"num_read_kb":16403,"num_write":332,"num_write_kb":30566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"50'11","reported_seq":99,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:56.230040+0000","last_change":"2024-05-27T09:37:56.230145+0000","last_active":"2024-05-27T09:37:56.230040+0000","last_peered":"2024-05-27T09:37:56.230040+0000","last_clean":"2024-05-27T09:37:56.230040+0000","last_became_active":"2024-05-27T09:27:17.554860+0000","last_became_peered":"2024-05-27T09:27:17.554860+0000","last_unstale":"2024-05-27T09:37:56.230040+0000","last_undegraded":"2024-05-27T09:37:56.230040+0000","last_fullsized":"2024-05-27T09:37:56.230040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'11","last_scrub_stamp":"2024-05-27T09:37:56.229839+0000","last_deep_scrub":"50'11","last_deep_scrub_stamp":"2024-05-27T09:37:56.229839+0000","last_clean_scrub_stamp":"2024-05-27T09:37:56.229839+0000","objects_scrubbed":3,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:04:13.966920+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":9,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":12541,"num_omap_keys":21,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"50'529","reported_seq":2669,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:05.643917+0000","last_change":"2024-05-27T09:38:05.644050+0000","last_active":"2024-05-27T09:38:05.643917+0000","last_peered":"2024-05-27T09:38:05.643917+0000","last_clean":"2024-05-27T09:38:05.643917+0000","last_became_active":"2024-05-27T09:27:18.572024+0000","last_became_peered":"2024-05-27T09:27:18.572024+0000","last_unstale":"2024-05-27T09:38:05.643917+0000","last_undegraded":"2024-05-27T09:38:05.643917+0000","last_fullsized":"2024-05-27T09:38:05.643917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'529","last_scrub_stamp":"2024-05-27T09:38:05.643771+0000","last_deep_scrub":"50'529","last_deep_scrub_stamp":"2024-05-27T09:38:05.643771+0000","last_clean_scrub_stamp":"2024-05-27T09:38:05.643771+0000","objects_scrubbed":170,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:42:44.516672+0000","scrub_duration":320,"objects_trimmed":0,"snaptrim_duration":0.010292084,"stat_sum":{"num_bytes":12019831,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":346,"num_write_kb":11826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"50'14","reported_seq":108,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:51.618487+0000","last_change":"2024-05-27T09:37:51.618539+0000","last_active":"2024-05-27T09:37:51.618487+0000","last_peered":"2024-05-27T09:37:51.618487+0000","last_clean":"2024-05-27T09:37:51.618487+0000","last_became_active":"2024-05-27T09:27:17.574695+0000","last_became_peered":"2024-05-27T09:27:17.574695+0000","last_unstale":"2024-05-27T09:37:51.618487+0000","last_undegraded":"2024-05-27T09:37:51.618487+0000","last_fullsized":"2024-05-27T09:37:51.618487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'14","last_scrub_stamp":"2024-05-27T09:37:51.618329+0000","last_deep_scrub":"50'14","last_deep_scrub_stamp":"2024-05-27T09:37:51.618329+0000","last_clean_scrub_stamp":"2024-05-27T09:37:51.618329+0000","objects_scrubbed":3,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:29:12.388346+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1331,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":9,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5952,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"50'507","reported_seq":2644,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:51.127580+0000","last_change":"2024-05-27T09:37:51.127666+0000","last_active":"2024-05-27T09:37:51.127580+0000","last_peered":"2024-05-27T09:37:51.127580+0000","last_clean":"2024-05-27T09:37:51.127580+0000","last_became_active":"2024-05-27T09:27:18.554932+0000","last_became_peered":"2024-05-27T09:27:18.554932+0000","last_unstale":"2024-05-27T09:37:51.127580+0000","last_undegraded":"2024-05-27T09:37:51.127580+0000","last_fullsized":"2024-05-27T09:37:51.127580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'507","last_scrub_stamp":"2024-05-27T09:37:51.127478+0000","last_deep_scrub":"50'507","last_deep_scrub_stamp":"2024-05-27T09:37:51.127478+0000","last_clean_scrub_stamp":"2024-05-27T09:37:51.127478+0000","objects_scrubbed":167,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:18:30.177235+0000","scrub_duration":183,"objects_trimmed":0,"snaptrim_duration":0.0083893030000000007,"stat_sum":{"num_bytes":24206949,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":20,"num_read_kb":12303,"num_write":329,"num_write_kb":23805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"50'78","reported_seq":182,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:49.486654+0000","last_change":"2024-05-27T09:37:49.486715+0000","last_active":"2024-05-27T09:37:49.486654+0000","last_peered":"2024-05-27T09:37:49.486654+0000","last_clean":"2024-05-27T09:37:49.486654+0000","last_became_active":"2024-05-27T09:27:17.547523+0000","last_became_peered":"2024-05-27T09:27:17.547523+0000","last_unstale":"2024-05-27T09:37:49.486654+0000","last_undegraded":"2024-05-27T09:37:49.486654+0000","last_fullsized":"2024-05-27T09:37:49.486654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'78","last_scrub_stamp":"2024-05-27T09:37:49.486545+0000","last_deep_scrub":"50'78","last_deep_scrub_stamp":"2024-05-27T09:37:49.486545+0000","last_clean_scrub_stamp":"2024-05-27T09:37:49.486545+0000","objects_scrubbed":10,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:06:39.622630+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":66,"num_write_kb":8220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"50'552","reported_seq":2693,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:09.677111+0000","last_change":"2024-05-27T09:38:09.677160+0000","last_active":"2024-05-27T09:38:09.677111+0000","last_peered":"2024-05-27T09:38:09.677111+0000","last_clean":"2024-05-27T09:38:09.677111+0000","last_became_active":"2024-05-27T09:27:18.582360+0000","last_became_peered":"2024-05-27T09:27:18.582360+0000","last_unstale":"2024-05-27T09:38:09.677111+0000","last_undegraded":"2024-05-27T09:38:09.677111+0000","last_fullsized":"2024-05-27T09:38:09.677111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'552","last_scrub_stamp":"2024-05-27T09:38:09.677013+0000","last_deep_scrub":"50'552","last_deep_scrub_stamp":"2024-05-27T09:38:09.677013+0000","last_clean_scrub_stamp":"2024-05-27T09:38:09.677013+0000","objects_scrubbed":186,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:48:17.613159+0000","scrub_duration":363,"objects_trimmed":0,"snaptrim_duration":0.0087858729999999996,"stat_sum":{"num_bytes":23260375,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":23,"num_read_kb":13874,"num_write":361,"num_write_kb":35144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"50'47","reported_seq":137,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:50.658360+0000","last_change":"2024-05-27T09:37:50.658423+0000","last_active":"2024-05-27T09:37:50.658360+0000","last_peered":"2024-05-27T09:37:50.658360+0000","last_clean":"2024-05-27T09:37:50.658360+0000","last_became_active":"2024-05-27T09:27:17.571146+0000","last_became_peered":"2024-05-27T09:27:17.571146+0000","last_unstale":"2024-05-27T09:37:50.658360+0000","last_undegraded":"2024-05-27T09:37:50.658360+0000","last_fullsized":"2024-05-27T09:37:50.658360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'47","last_scrub_stamp":"2024-05-27T09:37:50.658274+0000","last_deep_scrub":"50'47","last_deep_scrub_stamp":"2024-05-27T09:37:50.658274+0000","last_clean_scrub_stamp":"2024-05-27T09:37:50.658274+0000","objects_scrubbed":5,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:55:34.128507+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":8,"num_read_kb":4103,"num_write":60,"num_write_kb":4132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15232,"num_omap_keys":26,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"50'474","reported_seq":2604,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:53.697910+0000","last_change":"2024-05-27T09:37:53.697967+0000","last_active":"2024-05-27T09:37:53.697910+0000","last_peered":"2024-05-27T09:37:53.697910+0000","last_clean":"2024-05-27T09:37:53.697910+0000","last_became_active":"2024-05-27T09:27:18.579440+0000","last_became_peered":"2024-05-27T09:27:18.579440+0000","last_unstale":"2024-05-27T09:37:53.697910+0000","last_undegraded":"2024-05-27T09:37:53.697910+0000","last_fullsized":"2024-05-27T09:37:53.697910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'474","last_scrub_stamp":"2024-05-27T09:37:53.697827+0000","last_deep_scrub":"50'474","last_deep_scrub_stamp":"2024-05-27T09:37:53.697827+0000","last_clean_scrub_stamp":"2024-05-27T09:37:53.697827+0000","objects_scrubbed":172,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:11:09.002496+0000","scrub_duration":270,"objects_trimmed":0,"snaptrim_duration":0.0079260630000000006,"stat_sum":{"num_bytes":7949059,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":12,"num_read_kb":4346,"num_write":296,"num_write_kb":16109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"50'33","reported_seq":119,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:51.485868+0000","last_change":"2024-05-27T09:37:51.485922+0000","last_active":"2024-05-27T09:37:51.485868+0000","last_peered":"2024-05-27T09:37:51.485868+0000","last_clean":"2024-05-27T09:37:51.485868+0000","last_became_active":"2024-05-27T09:27:17.550861+0000","last_became_peered":"2024-05-27T09:27:17.550861+0000","last_unstale":"2024-05-27T09:37:51.485868+0000","last_undegraded":"2024-05-27T09:37:51.485868+0000","last_fullsized":"2024-05-27T09:37:51.485868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'33","last_scrub_stamp":"2024-05-27T09:37:51.485753+0000","last_deep_scrub":"50'33","last_deep_scrub_stamp":"2024-05-27T09:37:51.485753+0000","last_clean_scrub_stamp":"2024-05-27T09:37:51.485753+0000","objects_scrubbed":6,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:03:51.622673+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":4099,"num_write":31,"num_write_kb":4134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":29912,"num_omap_keys":50,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"50'503","reported_seq":2641,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:54.745593+0000","last_change":"2024-05-27T09:37:54.745645+0000","last_active":"2024-05-27T09:37:54.745593+0000","last_peered":"2024-05-27T09:37:54.745593+0000","last_clean":"2024-05-27T09:37:54.745593+0000","last_became_active":"2024-05-27T09:27:18.543683+0000","last_became_peered":"2024-05-27T09:27:18.543683+0000","last_unstale":"2024-05-27T09:37:54.745593+0000","last_undegraded":"2024-05-27T09:37:54.745593+0000","last_fullsized":"2024-05-27T09:37:54.745593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'503","last_scrub_stamp":"2024-05-27T09:37:54.745516+0000","last_deep_scrub":"50'503","last_deep_scrub_stamp":"2024-05-27T09:37:54.745516+0000","last_clean_scrub_stamp":"2024-05-27T09:37:54.745516+0000","objects_scrubbed":166,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:29:25.882219+0000","scrub_duration":199,"objects_trimmed":0,"snaptrim_duration":0.012478668,"stat_sum":{"num_bytes":14963144,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":20,"num_read_kb":4115,"num_write":326,"num_write_kb":22968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"50'98","reported_seq":185,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:53.843783+0000","last_change":"2024-05-27T09:37:53.843818+0000","last_active":"2024-05-27T09:37:53.843783+0000","last_peered":"2024-05-27T09:37:53.843783+0000","last_clean":"2024-05-27T09:37:53.843783+0000","last_became_active":"2024-05-27T09:27:17.558973+0000","last_became_peered":"2024-05-27T09:27:17.558973+0000","last_unstale":"2024-05-27T09:37:53.843783+0000","last_undegraded":"2024-05-27T09:37:53.843783+0000","last_fullsized":"2024-05-27T09:37:53.843783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'98","last_scrub_stamp":"2024-05-27T09:37:53.843688+0000","last_deep_scrub":"50'98","last_deep_scrub_stamp":"2024-05-27T09:37:53.843688+0000","last_clean_scrub_stamp":"2024-05-27T09:37:53.843688+0000","objects_scrubbed":5,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:33:22.712060+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":4097,"num_write":96,"num_write_kb":4263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23922,"num_omap_keys":40,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"50'547","reported_seq":2680,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:55.781434+0000","last_change":"2024-05-27T09:37:55.781490+0000","last_active":"2024-05-27T09:37:55.781434+0000","last_peered":"2024-05-27T09:37:55.781434+0000","last_clean":"2024-05-27T09:37:55.781434+0000","last_became_active":"2024-05-27T09:27:18.557076+0000","last_became_peered":"2024-05-27T09:27:18.557076+0000","last_unstale":"2024-05-27T09:37:55.781434+0000","last_undegraded":"2024-05-27T09:37:55.781434+0000","last_fullsized":"2024-05-27T09:37:55.781434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'547","last_scrub_stamp":"2024-05-27T09:37:55.781315+0000","last_deep_scrub":"50'547","last_deep_scrub_stamp":"2024-05-27T09:37:55.781315+0000","last_clean_scrub_stamp":"2024-05-27T09:37:55.781315+0000","objects_scrubbed":181,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:06:08.215786+0000","scrub_duration":221,"objects_trimmed":0,"snaptrim_duration":0.014584746000000001,"stat_sum":{"num_bytes":15385587,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":358,"num_write_kb":15182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"50'53","reported_seq":147,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:54.125558+0000","last_change":"2024-05-27T09:37:54.125598+0000","last_active":"2024-05-27T09:37:54.125558+0000","last_peered":"2024-05-27T09:37:54.125558+0000","last_clean":"2024-05-27T09:37:54.125558+0000","last_became_active":"2024-05-27T09:27:17.573484+0000","last_became_peered":"2024-05-27T09:27:17.573484+0000","last_unstale":"2024-05-27T09:37:54.125558+0000","last_undegraded":"2024-05-27T09:37:54.125558+0000","last_fullsized":"2024-05-27T09:37:54.125558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'53","last_scrub_stamp":"2024-05-27T09:37:54.125443+0000","last_deep_scrub":"50'53","last_deep_scrub_stamp":"2024-05-27T09:37:54.125443+0000","last_clean_scrub_stamp":"2024-05-27T09:37:54.125443+0000","objects_scrubbed":4,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:03:54.262363+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":51,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9584,"num_omap_keys":16,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"50'490","reported_seq":2631,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:07.610727+0000","last_change":"2024-05-27T09:38:07.610763+0000","last_active":"2024-05-27T09:38:07.610727+0000","last_peered":"2024-05-27T09:38:07.610727+0000","last_clean":"2024-05-27T09:38:07.610727+0000","last_became_active":"2024-05-27T09:27:18.574959+0000","last_became_peered":"2024-05-27T09:27:18.574959+0000","last_unstale":"2024-05-27T09:38:07.610727+0000","last_undegraded":"2024-05-27T09:38:07.610727+0000","last_fullsized":"2024-05-27T09:38:07.610727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'490","last_scrub_stamp":"2024-05-27T09:38:07.610645+0000","last_deep_scrub":"50'490","last_deep_scrub_stamp":"2024-05-27T09:38:07.610645+0000","last_clean_scrub_stamp":"2024-05-27T09:38:07.610645+0000","objects_scrubbed":154,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:45:11.807913+0000","scrub_duration":316,"objects_trimmed":0,"snaptrim_duration":0.0093604480000000004,"stat_sum":{"num_bytes":20164393,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":23,"num_read_kb":12305,"num_write":329,"num_write_kb":28020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"50'74","reported_seq":184,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:46.937525+0000","last_change":"2024-05-27T09:37:46.937605+0000","last_active":"2024-05-27T09:37:46.937525+0000","last_peered":"2024-05-27T09:37:46.937525+0000","last_clean":"2024-05-27T09:37:46.937525+0000","last_became_active":"2024-05-27T09:27:12.690301+0000","last_became_peered":"2024-05-27T09:27:12.690301+0000","last_unstale":"2024-05-27T09:37:46.937525+0000","last_undegraded":"2024-05-27T09:37:46.937525+0000","last_fullsized":"2024-05-27T09:37:46.937525+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'74","last_scrub_stamp":"2024-05-27T09:37:46.937317+0000","last_deep_scrub":"50'74","last_deep_scrub_stamp":"2024-05-27T09:37:46.937317+0000","last_clean_scrub_stamp":"2024-05-27T09:37:46.937317+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:22.317649+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'18","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:00.230618+0000","last_change":"2024-05-27T09:38:00.230715+0000","last_active":"2024-05-27T09:38:00.230618+0000","last_peered":"2024-05-27T09:38:00.230618+0000","last_clean":"2024-05-27T09:38:00.230618+0000","last_became_active":"2024-05-27T09:27:17.566292+0000","last_became_peered":"2024-05-27T09:27:17.566292+0000","last_unstale":"2024-05-27T09:38:00.230618+0000","last_undegraded":"2024-05-27T09:38:00.230618+0000","last_fullsized":"2024-05-27T09:38:00.230618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'18","last_scrub_stamp":"2024-05-27T09:38:00.230408+0000","last_deep_scrub":"50'18","last_deep_scrub_stamp":"2024-05-27T09:38:00.230408+0000","last_clean_scrub_stamp":"2024-05-27T09:38:00.230408+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:14:16.875793+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194338,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":21,"num_write":14,"num_write_kb":4206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":18178,"num_omap_keys":32,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"50'506","reported_seq":2637,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:59.616189+0000","last_change":"2024-05-27T09:37:59.616224+0000","last_active":"2024-05-27T09:37:59.616189+0000","last_peered":"2024-05-27T09:37:59.616189+0000","last_clean":"2024-05-27T09:37:59.616189+0000","last_became_active":"2024-05-27T09:27:18.570782+0000","last_became_peered":"2024-05-27T09:27:18.570782+0000","last_unstale":"2024-05-27T09:37:59.616189+0000","last_undegraded":"2024-05-27T09:37:59.616189+0000","last_fullsized":"2024-05-27T09:37:59.616189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'506","last_scrub_stamp":"2024-05-27T09:37:59.616116+0000","last_deep_scrub":"50'506","last_deep_scrub_stamp":"2024-05-27T09:37:59.616116+0000","last_clean_scrub_stamp":"2024-05-27T09:37:59.616116+0000","objects_scrubbed":165,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:03:39.985732+0000","scrub_duration":233,"objects_trimmed":0,"snaptrim_duration":0.0044685039999999999,"stat_sum":{"num_bytes":10376859,"num_objects":165,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":333,"num_write_kb":18496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"50'349","reported_seq":442,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:58.280979+0000","last_change":"2024-05-27T09:37:58.281077+0000","last_active":"2024-05-27T09:37:58.280979+0000","last_peered":"2024-05-27T09:37:58.280979+0000","last_clean":"2024-05-27T09:37:58.280979+0000","last_became_active":"2024-05-27T09:27:17.563010+0000","last_became_peered":"2024-05-27T09:27:17.563010+0000","last_unstale":"2024-05-27T09:37:58.280979+0000","last_undegraded":"2024-05-27T09:37:58.280979+0000","last_fullsized":"2024-05-27T09:37:58.280979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'349","last_scrub_stamp":"2024-05-27T09:37:58.280821+0000","last_deep_scrub":"50'349","last_deep_scrub_stamp":"2024-05-27T09:37:58.280821+0000","last_clean_scrub_stamp":"2024-05-27T09:37:58.280821+0000","objects_scrubbed":5,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:34:10.665565+0000","scrub_duration":72,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":8210,"num_write":339,"num_write_kb":12482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7605,"num_omap_keys":13,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"50'483","reported_seq":2613,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:01.454472+0000","last_change":"2024-05-27T09:38:01.454511+0000","last_active":"2024-05-27T09:38:01.454472+0000","last_peered":"2024-05-27T09:38:01.454472+0000","last_clean":"2024-05-27T09:38:01.454472+0000","last_became_active":"2024-05-27T09:27:18.574090+0000","last_became_peered":"2024-05-27T09:27:18.574090+0000","last_unstale":"2024-05-27T09:38:01.454472+0000","last_undegraded":"2024-05-27T09:38:01.454472+0000","last_fullsized":"2024-05-27T09:38:01.454472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'483","last_scrub_stamp":"2024-05-27T09:38:01.454366+0000","last_deep_scrub":"50'483","last_deep_scrub_stamp":"2024-05-27T09:38:01.454366+0000","last_clean_scrub_stamp":"2024-05-27T09:38:01.454366+0000","objects_scrubbed":160,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:24:02.356989+0000","scrub_duration":230,"objects_trimmed":0,"snaptrim_duration":0.002904064,"stat_sum":{"num_bytes":5516261,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":316,"num_write_kb":17833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"50'127","reported_seq":331,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:53.207679+0000","last_change":"2024-05-27T09:37:53.207708+0000","last_active":"2024-05-27T09:37:53.207679+0000","last_peered":"2024-05-27T09:37:53.207679+0000","last_clean":"2024-05-27T09:37:53.207679+0000","last_became_active":"2024-05-27T09:27:17.550982+0000","last_became_peered":"2024-05-27T09:27:17.550982+0000","last_unstale":"2024-05-27T09:37:53.207679+0000","last_undegraded":"2024-05-27T09:37:53.207679+0000","last_fullsized":"2024-05-27T09:37:53.207679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'127","last_scrub_stamp":"2024-05-27T09:37:53.207607+0000","last_deep_scrub":"50'127","last_deep_scrub_stamp":"2024-05-27T09:37:53.207607+0000","last_clean_scrub_stamp":"2024-05-27T09:37:53.207607+0000","objects_scrubbed":5,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:35:15.641392+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":534380,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":130,"num_read_kb":615,"num_write":121,"num_write_kb":621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19509,"num_omap_keys":33,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"50'534","reported_seq":2675,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:54.971188+0000","last_change":"2024-05-27T09:37:54.971228+0000","last_active":"2024-05-27T09:37:54.971188+0000","last_peered":"2024-05-27T09:37:54.971188+0000","last_clean":"2024-05-27T09:37:54.971188+0000","last_became_active":"2024-05-27T09:27:18.559017+0000","last_became_peered":"2024-05-27T09:27:18.559017+0000","last_unstale":"2024-05-27T09:37:54.971188+0000","last_undegraded":"2024-05-27T09:37:54.971188+0000","last_fullsized":"2024-05-27T09:37:54.971188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'534","last_scrub_stamp":"2024-05-27T09:37:54.971090+0000","last_deep_scrub":"50'534","last_deep_scrub_stamp":"2024-05-27T09:37:54.971090+0000","last_clean_scrub_stamp":"2024-05-27T09:37:54.971090+0000","objects_scrubbed":186,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:30:20.103519+0000","scrub_duration":194,"objects_trimmed":0,"snaptrim_duration":0.0065168789999999997,"stat_sum":{"num_bytes":26414737,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":21,"num_read_kb":8209,"num_write":340,"num_write_kb":34165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"50'51","reported_seq":138,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:59.216350+0000","last_change":"2024-05-27T09:37:59.216511+0000","last_active":"2024-05-27T09:37:59.216350+0000","last_peered":"2024-05-27T09:37:59.216350+0000","last_clean":"2024-05-27T09:37:59.216350+0000","last_became_active":"2024-05-27T09:27:17.563640+0000","last_became_peered":"2024-05-27T09:27:17.563640+0000","last_unstale":"2024-05-27T09:37:59.216350+0000","last_undegraded":"2024-05-27T09:37:59.216350+0000","last_fullsized":"2024-05-27T09:37:59.216350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'51","last_scrub_stamp":"2024-05-27T09:37:59.216084+0000","last_deep_scrub":"50'51","last_deep_scrub_stamp":"2024-05-27T09:37:59.216084+0000","last_clean_scrub_stamp":"2024-05-27T09:37:59.216084+0000","objects_scrubbed":3,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:41:44.336951+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":43,"num_write_kb":8223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9584,"num_omap_keys":16,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"50'533","reported_seq":2669,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:59.699980+0000","last_change":"2024-05-27T09:37:59.700064+0000","last_active":"2024-05-27T09:37:59.699980+0000","last_peered":"2024-05-27T09:37:59.699980+0000","last_clean":"2024-05-27T09:37:59.699980+0000","last_became_active":"2024-05-27T09:27:18.574710+0000","last_became_peered":"2024-05-27T09:27:18.574710+0000","last_unstale":"2024-05-27T09:37:59.699980+0000","last_undegraded":"2024-05-27T09:37:59.699980+0000","last_fullsized":"2024-05-27T09:37:59.699980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'533","last_scrub_stamp":"2024-05-27T09:37:59.699881+0000","last_deep_scrub":"50'533","last_deep_scrub_stamp":"2024-05-27T09:37:59.699881+0000","last_clean_scrub_stamp":"2024-05-27T09:37:59.699881+0000","objects_scrubbed":174,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:16:05.114715+0000","scrub_duration":239,"objects_trimmed":0,"snaptrim_duration":0.0075561990000000004,"stat_sum":{"num_bytes":9767075,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":17,"num_read_kb":8206,"num_write":352,"num_write_kb":9721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"50'85","reported_seq":171,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:51.892325+0000","last_change":"2024-05-27T09:37:51.892361+0000","last_active":"2024-05-27T09:37:51.892325+0000","last_peered":"2024-05-27T09:37:51.892325+0000","last_clean":"2024-05-27T09:37:51.892325+0000","last_became_active":"2024-05-27T09:27:17.550378+0000","last_became_peered":"2024-05-27T09:27:17.550378+0000","last_unstale":"2024-05-27T09:37:51.892325+0000","last_undegraded":"2024-05-27T09:37:51.892325+0000","last_fullsized":"2024-05-27T09:37:51.892325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'85","last_scrub_stamp":"2024-05-27T09:37:51.892237+0000","last_deep_scrub":"50'85","last_deep_scrub_stamp":"2024-05-27T09:37:51.892237+0000","last_clean_scrub_stamp":"2024-05-27T09:37:51.892237+0000","objects_scrubbed":4,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:23:31.162250+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83,"num_write_kb":4165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34670,"num_omap_keys":59,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"50'528","reported_seq":2668,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:49.540686+0000","last_change":"2024-05-27T09:37:49.540740+0000","last_active":"2024-05-27T09:37:49.540686+0000","last_peered":"2024-05-27T09:37:49.540686+0000","last_clean":"2024-05-27T09:37:49.540686+0000","last_became_active":"2024-05-27T09:27:18.566326+0000","last_became_peered":"2024-05-27T09:27:18.566326+0000","last_unstale":"2024-05-27T09:37:49.540686+0000","last_undegraded":"2024-05-27T09:37:49.540686+0000","last_fullsized":"2024-05-27T09:37:49.540686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'528","last_scrub_stamp":"2024-05-27T09:37:49.540609+0000","last_deep_scrub":"50'528","last_deep_scrub_stamp":"2024-05-27T09:37:49.540609+0000","last_clean_scrub_stamp":"2024-05-27T09:37:49.540609+0000","objects_scrubbed":187,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:53:38.730540+0000","scrub_duration":236,"objects_trimmed":0,"snaptrim_duration":0.0075094710000000002,"stat_sum":{"num_bytes":14788118,"num_objects":187,"num_object_clones":0,"num_object_copies":374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":187,"num_whiteouts":0,"num_read":20,"num_read_kb":8209,"num_write":356,"num_write_kb":35101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'358","reported_seq":446,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:54.301752+0000","last_change":"2024-05-27T09:37:54.301839+0000","last_active":"2024-05-27T09:37:54.301752+0000","last_peered":"2024-05-27T09:37:54.301752+0000","last_clean":"2024-05-27T09:37:54.301752+0000","last_became_active":"2024-05-27T09:27:17.557216+0000","last_became_peered":"2024-05-27T09:27:17.557216+0000","last_unstale":"2024-05-27T09:37:54.301752+0000","last_undegraded":"2024-05-27T09:37:54.301752+0000","last_fullsized":"2024-05-27T09:37:54.301752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'358","last_scrub_stamp":"2024-05-27T09:37:54.301531+0000","last_deep_scrub":"50'358","last_deep_scrub_stamp":"2024-05-27T09:37:54.301531+0000","last_clean_scrub_stamp":"2024-05-27T09:37:54.301531+0000","objects_scrubbed":6,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:51:20.515456+0000","scrub_duration":56,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":4098,"num_write":353,"num_write_kb":8433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30283,"num_omap_keys":51,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"50'523","reported_seq":2656,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:59.359322+0000","last_change":"2024-05-27T09:37:59.359350+0000","last_active":"2024-05-27T09:37:59.359322+0000","last_peered":"2024-05-27T09:37:59.359322+0000","last_clean":"2024-05-27T09:37:59.359322+0000","last_became_active":"2024-05-27T09:27:18.562565+0000","last_became_peered":"2024-05-27T09:27:18.562565+0000","last_unstale":"2024-05-27T09:37:59.359322+0000","last_undegraded":"2024-05-27T09:37:59.359322+0000","last_fullsized":"2024-05-27T09:37:59.359322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'523","last_scrub_stamp":"2024-05-27T09:37:59.359247+0000","last_deep_scrub":"50'523","last_deep_scrub_stamp":"2024-05-27T09:37:59.359247+0000","last_clean_scrub_stamp":"2024-05-27T09:37:59.359247+0000","objects_scrubbed":176,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:54:57.249245+0000","scrub_duration":152,"objects_trimmed":0,"snaptrim_duration":0.0030336680000000002,"stat_sum":{"num_bytes":10461277,"num_objects":176,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":176,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":338,"num_write_kb":10392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"50'202","reported_seq":294,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:53.507646+0000","last_change":"2024-05-27T09:37:53.507646+0000","last_active":"2024-05-27T09:37:53.507646+0000","last_peered":"2024-05-27T09:37:53.507646+0000","last_clean":"2024-05-27T09:37:53.507646+0000","last_became_active":"2024-05-27T09:27:17.571551+0000","last_became_peered":"2024-05-27T09:27:17.571551+0000","last_unstale":"2024-05-27T09:37:53.507646+0000","last_undegraded":"2024-05-27T09:37:53.507646+0000","last_fullsized":"2024-05-27T09:37:53.507646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'202","last_scrub_stamp":"2024-05-27T09:37:53.507457+0000","last_deep_scrub":"50'202","last_deep_scrub_stamp":"2024-05-27T09:37:53.507457+0000","last_clean_scrub_stamp":"2024-05-27T09:37:53.507457+0000","objects_scrubbed":5,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T21:22:45.963375+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":455,"num_write_kb":2056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":21564,"num_omap_keys":36,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"50'483","reported_seq":2617,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:58.524059+0000","last_change":"2024-05-27T09:37:58.524059+0000","last_active":"2024-05-27T09:37:58.524059+0000","last_peered":"2024-05-27T09:37:58.524059+0000","last_clean":"2024-05-27T09:37:58.524059+0000","last_became_active":"2024-05-27T09:27:18.558463+0000","last_became_peered":"2024-05-27T09:27:18.558463+0000","last_unstale":"2024-05-27T09:37:58.524059+0000","last_undegraded":"2024-05-27T09:37:58.524059+0000","last_fullsized":"2024-05-27T09:37:58.524059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'483","last_scrub_stamp":"2024-05-27T09:37:58.523917+0000","last_deep_scrub":"50'483","last_deep_scrub_stamp":"2024-05-27T09:37:58.523917+0000","last_clean_scrub_stamp":"2024-05-27T09:37:58.523917+0000","objects_scrubbed":154,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:34:07.106678+0000","scrub_duration":159,"objects_trimmed":0,"snaptrim_duration":0.0041648589999999999,"stat_sum":{"num_bytes":14892598,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":326,"num_write_kb":22861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"50'179","reported_seq":269,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:01.281318+0000","last_change":"2024-05-27T09:38:01.281318+0000","last_active":"2024-05-27T09:38:01.281318+0000","last_peered":"2024-05-27T09:38:01.281318+0000","last_clean":"2024-05-27T09:38:01.281318+0000","last_became_active":"2024-05-27T09:27:17.567650+0000","last_became_peered":"2024-05-27T09:27:17.567650+0000","last_unstale":"2024-05-27T09:38:01.281318+0000","last_undegraded":"2024-05-27T09:38:01.281318+0000","last_fullsized":"2024-05-27T09:38:01.281318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'179","last_scrub_stamp":"2024-05-27T09:38:01.280999+0000","last_deep_scrub":"50'179","last_deep_scrub_stamp":"2024-05-27T09:38:01.280999+0000","last_clean_scrub_stamp":"2024-05-27T09:38:01.280999+0000","objects_scrubbed":8,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:00:14.757054+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":176,"num_write_kb":4290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":35788,"num_omap_keys":60,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"50'513","reported_seq":2658,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:56.537037+0000","last_change":"2024-05-27T09:37:56.537103+0000","last_active":"2024-05-27T09:37:56.537037+0000","last_peered":"2024-05-27T09:37:56.537037+0000","last_clean":"2024-05-27T09:37:56.537037+0000","last_became_active":"2024-05-27T09:27:18.554462+0000","last_became_peered":"2024-05-27T09:27:18.554462+0000","last_unstale":"2024-05-27T09:37:56.537037+0000","last_undegraded":"2024-05-27T09:37:56.537037+0000","last_fullsized":"2024-05-27T09:37:56.537037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'513","last_scrub_stamp":"2024-05-27T09:37:56.536947+0000","last_deep_scrub":"50'513","last_deep_scrub_stamp":"2024-05-27T09:37:56.536947+0000","last_clean_scrub_stamp":"2024-05-27T09:37:56.536947+0000","objects_scrubbed":166,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:02:25.350277+0000","scrub_duration":143,"objects_trimmed":0,"snaptrim_duration":0.0042444429999999997,"stat_sum":{"num_bytes":6471160,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":27,"num_read_kb":4122,"num_write":339,"num_write_kb":6465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"50'129","reported_seq":220,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:48.652051+0000","last_change":"2024-05-27T09:37:48.652051+0000","last_active":"2024-05-27T09:37:48.652051+0000","last_peered":"2024-05-27T09:37:48.652051+0000","last_clean":"2024-05-27T09:37:48.652051+0000","last_became_active":"2024-05-27T09:27:17.569685+0000","last_became_peered":"2024-05-27T09:27:17.569685+0000","last_unstale":"2024-05-27T09:37:48.652051+0000","last_undegraded":"2024-05-27T09:37:48.652051+0000","last_fullsized":"2024-05-27T09:37:48.652051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'129","last_scrub_stamp":"2024-05-27T09:37:48.651868+0000","last_deep_scrub":"50'129","last_deep_scrub_stamp":"2024-05-27T09:37:48.651868+0000","last_clean_scrub_stamp":"2024-05-27T09:37:48.651868+0000","objects_scrubbed":3,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:28:51.821813+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":4108,"num_write":127,"num_write_kb":4196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11381,"num_omap_keys":19,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"50'524","reported_seq":2669,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:51.435472+0000","last_change":"2024-05-27T09:37:51.435472+0000","last_active":"2024-05-27T09:37:51.435472+0000","last_peered":"2024-05-27T09:37:51.435472+0000","last_clean":"2024-05-27T09:37:51.435472+0000","last_became_active":"2024-05-27T09:27:18.562455+0000","last_became_peered":"2024-05-27T09:27:18.562455+0000","last_unstale":"2024-05-27T09:37:51.435472+0000","last_undegraded":"2024-05-27T09:37:51.435472+0000","last_fullsized":"2024-05-27T09:37:51.435472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'524","last_scrub_stamp":"2024-05-27T09:37:51.434886+0000","last_deep_scrub":"50'524","last_deep_scrub_stamp":"2024-05-27T09:37:51.434886+0000","last_clean_scrub_stamp":"2024-05-27T09:37:51.434886+0000","objects_scrubbed":169,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:58:26.663400+0000","scrub_duration":216,"objects_trimmed":0,"snaptrim_duration":0.0082016410000000008,"stat_sum":{"num_bytes":31948882,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":27,"num_read_kb":17799,"num_write":353,"num_write_kb":35479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"50'175","reported_seq":266,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:52.496790+0000","last_change":"2024-05-27T09:37:52.496790+0000","last_active":"2024-05-27T09:37:52.496790+0000","last_peered":"2024-05-27T09:37:52.496790+0000","last_clean":"2024-05-27T09:37:52.496790+0000","last_became_active":"2024-05-27T09:27:17.562941+0000","last_became_peered":"2024-05-27T09:27:17.562941+0000","last_unstale":"2024-05-27T09:37:52.496790+0000","last_undegraded":"2024-05-27T09:37:52.496790+0000","last_fullsized":"2024-05-27T09:37:52.496790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'175","last_scrub_stamp":"2024-05-27T09:37:52.496655+0000","last_deep_scrub":"50'175","last_deep_scrub_stamp":"2024-05-27T09:37:52.496655+0000","last_clean_scrub_stamp":"2024-05-27T09:37:52.496655+0000","objects_scrubbed":4,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:22:31.378934+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":171,"num_write_kb":4214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"50'518","reported_seq":2655,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:57.531641+0000","last_change":"2024-05-27T09:37:57.531641+0000","last_active":"2024-05-27T09:37:57.531641+0000","last_peered":"2024-05-27T09:37:57.531641+0000","last_clean":"2024-05-27T09:37:57.531641+0000","last_became_active":"2024-05-27T09:27:18.555280+0000","last_became_peered":"2024-05-27T09:27:18.555280+0000","last_unstale":"2024-05-27T09:37:57.531641+0000","last_undegraded":"2024-05-27T09:37:57.531641+0000","last_fullsized":"2024-05-27T09:37:57.531641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'518","last_scrub_stamp":"2024-05-27T09:37:57.531519+0000","last_deep_scrub":"50'518","last_deep_scrub_stamp":"2024-05-27T09:37:57.531519+0000","last_clean_scrub_stamp":"2024-05-27T09:37:57.531519+0000","objects_scrubbed":169,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:44:31.172614+0000","scrub_duration":178,"objects_trimmed":0,"snaptrim_duration":0.0041574209999999997,"stat_sum":{"num_bytes":13440874,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":19,"num_read_kb":8208,"num_write":339,"num_write_kb":17394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"50'293","reported_seq":382,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:05.518234+0000","last_change":"2024-05-27T09:37:52.128861+0000","last_active":"2024-05-27T09:38:05.518234+0000","last_peered":"2024-05-27T09:38:05.518234+0000","last_clean":"2024-05-27T09:38:05.518234+0000","last_became_active":"2024-05-27T09:27:17.547938+0000","last_became_peered":"2024-05-27T09:27:17.547938+0000","last_unstale":"2024-05-27T09:38:05.518234+0000","last_undegraded":"2024-05-27T09:38:05.518234+0000","last_fullsized":"2024-05-27T09:38:05.518234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'282","last_scrub_stamp":"2024-05-27T09:37:52.128744+0000","last_deep_scrub":"50'282","last_deep_scrub_stamp":"2024-05-27T09:37:52.128744+0000","last_clean_scrub_stamp":"2024-05-27T09:37:52.128744+0000","objects_scrubbed":4,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:23:08.559629+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7516400,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5,"num_read_kb":4100,"num_write":320,"num_write_kb":7506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11980,"num_omap_keys":20,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"50'514","reported_seq":2647,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:56.367802+0000","last_change":"2024-05-27T09:37:56.367854+0000","last_active":"2024-05-27T09:37:56.367802+0000","last_peered":"2024-05-27T09:37:56.367802+0000","last_clean":"2024-05-27T09:37:56.367802+0000","last_became_active":"2024-05-27T09:27:18.563227+0000","last_became_peered":"2024-05-27T09:27:18.563227+0000","last_unstale":"2024-05-27T09:37:56.367802+0000","last_undegraded":"2024-05-27T09:37:56.367802+0000","last_fullsized":"2024-05-27T09:37:56.367802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'514","last_scrub_stamp":"2024-05-27T09:37:56.367725+0000","last_deep_scrub":"50'514","last_deep_scrub_stamp":"2024-05-27T09:37:56.367725+0000","last_clean_scrub_stamp":"2024-05-27T09:37:56.367725+0000","objects_scrubbed":172,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:24:49.179816+0000","scrub_duration":183,"objects_trimmed":0,"snaptrim_duration":0.0010490250000000001,"stat_sum":{"num_bytes":10717337,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":15,"num_read_kb":2569,"num_write":334,"num_write_kb":14740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"50'349","reported_seq":440,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:57.341569+0000","last_change":"2024-05-27T09:37:57.341620+0000","last_active":"2024-05-27T09:37:57.341569+0000","last_peered":"2024-05-27T09:37:57.341569+0000","last_clean":"2024-05-27T09:37:57.341569+0000","last_became_active":"2024-05-27T09:27:17.561857+0000","last_became_peered":"2024-05-27T09:27:17.561857+0000","last_unstale":"2024-05-27T09:37:57.341569+0000","last_undegraded":"2024-05-27T09:37:57.341569+0000","last_fullsized":"2024-05-27T09:37:57.341569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'349","last_scrub_stamp":"2024-05-27T09:37:57.341468+0000","last_deep_scrub":"50'349","last_deep_scrub_stamp":"2024-05-27T09:37:57.341468+0000","last_clean_scrub_stamp":"2024-05-27T09:37:57.341468+0000","objects_scrubbed":7,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:22:48.196622+0000","scrub_duration":87,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":16,"num_read_kb":8206,"num_write":340,"num_write_kb":12481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":21330,"num_omap_keys":36,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"50'545","reported_seq":2677,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:02.510524+0000","last_change":"2024-05-27T09:38:02.510524+0000","last_active":"2024-05-27T09:38:02.510524+0000","last_peered":"2024-05-27T09:38:02.510524+0000","last_clean":"2024-05-27T09:38:02.510524+0000","last_became_active":"2024-05-27T09:27:18.572339+0000","last_became_peered":"2024-05-27T09:27:18.572339+0000","last_unstale":"2024-05-27T09:38:02.510524+0000","last_undegraded":"2024-05-27T09:38:02.510524+0000","last_fullsized":"2024-05-27T09:38:02.510524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'545","last_scrub_stamp":"2024-05-27T09:38:02.510397+0000","last_deep_scrub":"50'545","last_deep_scrub_stamp":"2024-05-27T09:38:02.510397+0000","last_clean_scrub_stamp":"2024-05-27T09:38:02.510397+0000","objects_scrubbed":185,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:43:40.815595+0000","scrub_duration":273,"objects_trimmed":0,"snaptrim_duration":0.011410574,"stat_sum":{"num_bytes":1288358,"num_objects":185,"num_object_clones":0,"num_object_copies":370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":185,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":355,"num_write_kb":9639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"50'493","reported_seq":2633,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:03.576295+0000","last_change":"2024-05-27T09:38:03.576351+0000","last_active":"2024-05-27T09:38:03.576295+0000","last_peered":"2024-05-27T09:38:03.576295+0000","last_clean":"2024-05-27T09:38:03.576295+0000","last_became_active":"2024-05-27T09:27:18.572307+0000","last_became_peered":"2024-05-27T09:27:18.572307+0000","last_unstale":"2024-05-27T09:38:03.576295+0000","last_undegraded":"2024-05-27T09:38:03.576295+0000","last_fullsized":"2024-05-27T09:38:03.576295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'493","last_scrub_stamp":"2024-05-27T09:38:03.576203+0000","last_deep_scrub":"50'493","last_deep_scrub_stamp":"2024-05-27T09:38:03.576203+0000","last_clean_scrub_stamp":"2024-05-27T09:38:03.576203+0000","objects_scrubbed":163,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:32:47.632307+0000","scrub_duration":317,"objects_trimmed":0,"snaptrim_duration":0.0089418529999999996,"stat_sum":{"num_bytes":7551421,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":315,"num_write_kb":11635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"50'37","reported_seq":124,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:49.646606+0000","last_change":"2024-05-27T09:37:49.646671+0000","last_active":"2024-05-27T09:37:49.646606+0000","last_peered":"2024-05-27T09:37:49.646606+0000","last_clean":"2024-05-27T09:37:49.646606+0000","last_became_active":"2024-05-27T09:27:17.569866+0000","last_became_peered":"2024-05-27T09:27:17.569866+0000","last_unstale":"2024-05-27T09:37:49.646606+0000","last_undegraded":"2024-05-27T09:37:49.646606+0000","last_fullsized":"2024-05-27T09:37:49.646606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'37","last_scrub_stamp":"2024-05-27T09:37:49.646489+0000","last_deep_scrub":"50'37","last_deep_scrub_stamp":"2024-05-27T09:37:49.646489+0000","last_clean_scrub_stamp":"2024-05-27T09:37:49.646489+0000","objects_scrubbed":5,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:29:34.877924+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4099,"num_write":35,"num_write_kb":4131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24702,"num_omap_keys":42,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"50'530","reported_seq":2663,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:50.231030+0000","last_change":"2024-05-27T09:37:50.231090+0000","last_active":"2024-05-27T09:37:50.231030+0000","last_peered":"2024-05-27T09:37:50.231030+0000","last_clean":"2024-05-27T09:37:50.231030+0000","last_became_active":"2024-05-27T09:27:18.549035+0000","last_became_peered":"2024-05-27T09:27:18.549035+0000","last_unstale":"2024-05-27T09:37:50.231030+0000","last_undegraded":"2024-05-27T09:37:50.231030+0000","last_fullsized":"2024-05-27T09:37:50.231030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'530","last_scrub_stamp":"2024-05-27T09:37:50.230914+0000","last_deep_scrub":"50'530","last_deep_scrub_stamp":"2024-05-27T09:37:50.230914+0000","last_clean_scrub_stamp":"2024-05-27T09:37:50.230914+0000","objects_scrubbed":183,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:55:23.381978+0000","scrub_duration":276,"objects_trimmed":0,"snaptrim_duration":0.0090041500000000007,"stat_sum":{"num_bytes":20118472,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":14,"num_read_kb":13,"num_write":350,"num_write_kb":27987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"50'150","reported_seq":242,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:47.648122+0000","last_change":"2024-05-27T09:37:47.648164+0000","last_active":"2024-05-27T09:37:47.648122+0000","last_peered":"2024-05-27T09:37:47.648122+0000","last_clean":"2024-05-27T09:37:47.648122+0000","last_became_active":"2024-05-27T09:27:17.567235+0000","last_became_peered":"2024-05-27T09:27:17.567235+0000","last_unstale":"2024-05-27T09:37:47.648122+0000","last_undegraded":"2024-05-27T09:37:47.648122+0000","last_fullsized":"2024-05-27T09:37:47.648122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'150","last_scrub_stamp":"2024-05-27T09:37:47.648025+0000","last_deep_scrub":"50'150","last_deep_scrub_stamp":"2024-05-27T09:37:47.648025+0000","last_clean_scrub_stamp":"2024-05-27T09:37:47.648025+0000","objects_scrubbed":7,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:04:18.497254+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":15,"num_read_kb":12312,"num_write":136,"num_write_kb":12381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":22371,"num_omap_keys":39,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"50'471","reported_seq":2611,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:58.019968+0000","last_change":"2024-05-27T09:37:58.019998+0000","last_active":"2024-05-27T09:37:58.019968+0000","last_peered":"2024-05-27T09:37:58.019968+0000","last_clean":"2024-05-27T09:37:58.019968+0000","last_became_active":"2024-05-27T09:27:18.562919+0000","last_became_peered":"2024-05-27T09:27:18.562919+0000","last_unstale":"2024-05-27T09:37:58.019968+0000","last_undegraded":"2024-05-27T09:37:58.019968+0000","last_fullsized":"2024-05-27T09:37:58.019968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'471","last_scrub_stamp":"2024-05-27T09:37:58.019892+0000","last_deep_scrub":"50'471","last_deep_scrub_stamp":"2024-05-27T09:37:58.019892+0000","last_clean_scrub_stamp":"2024-05-27T09:37:58.019892+0000","objects_scrubbed":152,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:18:29.371418+0000","scrub_duration":286,"objects_trimmed":0,"snaptrim_duration":0.0063913879999999996,"stat_sum":{"num_bytes":22598856,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":21,"num_read_kb":8211,"num_write":313,"num_write_kb":22222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"50'89","reported_seq":176,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:52.847001+0000","last_change":"2024-05-27T09:37:52.847033+0000","last_active":"2024-05-27T09:37:52.847001+0000","last_peered":"2024-05-27T09:37:52.847001+0000","last_clean":"2024-05-27T09:37:52.847001+0000","last_became_active":"2024-05-27T09:27:17.550849+0000","last_became_peered":"2024-05-27T09:27:17.550849+0000","last_unstale":"2024-05-27T09:37:52.847001+0000","last_undegraded":"2024-05-27T09:37:52.847001+0000","last_fullsized":"2024-05-27T09:37:52.847001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'89","last_scrub_stamp":"2024-05-27T09:37:52.846920+0000","last_deep_scrub":"50'89","last_deep_scrub_stamp":"2024-05-27T09:37:52.846920+0000","last_clean_scrub_stamp":"2024-05-27T09:37:52.846920+0000","objects_scrubbed":2,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:50:20.087476+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":81,"num_write_kb":4153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9993,"num_omap_keys":17,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"50'492","reported_seq":2625,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:38:04.925825+0000","last_change":"2024-05-27T09:38:04.925889+0000","last_active":"2024-05-27T09:38:04.925825+0000","last_peered":"2024-05-27T09:38:04.925825+0000","last_clean":"2024-05-27T09:38:04.925825+0000","last_became_active":"2024-05-27T09:27:18.579574+0000","last_became_peered":"2024-05-27T09:27:18.579574+0000","last_unstale":"2024-05-27T09:38:04.925825+0000","last_undegraded":"2024-05-27T09:38:04.925825+0000","last_fullsized":"2024-05-27T09:38:04.925825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'492","last_scrub_stamp":"2024-05-27T09:38:04.925729+0000","last_deep_scrub":"50'492","last_deep_scrub_stamp":"2024-05-27T09:38:04.925729+0000","last_clean_scrub_stamp":"2024-05-27T09:38:04.925729+0000","objects_scrubbed":156,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:08:16.277076+0000","scrub_duration":232,"objects_trimmed":0,"snaptrim_duration":0.0062209780000000003,"stat_sum":{"num_bytes":14492175,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":14,"num_read_kb":4108,"num_write":330,"num_write_kb":14321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"50'93","reported_seq":193,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-05-27T09:37:50.460509+0000","last_change":"2024-05-27T09:37:50.460545+0000","last_active":"2024-05-27T09:37:50.460509+0000","last_peered":"2024-05-27T09:37:50.460509+0000","last_clean":"2024-05-27T09:37:50.460509+0000","last_became_active":"2024-05-27T09:27:17.549291+0000","last_became_peered":"2024-05-27T09:27:17.549291+0000","last_unstale":"2024-05-27T09:37:50.460509+0000","last_undegraded":"2024-05-27T09:37:50.460509+0000","last_fullsized":"2024-05-27T09:37:50.460509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'93","last_scrub_stamp":"2024-05-27T09:37:50.460440+0000","last_deep_scrub":"50'93","last_deep_scrub_stamp":"2024-05-27T09:37:50.460440+0000","last_clean_scrub_stamp":"2024-05-27T09:37:50.460440+0000","objects_scrubbed":11,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:49:00.311556+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":41,"num_read_kb":4136,"num_write":85,"num_write_kb":8333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":38631,"num_omap_keys":65,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":952881208,"num_objects":10697,"num_object_clones":0,"num_object_copies":21394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10697,"num_whiteouts":0,"num_read":1250,"num_read_kb":378565,"num_write":21433,"num_write_kb":1379166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1734066176,"data_stored":1904222320,"data_compressed":168711384,"data_compressed_allocated":186425344,"data_compressed_original":412123136,"omap_allocated":0,"internal_metadata":0},"log_size":32583,"ondisk_log_size":32583,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":188411414,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":557,"num_read_kb":123805,"num_write":4458,"num_write_kb":189478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":105,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":668876,"num_omap_keys":1129,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":84320256,"data_stored":376822828,"data_compressed":49815968,"data_compressed_allocated":62562304,"data_compressed_original":355221504,"omap_allocated":0,"internal_metadata":0},"log_size":4327,"ondisk_log_size":4327,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65078,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509576,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":372912,"kb_used_data":309800,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":93998928,"statfs":{"total":96636764160,"available":96254902272,"internally_reserved":0,"allocated":317235200,"data_stored":412122108,"data_compressed":41350796,"data_compressed_allocated":47357952,"data_compressed_original":150941696,"omap_allocated":1593,"internal_metadata":64616903},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509576,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308644,"kb_used_data":233052,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94063196,"statfs":{"total":96636764160,"available":96320712704,"internally_reserved":0,"allocated":238645248,"data_stored":280794177,"data_compressed":23230909,"data_compressed_allocated":26537984,"data_compressed_original":76705792,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509576,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254808,"kb_used_data":201424,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94117032,"statfs":{"total":96636764160,"available":96375840768,"internally_reserved":0,"allocated":206258176,"data_stored":257021246,"data_compressed":23229218,"data_compressed_allocated":26480640,"data_compressed_original":83582976,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509576,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":348544,"kb_used_data":285432,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94023296,"statfs":{"total":96636764160,"available":96279855104,"internally_reserved":0,"allocated":292282368,"data_stored":379742172,"data_compressed":39531620,"data_compressed_allocated":45047808,"data_compressed_original":141885440,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542279,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":161328,"kb_used_data":130152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94210512,"statfs":{"total":96636764160,"available":96471564288,"internally_reserved":0,"allocated":133275648,"data_stored":168154774,"data_compressed":18202841,"data_compressed_allocated":20492288,"data_compressed_original":60579840,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542279,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":204556,"kb_used_data":147012,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94167284,"statfs":{"total":96636764160,"available":96427298816,"internally_reserved":0,"allocated":150540288,"data_stored":186187785,"data_compressed":18876629,"data_compressed_allocated":21614592,"data_compressed_original":63655936,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542279,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":368692,"kb_used_data":305580,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94003148,"statfs":{"total":96636764160,"available":96259223552,"internally_reserved":0,"allocated":312913920,"data_stored":372795540,"data_compressed":31472041,"data_compressed_allocated":35520512,"data_compressed_original":103694336,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542279,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":212924,"kb_used_data":167860,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94158916,"statfs":{"total":96636764160,"available":96418729984,"internally_reserved":0,"allocated":171888640,"data_stored":227475258,"data_compressed":22698376,"data_compressed_allocated":26025984,"data_compressed_original":87445504,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32539,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13996032,"data_stored":67110923,"data_compressed":8258411,"data_compressed_allocated":10555392,"data_compressed_original":63688704,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7446528,"data_stored":37748896,"data_compressed":4390710,"data_compressed_allocated":5623808,"data_compressed_original":35934208,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7921664,"data_stored":33555138,"data_compressed":4517094,"data_compressed_allocated":5791744,"data_compressed_original":31436800,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19890176,"data_stored":83888049,"data_compressed":11332765,"data_compressed_allocated":14295040,"data_compressed_original":78307328,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12206080,"data_stored":50333888,"data_compressed":7575831,"data_compressed_allocated":9129984,"data_compressed_original":47296512,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10158080,"data_stored":45800857,"data_compressed":5688921,"data_compressed_allocated":7184384,"data_compressed_original":42852352,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7270400,"data_stored":32684055,"data_compressed":4506622,"data_compressed_allocated":5656576,"data_compressed_original":31084544,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5431296,"data_stored":25701022,"data_compressed":3545614,"data_compressed_allocated":4325376,"data_compressed_original":24621056,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":277721088,"data_stored":312372852,"data_compressed":31273209,"data_compressed_allocated":34492416,"data_compressed_original":78196736,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":198180864,"data_stored":218423585,"data_compressed":18805969,"data_compressed_allocated":20811776,"data_compressed_original":47075328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":230158336,"data_stored":246980642,"data_compressed":18713815,"data_compressed_allocated":20746240,"data_compressed_original":45268992,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":296779776,"data_stored":327975662,"data_compressed":30018031,"data_compressed_allocated":33062912,"data_compressed_original":72634368,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":159117312,"data_stored":176882973,"data_compressed":15122545,"data_compressed_allocated":16896000,"data_compressed_original":40148992,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":302125056,"data_stored":326145918,"data_compressed":25750581,"data_compressed_allocated":28291072,"data_compressed_original":60268544,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":142704640,"data_stored":153245333,"data_compressed":14370007,"data_compressed_allocated":15958016,"data_compressed_original":32571392,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":127279104,"data_stored":142195355,"data_compressed":14657227,"data_compressed_allocated":16166912,"data_compressed_original":35958784,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T09:38:13.433 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-27T09:38:13.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.741+0000 7f2f73800640 1 Processor -- start 2024-05-27T09:38:13.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.741+0000 7f2f73800640 1 -- start start 2024-05-27T09:38:13.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.743+0000 7f2f73800640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2f6c154670 0x7f2f6c15de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:13.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.743+0000 7f2f73800640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f6c154a80 con 0x7f2f6c074a10 2024-05-27T09:38:13.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.744+0000 7f2f73800640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f6c0c9040 con 0x7f2f6c15e360 2024-05-27T09:38:13.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.744+0000 7f2f73800640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7f2f6c0c9320 con 0x7f2f6c154670 2024-05-27T09:38:13.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.744+0000 7f2f70e00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7f2f6c074a10 0x7f2f6c073260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:41644/0 (socket says 172.21.15.114:41644) 2024-05-27T09:38:13.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.744+0000 7f2f70e00640 1 -- 172.21.15.114:0/2488150086 learned_addr learned my addr 172.21.15.114:0/2488150086 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:38:13.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.744+0000 7f2f6be00640 1 --2- 172.21.15.114:0/2488150086 >> v2:172.21.15.163:3300/0 conn(0x7f2f6c154670 0x7f2f6c15de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:13.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.744+0000 7f2f6b400640 1 -- 172.21.15.114:0/2488150086 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3541754403 0 0) 0x7f2f6c0c9040 con 0x7f2f6c15e360 2024-05-27T09:38:13.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.744+0000 7f2f6b400640 1 -- 172.21.15.114:0/2488150086 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f54003520 con 0x7f2f6c15e360 2024-05-27T09:38:13.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.744+0000 7f2f6be00640 1 -- 172.21.15.114:0/2488150086 >> v1:172.21.15.163:6789/0 conn(0x7f2f6c15e360 legacy=0x7f2f6c0ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:13.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.744+0000 7f2f6b400640 1 -- 172.21.15.114:0/2488150086 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2250159503 0 0) 0x7f2f6c154a80 con 0x7f2f6c074a10 2024-05-27T09:38:13.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.744+0000 7f2f6be00640 1 -- 172.21.15.114:0/2488150086 >> v1:172.21.15.114:6789/0 conn(0x7f2f6c074a10 legacy=0x7f2f6c073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:13.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.744+0000 7f2f6be00640 1 -- 172.21.15.114:0/2488150086 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c0c9600 con 0x7f2f6c154670 2024-05-27T09:38:13.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.746+0000 7f2f6be00640 1 --2- 172.21.15.114:0/2488150086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f6c154670 0x7f2f6c15de20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2f580097a0 tx=0x7f2f5802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37a67c07c798aeb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:13.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.746+0000 7f2f6b400640 1 -- 172.21.15.114:0/2488150086 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f58037070 con 0x7f2f6c154670 2024-05-27T09:38:13.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.746+0000 7f2f6b400640 1 -- 172.21.15.114:0/2488150086 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f58032440 con 0x7f2f6c154670 2024-05-27T09:38:13.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.746+0000 7f2f6b400640 1 -- 172.21.15.114:0/2488150086 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f580327f0 con 0x7f2f6c154670 2024-05-27T09:38:13.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.747+0000 7f2f73800640 1 -- 172.21.15.114:0/2488150086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f6c154670 msgr2=0x7f2f6c15de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:13.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.747+0000 7f2f73800640 1 --2- 172.21.15.114:0/2488150086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f6c154670 0x7f2f6c15de20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2f580097a0 tx=0x7f2f5802a230 comp rx=0 tx=0).stop 2024-05-27T09:38:13.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.747+0000 7f2f73800640 1 -- 172.21.15.114:0/2488150086 shutdown_connections 2024-05-27T09:38:13.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.747+0000 7f2f73800640 1 --2- 172.21.15.114:0/2488150086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f6c154670 0x7f2f6c15de20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:13.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.747+0000 7f2f73800640 1 -- 172.21.15.114:0/2488150086 >> 172.21.15.114:0/2488150086 conn(0x7f2f6c06f6e0 msgr2=0x7f2f6c070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:13.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.747+0000 7f2f73800640 1 -- 172.21.15.114:0/2488150086 shutdown_connections 2024-05-27T09:38:13.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.747+0000 7f2f73800640 1 -- 172.21.15.114:0/2488150086 wait complete. 2024-05-27T09:38:13.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.749+0000 7f2f73800640 1 Processor -- start 2024-05-27T09:38:13.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.749+0000 7f2f73800640 1 -- start start 2024-05-27T09:38:13.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.751+0000 7f2f73800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f6c074a10 0x7f2f6c159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:13.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.751+0000 7f2f73800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f6c154670 0x7f2f6c1f2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:13.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.751+0000 7f2f73800640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f6c15e360 0x7f2f6c0c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:13.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.751+0000 7f2f73800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c0c9bc0 con 0x7f2f6c074a10 2024-05-27T09:38:13.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.751+0000 7f2f73800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c15e770 con 0x7f2f6c154670 2024-05-27T09:38:13.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.751+0000 7f2f73800640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6c0c9d40 con 0x7f2f6c15e360 2024-05-27T09:38:13.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.751+0000 7f2f70e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f6c074a10 0x7f2f6c159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:13.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.751+0000 7f2f70e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f6c074a10 0x7f2f6c159160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59550/0 (socket says 172.21.15.114:59550) 2024-05-27T09:38:13.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.751+0000 7f2f70e00640 1 -- 172.21.15.114:0/145062097 learned_addr learned my addr 172.21.15.114:0/145062097 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:38:13.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.751+0000 7f2f6be00640 1 --2- 172.21.15.114:0/145062097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f6c154670 0x7f2f6c1f2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:13.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.751+0000 7f2f71800640 1 --2- 172.21.15.114:0/145062097 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f6c15e360 0x7f2f6c0c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:13.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.752+0000 7f2f6be00640 1 -- 172.21.15.114:0/145062097 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f6c15e360 msgr2=0x7f2f6c0c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:13.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.752+0000 7f2f6be00640 1 --2- 172.21.15.114:0/145062097 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f6c15e360 0x7f2f6c0c70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:13.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.752+0000 7f2f6be00640 1 -- 172.21.15.114:0/145062097 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f6c074a10 msgr2=0x7f2f6c159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:13.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.752+0000 7f2f6be00640 1 --2- 172.21.15.114:0/145062097 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f6c074a10 0x7f2f6c159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:13.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.752+0000 7f2f6be00640 1 -- 172.21.15.114:0/145062097 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c1f2980 con 0x7f2f6c154670 2024-05-27T09:38:13.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.752+0000 7f2f70e00640 1 --2- 172.21.15.114:0/145062097 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f6c074a10 0x7f2f6c159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:38:13.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.752+0000 7f2f6be00640 1 --2- 172.21.15.114:0/145062097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f6c154670 0x7f2f6c1f2440 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c06d830 tx=0x7f2f580046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:13.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.752+0000 7f2f68c00640 1 -- 172.21.15.114:0/145062097 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f58041020 con 0x7f2f6c154670 2024-05-27T09:38:13.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.752+0000 7f2f68c00640 1 -- 172.21.15.114:0/145062097 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f58030700 con 0x7f2f6c154670 2024-05-27T09:38:13.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.752+0000 7f2f68c00640 1 -- 172.21.15.114:0/145062097 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f58037040 con 0x7f2f6c154670 2024-05-27T09:38:13.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.752+0000 7f2f73800640 1 -- 172.21.15.114:0/145062097 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6c15aa70 con 0x7f2f6c154670 2024-05-27T09:38:13.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.753+0000 7f2f73800640 1 -- 172.21.15.114:0/145062097 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f6c15ad10 con 0x7f2f6c154670 2024-05-27T09:38:13.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.754+0000 7f2f73800640 1 -- 172.21.15.114:0/145062097 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f6c06a380 con 0x7f2f6c154670 2024-05-27T09:38:13.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.755+0000 7f2f68c00640 1 -- 172.21.15.114:0/145062097 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2f58044020 con 0x7f2f6c154670 2024-05-27T09:38:13.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.756+0000 7f2f68c00640 1 --2- 172.21.15.114:0/145062097 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2f4c0762d0 0x7f2f4c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:13.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.756+0000 7f2f68c00640 1 -- 172.21.15.114:0/145062097 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7f2f580b7f00 con 0x7f2f6c154670 2024-05-27T09:38:13.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.756+0000 7f2f70e00640 1 --2- 172.21.15.114:0/145062097 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2f4c0762d0 0x7f2f4c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:13.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.757+0000 7f2f70e00640 1 --2- 172.21.15.114:0/145062097 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2f4c0762d0 0x7f2f4c0787e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2f60000f60 tx=0x7f2f60009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:13.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.762+0000 7f2f68c00640 1 -- 172.21.15.114:0/145062097 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2f5804b030 con 0x7f2f6c154670 2024-05-27T09:38:13.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.969+0000 7f2f73800640 1 -- 172.21.15.114:0/145062097 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f2f6c1f21a0 con 0x7f2f6c154670 2024-05-27T09:38:13.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.977+0000 7f2f68c00640 1 -- 172.21.15.114:0/145062097 <== mon.1 v2:172.21.15.163:3300/0 7 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f2f58080a60 con 0x7f2f6c154670 2024-05-27T09:38:13.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.978+0000 7f2f68c00640 1 -- 172.21.15.114:0/145062097 <== mon.1 v2:172.21.15.163:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) ==== 123+0+0 (secure 0 0 0) 0x7f2f58080f50 con 0x7f2f6c154670 2024-05-27T09:38:13.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.981+0000 7f2f73800640 1 -- 172.21.15.114:0/145062097 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2f4c0762d0 msgr2=0x7f2f4c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:13.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.981+0000 7f2f73800640 1 --2- 172.21.15.114:0/145062097 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2f4c0762d0 0x7f2f4c0787e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2f60000f60 tx=0x7f2f60009040 comp rx=0 tx=0).stop 2024-05-27T09:38:13.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.981+0000 7f2f73800640 1 -- 172.21.15.114:0/145062097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f6c154670 msgr2=0x7f2f6c1f2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:13.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.981+0000 7f2f73800640 1 --2- 172.21.15.114:0/145062097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f6c154670 0x7f2f6c1f2440 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c06d830 tx=0x7f2f580046c0 comp rx=0 tx=0).stop 2024-05-27T09:38:13.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.981+0000 7f2f73800640 1 -- 172.21.15.114:0/145062097 shutdown_connections 2024-05-27T09:38:13.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.981+0000 7f2f73800640 1 --2- 172.21.15.114:0/145062097 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7f2f4c0762d0 0x7f2f4c0787e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:13.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.981+0000 7f2f73800640 1 --2- 172.21.15.114:0/145062097 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f6c15e360 0x7f2f6c0c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:13.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.981+0000 7f2f73800640 1 --2- 172.21.15.114:0/145062097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f6c154670 0x7f2f6c1f2440 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:13.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.981+0000 7f2f73800640 1 --2- 172.21.15.114:0/145062097 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2f6c074a10 0x7f2f6c159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:13.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.981+0000 7f2f73800640 1 -- 172.21.15.114:0/145062097 >> 172.21.15.114:0/145062097 conn(0x7f2f6c06f6e0 msgr2=0x7f2f6c0cb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:13.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.981+0000 7f2f73800640 1 -- 172.21.15.114:0/145062097 shutdown_connections 2024-05-27T09:38:13.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:13.982+0000 7f2f73800640 1 -- 172.21.15.114:0/145062097 wait complete. 2024-05-27T09:38:14.012 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-27T09:38:14.012 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:38:14.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.281+0000 7fa210e00640 1 Processor -- start 2024-05-27T09:38:14.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.281+0000 7fa210e00640 1 -- start start 2024-05-27T09:38:14.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.283+0000 7fa210e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa20c154820 0x7fa20c0ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:14.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.284+0000 7fa210e00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa20c0cac20 con 0x7fa20c0ca810 2024-05-27T09:38:14.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.284+0000 7fa210e00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa20c15a810 con 0x7fa20c074a60 2024-05-27T09:38:14.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.284+0000 7fa210e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa20c154c30 con 0x7fa20c154820 2024-05-27T09:38:14.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.284+0000 7fa20a800640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fa20c0ca810 0x7fa20c15dff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:41658/0 (socket says 172.21.15.114:41658) 2024-05-27T09:38:14.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.284+0000 7fa203e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa20c154820 0x7fa20c0ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:14.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.284+0000 7fa20a800640 1 -- 172.21.15.114:0/2934866279 learned_addr learned my addr 172.21.15.114:0/2934866279 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:38:14.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.284+0000 7fa203e00640 1 -- 172.21.15.114:0/2934866279 >> v1:172.21.15.163:6789/0 conn(0x7fa20c074a60 legacy=0x7fa20c073260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:14.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.284+0000 7fa203e00640 1 -- 172.21.15.114:0/2934866279 >> v1:172.21.15.114:6789/0 conn(0x7fa20c0ca810 legacy=0x7fa20c15dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:14.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.285+0000 7fa203e00640 1 -- 172.21.15.114:0/2934866279 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20c15aaf0 con 0x7fa20c154820 2024-05-27T09:38:14.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.285+0000 7fa209400640 1 -- 172.21.15.114:0/2934866279 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2548552954 0 0) 0x7fa20c0cac20 con 0x7fa20c0ca810 2024-05-27T09:38:14.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.286+0000 7fa203e00640 1 --2- 172.21.15.114:0/2934866279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa20c154820 0x7fa20c0ca2d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa1f40097d0 tx=0x7fa1f402a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51540be10460ec0f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:14.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.286+0000 7fa209400640 1 -- 172.21.15.114:0/2934866279 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4037070 con 0x7fa20c154820 2024-05-27T09:38:14.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.286+0000 7fa209400640 1 -- 172.21.15.114:0/2934866279 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa1f4032440 con 0x7fa20c154820 2024-05-27T09:38:14.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.286+0000 7fa209400640 1 -- 172.21.15.114:0/2934866279 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4032810 con 0x7fa20c154820 2024-05-27T09:38:14.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.287+0000 7fa210e00640 1 -- 172.21.15.114:0/2934866279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa20c154820 msgr2=0x7fa20c0ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:14.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.287+0000 7fa210e00640 1 --2- 172.21.15.114:0/2934866279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa20c154820 0x7fa20c0ca2d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa1f40097d0 tx=0x7fa1f402a290 comp rx=0 tx=0).stop 2024-05-27T09:38:14.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.287+0000 7fa210e00640 1 -- 172.21.15.114:0/2934866279 shutdown_connections 2024-05-27T09:38:14.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.287+0000 7fa210e00640 1 --2- 172.21.15.114:0/2934866279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa20c154820 0x7fa20c0ca2d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:14.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.287+0000 7fa210e00640 1 -- 172.21.15.114:0/2934866279 >> 172.21.15.114:0/2934866279 conn(0x7fa20c06f6e0 msgr2=0x7fa20c070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:14.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.287+0000 7fa210e00640 1 -- 172.21.15.114:0/2934866279 shutdown_connections 2024-05-27T09:38:14.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.287+0000 7fa210e00640 1 -- 172.21.15.114:0/2934866279 wait complete. 2024-05-27T09:38:14.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.289+0000 7fa210e00640 1 Processor -- start 2024-05-27T09:38:14.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.289+0000 7fa210e00640 1 -- start start 2024-05-27T09:38:14.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.291+0000 7fa210e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa20c074a60 0x7fa20c14fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:14.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.291+0000 7fa210e00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa20c0ca810 0x7fa20c14a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:14.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.291+0000 7fa210e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa20c154820 0x7fa20c0c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:14.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.291+0000 7fa210e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c15b0b0 con 0x7fa20c074a60 2024-05-27T09:38:14.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.291+0000 7fa210e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c15db60 con 0x7fa20c154820 2024-05-27T09:38:14.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.291+0000 7fa210e00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fa20c15b390 con 0x7fa20c0ca810 2024-05-27T09:38:14.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.291+0000 7fa209e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa20c074a60 0x7fa20c14fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:14.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.291+0000 7fa209e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa20c074a60 0x7fa20c14fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59560/0 (socket says 172.21.15.114:59560) 2024-05-27T09:38:14.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.291+0000 7fa209e00640 1 -- 172.21.15.114:0/67591494 learned_addr learned my addr 172.21.15.114:0/67591494 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:38:14.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.291+0000 7fa203e00640 1 --2- 172.21.15.114:0/67591494 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa20c0ca810 0x7fa20c14a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:14.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.291+0000 7fa20a800640 1 --2- 172.21.15.114:0/67591494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa20c154820 0x7fa20c0c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:14.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.292+0000 7fa209e00640 1 -- 172.21.15.114:0/67591494 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa20c0ca810 msgr2=0x7fa20c14a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:14.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.292+0000 7fa209e00640 1 --2- 172.21.15.114:0/67591494 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa20c0ca810 0x7fa20c14a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:14.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.292+0000 7fa209e00640 1 -- 172.21.15.114:0/67591494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa20c154820 msgr2=0x7fa20c0c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:14.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.292+0000 7fa209e00640 1 --2- 172.21.15.114:0/67591494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa20c154820 0x7fa20c0c6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:14.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.292+0000 7fa209e00640 1 -- 172.21.15.114:0/67591494 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20c0cb810 con 0x7fa20c074a60 2024-05-27T09:38:14.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.292+0000 7fa203e00640 1 --2- 172.21.15.114:0/67591494 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa20c0ca810 0x7fa20c14a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:38:14.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.292+0000 7fa209e00640 1 --2- 172.21.15.114:0/67591494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa20c074a60 0x7fa20c14fee0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00c430 tx=0x7fa1fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:14.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.292+0000 7fa202000640 1 -- 172.21.15.114:0/67591494 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc017070 con 0x7fa20c074a60 2024-05-27T09:38:14.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.292+0000 7fa202000640 1 -- 172.21.15.114:0/67591494 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa1fc0052d0 con 0x7fa20c074a60 2024-05-27T09:38:14.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.292+0000 7fa210e00640 1 -- 172.21.15.114:0/67591494 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20c14abc0 con 0x7fa20c074a60 2024-05-27T09:38:14.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.293+0000 7fa202000640 1 -- 172.21.15.114:0/67591494 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc003850 con 0x7fa20c074a60 2024-05-27T09:38:14.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.293+0000 7fa210e00640 1 -- 172.21.15.114:0/67591494 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa20c0cbd60 con 0x7fa20c074a60 2024-05-27T09:38:14.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.294+0000 7fa210e00640 1 -- 172.21.15.114:0/67591494 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa20c06a380 con 0x7fa20c074a60 2024-05-27T09:38:14.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.295+0000 7fa202000640 1 -- 172.21.15.114:0/67591494 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa1fc01a070 con 0x7fa20c074a60 2024-05-27T09:38:14.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.296+0000 7fa202000640 1 --2- 172.21.15.114:0/67591494 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa1e80762d0 0x7fa1e80787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:14.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.296+0000 7fa202000640 1 -- 172.21.15.114:0/67591494 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7fa1fc096cf0 con 0x7fa20c074a60 2024-05-27T09:38:14.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.297+0000 7fa203e00640 1 --2- 172.21.15.114:0/67591494 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa1e80762d0 0x7fa1e80787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:14.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.297+0000 7fa203e00640 1 --2- 172.21.15.114:0/67591494 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa1e80762d0 0x7fa1e80787e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4034000 tx=0x7fa1f402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:14.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.301+0000 7fa202000640 1 -- 172.21.15.114:0/67591494 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa20c14abc0 con 0x7fa20c074a60 2024-05-27T09:38:14.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.547+0000 7fa210e00640 1 -- 172.21.15.114:0/67591494 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa20c073ca0 con 0x7fa20c074a60 2024-05-27T09:38:14.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.548+0000 7fa202000640 1 -- 172.21.15.114:0/67591494 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+6062 (secure 0 0 0) 0x7fa1fc068380 con 0x7fa20c074a60 2024-05-27T09:38:14.550 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:38:14.550 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:37:36.353158+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":5072,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5072":{"gid":5072,"name":"f","rank":1,"incarnation":43,"state":"up:active","state_seq":27,"addr":"172.21.15.163:6839/1057092553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":1057092553},{"type":"v1","addr":"172.21.15.163:6839","nonce":1057092553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:active","state_seq":26,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5072,5086,5160]},"id":1}]} 2024-05-27T09:38:14.550 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 47 2024-05-27T09:38:14.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.551+0000 7fa210e00640 1 -- 172.21.15.114:0/67591494 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa1e80762d0 msgr2=0x7fa1e80787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:14.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.551+0000 7fa210e00640 1 --2- 172.21.15.114:0/67591494 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa1e80762d0 0x7fa1e80787e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4034000 tx=0x7fa1f402e000 comp rx=0 tx=0).stop 2024-05-27T09:38:14.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.551+0000 7fa210e00640 1 -- 172.21.15.114:0/67591494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa20c074a60 msgr2=0x7fa20c14fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:14.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.551+0000 7fa210e00640 1 --2- 172.21.15.114:0/67591494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa20c074a60 0x7fa20c14fee0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00c430 tx=0x7fa1fc00c8f0 comp rx=0 tx=0).stop 2024-05-27T09:38:14.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.551+0000 7fa210e00640 1 -- 172.21.15.114:0/67591494 shutdown_connections 2024-05-27T09:38:14.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.551+0000 7fa210e00640 1 --2- 172.21.15.114:0/67591494 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fa1e80762d0 0x7fa1e80787e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:14.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.551+0000 7fa210e00640 1 --2- 172.21.15.114:0/67591494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa20c154820 0x7fa20c0c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:14.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.551+0000 7fa210e00640 1 --2- 172.21.15.114:0/67591494 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa20c0ca810 0x7fa20c14a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:14.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.551+0000 7fa210e00640 1 --2- 172.21.15.114:0/67591494 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa20c074a60 0x7fa20c14fee0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:14.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.551+0000 7fa210e00640 1 -- 172.21.15.114:0/67591494 >> 172.21.15.114:0/67591494 conn(0x7fa20c06f6e0 msgr2=0x7fa20c152380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:14.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.551+0000 7fa210e00640 1 -- 172.21.15.114:0/67591494 shutdown_connections 2024-05-27T09:38:14.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.551+0000 7fa210e00640 1 -- 172.21.15.114:0/67591494 wait complete. 2024-05-27T09:38:14.578 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-27T09:38:14.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.840+0000 7fe1df200640 1 Processor -- start 2024-05-27T09:38:14.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.840+0000 7fe1df200640 1 -- start start 2024-05-27T09:38:14.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.841+0000 7fe1df200640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe1e01534c0 0x7fe1e00c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:14.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.841+0000 7fe1df200640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1e00c96d0 con 0x7fe1e00c8220 2024-05-27T09:38:14.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.841+0000 7fe1df200640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1e00c9970 con 0x7fe1e00c8630 2024-05-27T09:38:14.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.841+0000 7fe1df200640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fe1e00c2f40 con 0x7fe1e01534c0 2024-05-27T09:38:14.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.841+0000 7fe1dde00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe1e01534c0 0x7fe1e00c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:14.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.841+0000 7fe1dd400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fe1e00c8220 0x7fe1e0159900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:38118/0 (socket says 172.21.15.114:38118) 2024-05-27T09:38:14.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.841+0000 7fe1dd400640 1 -- 172.21.15.114:0/3050004566 learned_addr learned my addr 172.21.15.114:0/3050004566 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:38:14.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.842+0000 7fe1dde00640 1 -- 172.21.15.114:0/3050004566 >> v1:172.21.15.163:6789/0 conn(0x7fe1e00c8630 legacy=0x7fe1e0074140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:14.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.842+0000 7fe1dde00640 1 -- 172.21.15.114:0/3050004566 >> v1:172.21.15.114:6789/0 conn(0x7fe1e00c8220 legacy=0x7fe1e0159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:14.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.842+0000 7fe1dde00640 1 -- 172.21.15.114:0/3050004566 --> v2:172.21.15.114:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e00c9c50 con 0x7fe1e01534c0 2024-05-27T09:38:14.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.842+0000 7fe1dca00640 1 -- 172.21.15.114:0/3050004566 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1833277255 0 0) 0x7fe1e00c96d0 con 0x7fe1e00c8220 2024-05-27T09:38:14.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.843+0000 7fe1dde00640 1 --2- 172.21.15.114:0/3050004566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1e01534c0 0x7fe1e00c2a00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00097a0 tx=0x7fe1d002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe4b102beb0163c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:14.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.843+0000 7fe1dca00640 1 -- 172.21.15.114:0/3050004566 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0037070 con 0x7fe1e01534c0 2024-05-27T09:38:14.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.843+0000 7fe1dca00640 1 -- 172.21.15.114:0/3050004566 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe1d0032440 con 0x7fe1e01534c0 2024-05-27T09:38:14.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.843+0000 7fe1dca00640 1 -- 172.21.15.114:0/3050004566 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0032830 con 0x7fe1e01534c0 2024-05-27T09:38:14.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.844+0000 7fe1df200640 1 -- 172.21.15.114:0/3050004566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1e01534c0 msgr2=0x7fe1e00c2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:14.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.844+0000 7fe1df200640 1 --2- 172.21.15.114:0/3050004566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1e01534c0 0x7fe1e00c2a00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00097a0 tx=0x7fe1d002a230 comp rx=0 tx=0).stop 2024-05-27T09:38:14.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.844+0000 7fe1df200640 1 -- 172.21.15.114:0/3050004566 shutdown_connections 2024-05-27T09:38:14.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.844+0000 7fe1df200640 1 --2- 172.21.15.114:0/3050004566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1e01534c0 0x7fe1e00c2a00 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:14.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.844+0000 7fe1df200640 1 -- 172.21.15.114:0/3050004566 >> 172.21.15.114:0/3050004566 conn(0x7fe1e00ca320 msgr2=0x7fe1e00ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:14.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.844+0000 7fe1df200640 1 -- 172.21.15.114:0/3050004566 shutdown_connections 2024-05-27T09:38:14.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.844+0000 7fe1df200640 1 -- 172.21.15.114:0/3050004566 wait complete. 2024-05-27T09:38:14.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.846+0000 7fe1df200640 1 Processor -- start 2024-05-27T09:38:14.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.846+0000 7fe1df200640 1 -- start start 2024-05-27T09:38:14.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.848+0000 7fe1df200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e00c8220 0x7fe1e014a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:14.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.848+0000 7fe1df200640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e00c8630 0x7fe1e014c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:14.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.848+0000 7fe1df200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1e01534c0 0x7fe1e01512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:14.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.848+0000 7fe1df200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e01538d0 con 0x7fe1e01534c0 2024-05-27T09:38:14.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.848+0000 7fe1df200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e0159190 con 0x7fe1e00c8220 2024-05-27T09:38:14.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.848+0000 7fe1df200640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e01593e0 con 0x7fe1e00c8630 2024-05-27T09:38:14.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.848+0000 7fe1dde00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e00c8220 0x7fe1e014a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.848+0000 7fe1dd400640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e00c8630 0x7fe1e014c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.848+0000 7fe1de800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1e01534c0 0x7fe1e01512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.848+0000 7fe1dde00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e00c8220 0x7fe1e014a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:55316/0 (socket says 172.21.15.114:55316) 2024-05-27T09:38:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.848+0000 7fe1dde00640 1 -- 172.21.15.114:0/976402096 learned_addr learned my addr 172.21.15.114:0/976402096 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:38:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.849+0000 7fe1dd400640 1 -- 172.21.15.114:0/976402096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e00c8220 msgr2=0x7fe1e014a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.849+0000 7fe1dd400640 1 --2- 172.21.15.114:0/976402096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e00c8220 0x7fe1e014a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.849+0000 7fe1dd400640 1 -- 172.21.15.114:0/976402096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1e01534c0 msgr2=0x7fe1e01512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.849+0000 7fe1dd400640 1 --2- 172.21.15.114:0/976402096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1e01534c0 0x7fe1e01512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.849+0000 7fe1dd400640 1 -- 172.21.15.114:0/976402096 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e014c8b0 con 0x7fe1e00c8630 2024-05-27T09:38:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.849+0000 7fe1dde00640 1 --2- 172.21.15.114:0/976402096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e00c8220 0x7fe1e014a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:38:14.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.849+0000 7fe1de800640 1 --2- 172.21.15.114:0/976402096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1e01534c0 0x7fe1e01512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:38:14.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.849+0000 7fe1dd400640 1 --2- 172.21.15.114:0/976402096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e00c8630 0x7fe1e014c370 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00c390 tx=0x7fe1cc00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:14.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.849+0000 7fe1d6000640 1 -- 172.21.15.114:0/976402096 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc0074e0 con 0x7fe1e00c8630 2024-05-27T09:38:14.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.849+0000 7fe1d6000640 1 -- 172.21.15.114:0/976402096 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe1cc0047e0 con 0x7fe1e00c8630 2024-05-27T09:38:14.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.849+0000 7fe1df200640 1 -- 172.21.15.114:0/976402096 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e0150e70 con 0x7fe1e00c8630 2024-05-27T09:38:14.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.849+0000 7fe1d6000640 1 -- 172.21.15.114:0/976402096 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc004bf0 con 0x7fe1e00c8630 2024-05-27T09:38:14.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.849+0000 7fe1df200640 1 -- 172.21.15.114:0/976402096 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e014b2d0 con 0x7fe1e00c8630 2024-05-27T09:38:14.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.850+0000 7fe1df200640 1 -- 172.21.15.114:0/976402096 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1e0050860 con 0x7fe1e00c8630 2024-05-27T09:38:14.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.852+0000 7fe1d6000640 1 -- 172.21.15.114:0/976402096 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe1cc011070 con 0x7fe1e00c8630 2024-05-27T09:38:14.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.853+0000 7fe1d6000640 1 --2- 172.21.15.114:0/976402096 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe1c00762d0 0x7fe1c00787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:14.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.853+0000 7fe1d6000640 1 -- 172.21.15.114:0/976402096 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7fe1cc0976a0 con 0x7fe1e00c8630 2024-05-27T09:38:14.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.853+0000 7fe1dde00640 1 --2- 172.21.15.114:0/976402096 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe1c00762d0 0x7fe1c00787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:14.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.854+0000 7fe1dde00640 1 --2- 172.21.15.114:0/976402096 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe1c00762d0 0x7fe1c00787e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00097a0 tx=0x7fe1d002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:14.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:14.858+0000 7fe1d6000640 1 -- 172.21.15.114:0/976402096 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe1cc060800 con 0x7fe1e00c8630 2024-05-27T09:38:15.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.107+0000 7fe1df200640 1 -- 172.21.15.114:0/976402096 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1e00701e0 con 0x7fe1e00c8630 2024-05-27T09:38:15.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.108+0000 7fe1d6000640 1 -- 172.21.15.114:0/976402096 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+6062 (secure 0 0 0) 0x7fe1cc064b00 con 0x7fe1e00c8630 2024-05-27T09:38:15.112 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:38:15.112 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3513522318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3513522318},{"type":"v1","addr":"172.21.15.163:6835","nonce":3513522318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6838/1753596112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1753596112},{"type":"v1","addr":"172.21.15.163:6838","nonce":1753596112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:27:18.079214+0000","modified":"2024-05-27T09:37:36.353158+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5086,"mds_1":5072,"mds_2":5160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5072":{"gid":5072,"name":"f","rank":1,"incarnation":43,"state":"up:active","state_seq":27,"addr":"172.21.15.163:6839/1057092553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6837","nonce":1057092553},{"type":"v1","addr":"172.21.15.163:6839","nonce":1057092553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5086":{"gid":5086,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":27,"addr":"172.21.15.114:6834/1371544448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":1371544448},{"type":"v1","addr":"172.21.15.114:6834","nonce":1371544448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5160":{"gid":5160,"name":"e","rank":2,"incarnation":37,"state":"up:active","state_seq":26,"addr":"172.21.15.114:6837/1128209239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1128209239},{"type":"v1","addr":"172.21.15.114:6837","nonce":1128209239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5086,"qdb_cluster":[5072,5086,5160]},"id":1}]} 2024-05-27T09:38:15.113 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 47 2024-05-27T09:38:15.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.111+0000 7fe1df200640 1 -- 172.21.15.114:0/976402096 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe1c00762d0 msgr2=0x7fe1c00787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:15.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.111+0000 7fe1df200640 1 --2- 172.21.15.114:0/976402096 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe1c00762d0 0x7fe1c00787e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00097a0 tx=0x7fe1d002e000 comp rx=0 tx=0).stop 2024-05-27T09:38:15.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.111+0000 7fe1df200640 1 -- 172.21.15.114:0/976402096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e00c8630 msgr2=0x7fe1e014c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:15.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.111+0000 7fe1df200640 1 --2- 172.21.15.114:0/976402096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e00c8630 0x7fe1e014c370 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00c390 tx=0x7fe1cc00c850 comp rx=0 tx=0).stop 2024-05-27T09:38:15.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.111+0000 7fe1df200640 1 -- 172.21.15.114:0/976402096 shutdown_connections 2024-05-27T09:38:15.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.111+0000 7fe1df200640 1 --2- 172.21.15.114:0/976402096 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe1c00762d0 0x7fe1c00787e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:15.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.111+0000 7fe1df200640 1 --2- 172.21.15.114:0/976402096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe1e01534c0 0x7fe1e01512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:15.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.111+0000 7fe1df200640 1 --2- 172.21.15.114:0/976402096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e00c8630 0x7fe1e014c370 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:15.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.111+0000 7fe1df200640 1 --2- 172.21.15.114:0/976402096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e00c8220 0x7fe1e014a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:15.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.111+0000 7fe1df200640 1 -- 172.21.15.114:0/976402096 >> 172.21.15.114:0/976402096 conn(0x7fe1e00ca320 msgr2=0x7fe1e014fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:15.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.111+0000 7fe1df200640 1 -- 172.21.15.114:0/976402096 shutdown_connections 2024-05-27T09:38:15.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.111+0000 7fe1df200640 1 -- 172.21.15.114:0/976402096 wait complete. 2024-05-27T09:38:15.139 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T09:38:15.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.410+0000 7fbdcb000640 1 Processor -- start 2024-05-27T09:38:15.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.410+0000 7fbdcb000640 1 -- start start 2024-05-27T09:38:15.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.412+0000 7fbdcb000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbdc4074a60 0x7fbdc4154820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:15.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.412+0000 7fbdcb000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbdc40ca9e0 0x7fbdc4073670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:15.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.412+0000 7fbdcb000640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdc415a810 con 0x7fbdc4073260 2024-05-27T09:38:15.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.412+0000 7fbdcb000640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fbdc415db60 con 0x7fbdc40ca9e0 2024-05-27T09:38:15.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.412+0000 7fbdcb000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fbdc415aaf0 con 0x7fbdc4074a60 2024-05-27T09:38:15.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.413+0000 7fbdbb400640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7fbdc4073260 0x7fbdc40ca2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:38126/0 (socket says 172.21.15.114:38126) 2024-05-27T09:38:15.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.413+0000 7fbdbb400640 1 -- 172.21.15.114:0/3031943278 learned_addr learned my addr 172.21.15.114:0/3031943278 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:38:15.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.413+0000 7fbdc9000640 1 --2- 172.21.15.114:0/3031943278 >> v2:172.21.15.163:3300/0 conn(0x7fbdc40ca9e0 0x7fbdc4073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:15.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.413+0000 7fbdc3e00640 1 --2- 172.21.15.114:0/3031943278 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbdc4074a60 0x7fbdc4154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:15.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.413+0000 7fbdc3400640 1 -- 172.21.15.114:0/3031943278 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3099370767 0 0) 0x7fbdc415a810 con 0x7fbdc4073260 2024-05-27T09:38:15.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.413+0000 7fbdc3400640 1 -- 172.21.15.114:0/3031943278 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbda8003520 con 0x7fbdc4073260 2024-05-27T09:38:15.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.413+0000 7fbdc3e00640 1 -- 172.21.15.114:0/3031943278 >> v2:172.21.15.163:3300/0 conn(0x7fbdc40ca9e0 msgr2=0x7fbdc4073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:38:15.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.413+0000 7fbdc3e00640 1 --2- 172.21.15.114:0/3031943278 >> v2:172.21.15.163:3300/0 conn(0x7fbdc40ca9e0 0x7fbdc4073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:15.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.414+0000 7fbdc3e00640 1 -- 172.21.15.114:0/3031943278 >> v1:172.21.15.114:6789/0 conn(0x7fbdc4073260 legacy=0x7fbdc40ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:15.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.414+0000 7fbdc3e00640 1 -- 172.21.15.114:0/3031943278 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc415add0 con 0x7fbdc4074a60 2024-05-27T09:38:15.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.414+0000 7fbdc9000640 1 --2- 172.21.15.114:0/3031943278 >> v2:172.21.15.163:3300/0 conn(0x7fbdc40ca9e0 0x7fbdc4073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:38:15.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.415+0000 7fbdc3e00640 1 --2- 172.21.15.114:0/3031943278 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbdc4074a60 0x7fbdc4154820 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009d90 tx=0x7fbdb402a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f03b4c9fc95dcece server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:15.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.415+0000 7fbdc3400640 1 -- 172.21.15.114:0/3031943278 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4007740 con 0x7fbdc4074a60 2024-05-27T09:38:15.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.415+0000 7fbdc3400640 1 -- 172.21.15.114:0/3031943278 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fbdb40078e0 con 0x7fbdc4074a60 2024-05-27T09:38:15.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.415+0000 7fbdc3400640 1 -- 172.21.15.114:0/3031943278 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4007c90 con 0x7fbdc4074a60 2024-05-27T09:38:15.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.416+0000 7fbdcb000640 1 -- 172.21.15.114:0/3031943278 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbdc4074a60 msgr2=0x7fbdc4154820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:15.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.416+0000 7fbdcb000640 1 --2- 172.21.15.114:0/3031943278 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbdc4074a60 0x7fbdc4154820 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009d90 tx=0x7fbdb402a6f0 comp rx=0 tx=0).stop 2024-05-27T09:38:15.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.416+0000 7fbdcb000640 1 -- 172.21.15.114:0/3031943278 shutdown_connections 2024-05-27T09:38:15.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.416+0000 7fbdcb000640 1 --2- 172.21.15.114:0/3031943278 >> v2:172.21.15.163:3300/0 conn(0x7fbdc40ca9e0 0x7fbdc4073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:15.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.416+0000 7fbdcb000640 1 --2- 172.21.15.114:0/3031943278 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbdc4074a60 0x7fbdc4154820 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:15.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.416+0000 7fbdcb000640 1 -- 172.21.15.114:0/3031943278 >> 172.21.15.114:0/3031943278 conn(0x7fbdc406f6e0 msgr2=0x7fbdc4070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:15.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.416+0000 7fbdcb000640 1 -- 172.21.15.114:0/3031943278 shutdown_connections 2024-05-27T09:38:15.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.416+0000 7fbdcb000640 1 -- 172.21.15.114:0/3031943278 wait complete. 2024-05-27T09:38:15.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.418+0000 7fbdcb000640 1 Processor -- start 2024-05-27T09:38:15.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.418+0000 7fbdcb000640 1 -- start start 2024-05-27T09:38:15.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdcb000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbdc4073260 0x7fbdc4147250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:15.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdcb000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbdc4074a60 0x7fbdc40c3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:15.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdcb000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbdc40ca9e0 0x7fbdc40c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:15.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdcb000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc415a4e0 con 0x7fbdc4073260 2024-05-27T09:38:15.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdcb000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc415b390 con 0x7fbdc4074a60 2024-05-27T09:38:15.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdcb000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fbdc415dff0 con 0x7fbdc40ca9e0 2024-05-27T09:38:15.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdc3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbdc4073260 0x7fbdc4147250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:15.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdc3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbdc4073260 0x7fbdc4147250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49174/0 (socket says 172.21.15.114:49174) 2024-05-27T09:38:15.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdc3e00640 1 -- 172.21.15.114:0/2493267060 learned_addr learned my addr 172.21.15.114:0/2493267060 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:38:15.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdbb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbdc4074a60 0x7fbdc40c3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:15.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdc9000640 1 --2- 172.21.15.114:0/2493267060 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbdc40ca9e0 0x7fbdc40c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:15.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdbb400640 1 -- 172.21.15.114:0/2493267060 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbdc40ca9e0 msgr2=0x7fbdc40c41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:15.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdbb400640 1 --2- 172.21.15.114:0/2493267060 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbdc40ca9e0 0x7fbdc40c41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:15.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdbb400640 1 -- 172.21.15.114:0/2493267060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbdc4073260 msgr2=0x7fbdc4147250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:15.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdbb400640 1 --2- 172.21.15.114:0/2493267060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbdc4073260 0x7fbdc4147250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:15.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdbb400640 1 -- 172.21.15.114:0/2493267060 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc41f1fe0 con 0x7fbdc4074a60 2024-05-27T09:38:15.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdc9000640 1 --2- 172.21.15.114:0/2493267060 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbdc40ca9e0 0x7fbdc40c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:38:15.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdbb400640 1 --2- 172.21.15.114:0/2493267060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbdc4074a60 0x7fbdc40c3c60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00c390 tx=0x7fbdac00c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:15.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdc0c00640 1 -- 172.21.15.114:0/2493267060 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac017070 con 0x7fbdc4074a60 2024-05-27T09:38:15.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdc0c00640 1 -- 172.21.15.114:0/2493267060 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fbdac005ce0 con 0x7fbdc4074a60 2024-05-27T09:38:15.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdc0c00640 1 -- 172.21.15.114:0/2493267060 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac0072d0 con 0x7fbdc4074a60 2024-05-27T09:38:15.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdcb000640 1 -- 172.21.15.114:0/2493267060 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc41f22c0 con 0x7fbdc4074a60 2024-05-27T09:38:15.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.421+0000 7fbdcb000640 1 -- 172.21.15.114:0/2493267060 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc4147c70 con 0x7fbdc4074a60 2024-05-27T09:38:15.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.423+0000 7fbdcb000640 1 -- 172.21.15.114:0/2493267060 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc40cb200 con 0x7fbdc4074a60 2024-05-27T09:38:15.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.424+0000 7fbdc0c00640 1 -- 172.21.15.114:0/2493267060 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbdac01a070 con 0x7fbdc4074a60 2024-05-27T09:38:15.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.425+0000 7fbdc0c00640 1 --2- 172.21.15.114:0/2493267060 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbda00762d0 0x7fbda00787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:15.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.425+0000 7fbdc0c00640 1 -- 172.21.15.114:0/2493267060 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7fbdac096a90 con 0x7fbdc4074a60 2024-05-27T09:38:15.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.425+0000 7fbdc3e00640 1 --2- 172.21.15.114:0/2493267060 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbda00762d0 0x7fbda00787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:15.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.426+0000 7fbdc3e00640 1 --2- 172.21.15.114:0/2493267060 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbda00762d0 0x7fbda00787e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400c090 tx=0x7fbdb402f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:15.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.430+0000 7fbdc0c00640 1 -- 172.21.15.114:0/2493267060 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbdc40cb200 con 0x7fbdc4074a60 2024-05-27T09:38:15.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.641+0000 7fbdcb000640 1 -- 172.21.15.114:0/2493267060 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbdc415a810 con 0x7fbdc4074a60 2024-05-27T09:38:15.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.645+0000 7fbdc0c00640 1 -- 172.21.15.114:0/2493267060 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+17505 (secure 0 0 0) 0x7fbdc415a810 con 0x7fbdc4074a60 2024-05-27T09:38:15.646 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-05-27T09:38:15.646 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":50,"fsid":"2efb71a6-bd3b-4b4e-9c53-c44c9c2eae48","created":"2024-05-27T09:27:00.307278+0000","modified":"2024-05-27T09:37:15.516868+0000","last_up_change":"2024-05-27T09:27:11.463226+0000","last_in_change":"2024-05-27T09:27:02.972880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T09:27:10.796859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T09:27:16.063199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T09:27:17.064857+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":46,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3561c750-1d63-42f6-a6ae-0ee63721e15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6817","nonce":1960550710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6819","nonce":1960550710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6824","nonce":1960550710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1960550710},{"type":"v1","addr":"172.21.15.114:6821","nonce":1960550710}]},"public_addr":"172.21.15.114:6817/1960550710","cluster_addr":"172.21.15.114:6819/1960550710","heartbeat_back_addr":"172.21.15.114:6824/1960550710","heartbeat_front_addr":"172.21.15.114:6821/1960550710","state":["exists","up"]},{"osd":1,"uuid":"9a407d04-38c0-4676-928f-d04e8116bd19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6825","nonce":326580057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6827","nonce":326580057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6831","nonce":326580057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":326580057},{"type":"v1","addr":"172.21.15.114:6829","nonce":326580057}]},"public_addr":"172.21.15.114:6825/326580057","cluster_addr":"172.21.15.114:6827/326580057","heartbeat_back_addr":"172.21.15.114:6831/326580057","heartbeat_front_addr":"172.21.15.114:6829/326580057","state":["exists","up"]},{"osd":2,"uuid":"255a1cc6-9357-43f7-9db5-d760f4806ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6809","nonce":865568619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6811","nonce":865568619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6815","nonce":865568619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":865568619},{"type":"v1","addr":"172.21.15.114:6813","nonce":865568619}]},"public_addr":"172.21.15.114:6809/865568619","cluster_addr":"172.21.15.114:6811/865568619","heartbeat_back_addr":"172.21.15.114:6815/865568619","heartbeat_front_addr":"172.21.15.114:6813/865568619","state":["exists","up"]},{"osd":3,"uuid":"cb4d7692-9a17-41b9-927e-d8eac3e217c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6801","nonce":1601792164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6803","nonce":1601792164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6807","nonce":1601792164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":1601792164},{"type":"v1","addr":"172.21.15.114:6805","nonce":1601792164}]},"public_addr":"172.21.15.114:6801/1601792164","cluster_addr":"172.21.15.114:6803/1601792164","heartbeat_back_addr":"172.21.15.114:6807/1601792164","heartbeat_front_addr":"172.21.15.114:6805/1601792164","state":["exists","up"]},{"osd":4,"uuid":"cdef1940-3453-4fbd-b0f1-459f5f5c7142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6817","nonce":1982246015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6819","nonce":1982246015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6823","nonce":1982246015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":1982246015},{"type":"v1","addr":"172.21.15.163:6821","nonce":1982246015}]},"public_addr":"172.21.15.163:6817/1982246015","cluster_addr":"172.21.15.163:6819/1982246015","heartbeat_back_addr":"172.21.15.163:6823/1982246015","heartbeat_front_addr":"172.21.15.163:6821/1982246015","state":["exists","up"]},{"osd":5,"uuid":"220a7b32-abcc-492f-bca8-4615cc2ea111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6801","nonce":1695294004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6803","nonce":1695294004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6807","nonce":1695294004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":1695294004},{"type":"v1","addr":"172.21.15.163:6805","nonce":1695294004}]},"public_addr":"172.21.15.163:6801/1695294004","cluster_addr":"172.21.15.163:6803/1695294004","heartbeat_back_addr":"172.21.15.163:6807/1695294004","heartbeat_front_addr":"172.21.15.163:6805/1695294004","state":["exists","up"]},{"osd":6,"uuid":"43a74d13-d03f-493a-a6c0-e73b0a056c10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6809","nonce":3649718505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6811","nonce":3649718505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6815","nonce":3649718505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":3649718505},{"type":"v1","addr":"172.21.15.163:6813","nonce":3649718505}]},"public_addr":"172.21.15.163:6809/3649718505","cluster_addr":"172.21.15.163:6811/3649718505","heartbeat_back_addr":"172.21.15.163:6815/3649718505","heartbeat_front_addr":"172.21.15.163:6813/3649718505","state":["exists","up"]},{"osd":7,"uuid":"134629c2-70af-4b84-83a6-7ddf53681baa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6825","nonce":2539451311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6827","nonce":2539451311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6831","nonce":2539451311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2539451311},{"type":"v1","addr":"172.21.15.163:6829","nonce":2539451311}]},"public_addr":"172.21.15.163:6825/2539451311","cluster_addr":"172.21.15.163:6827/2539451311","heartbeat_back_addr":"172.21.15.163:6831/2539451311","heartbeat_front_addr":"172.21.15.163:6829/2539451311","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.388942+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.515675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.646761+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.400212+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.220072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.079531+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.180419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:27:08.175008+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.34","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:6835/2747052034":"2024-05-28T09:37:15.516770+0000","172.21.15.114:6833/2747052034":"2024-05-28T09:37:15.516770+0000","172.21.15.163:6837/3371890103":"2024-05-28T09:35:40.437044+0000","172.21.15.114:6837/1448950017":"2024-05-28T09:34:05.407657+0000","172.21.15.163:6839/3371890103":"2024-05-28T09:35:40.437044+0000","172.21.15.114:6834/657027519":"2024-05-28T09:32:30.373988+0000","172.21.15.114:6836/1448950017":"2024-05-28T09:34:05.407657+0000","172.21.15.114:6832/657027519":"2024-05-28T09:32:30.373988+0000","172.21.15.114:6833/1325619883":"2024-05-28T09:31:00.345342+0000","172.21.15.163:6839/444766663":"2024-05-28T09:29:45.323526+0000","172.21.15.114:6835/1325619883":"2024-05-28T09:31:00.345342+0000","172.21.15.163:6837/444766663":"2024-05-28T09:29:45.323526+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T09:38:15.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.646+0000 7fbdcb000640 1 -- 172.21.15.114:0/2493267060 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbda00762d0 msgr2=0x7fbda00787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:15.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.646+0000 7fbdcb000640 1 --2- 172.21.15.114:0/2493267060 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbda00762d0 0x7fbda00787e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400c090 tx=0x7fbdb402f000 comp rx=0 tx=0).stop 2024-05-27T09:38:15.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.646+0000 7fbdcb000640 1 -- 172.21.15.114:0/2493267060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbdc4074a60 msgr2=0x7fbdc40c3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:15.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.646+0000 7fbdcb000640 1 --2- 172.21.15.114:0/2493267060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbdc4074a60 0x7fbdc40c3c60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00c390 tx=0x7fbdac00c790 comp rx=0 tx=0).stop 2024-05-27T09:38:15.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.646+0000 7fbdcb000640 1 -- 172.21.15.114:0/2493267060 shutdown_connections 2024-05-27T09:38:15.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.646+0000 7fbdcb000640 1 --2- 172.21.15.114:0/2493267060 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fbda00762d0 0x7fbda00787e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:15.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.646+0000 7fbdcb000640 1 --2- 172.21.15.114:0/2493267060 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbdc40ca9e0 0x7fbdc40c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:15.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.646+0000 7fbdcb000640 1 --2- 172.21.15.114:0/2493267060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbdc4074a60 0x7fbdc40c3c60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:15.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.646+0000 7fbdcb000640 1 --2- 172.21.15.114:0/2493267060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbdc4073260 0x7fbdc4147250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:15.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.646+0000 7fbdcb000640 1 -- 172.21.15.114:0/2493267060 >> 172.21.15.114:0/2493267060 conn(0x7fbdc406f6e0 msgr2=0x7fbdc41e3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:15.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.647+0000 7fbdcb000640 1 -- 172.21.15.114:0/2493267060 shutdown_connections 2024-05-27T09:38:15.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.647+0000 7fbdcb000640 1 -- 172.21.15.114:0/2493267060 wait complete. 2024-05-27T09:38:15.673 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-27T09:38:15.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.943+0000 7fd126000640 1 Processor -- start 2024-05-27T09:38:15.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.944+0000 7fd126000640 1 -- start start 2024-05-27T09:38:15.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.946+0000 7fd126000640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd1200741f0 0x7fd1200c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:15.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.946+0000 7fd126000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd1201547d0 0x7fd12015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:15.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.946+0000 7fd126000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd120074840 0x7fd12006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:15.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.946+0000 7fd126000640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fd1200703d0 con 0x7fd1200741f0 2024-05-27T09:38:15.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.946+0000 7fd126000640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fd12015b260 con 0x7fd1201547d0 2024-05-27T09:38:15.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.946+0000 7fd126000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fd12006fe50 con 0x7fd120074840 2024-05-27T09:38:15.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.946+0000 7fd125600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd120074840 0x7fd12006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:15.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.946+0000 7fd125600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd120074840 0x7fd12006edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:46354/0 (socket says 172.21.15.114:46354) 2024-05-27T09:38:15.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.946+0000 7fd124c00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fd1200741f0 0x7fd1200c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:15.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.946+0000 7fd125600640 1 -- 172.21.15.114:0/1378108822 learned_addr learned my addr 172.21.15.114:0/1378108822 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:38:15.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.946+0000 7fd117e00640 1 --2- 172.21.15.114:0/1378108822 >> v2:172.21.15.163:3300/0 conn(0x7fd1201547d0 0x7fd12015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:15.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.947+0000 7fd125600640 1 -- 172.21.15.114:0/1378108822 >> v2:172.21.15.163:3300/0 conn(0x7fd1201547d0 msgr2=0x7fd12015ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:38:15.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.947+0000 7fd125600640 1 --2- 172.21.15.114:0/1378108822 >> v2:172.21.15.163:3300/0 conn(0x7fd1201547d0 0x7fd12015ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:15.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.947+0000 7fd125600640 1 -- 172.21.15.114:0/1378108822 >> v2:172.21.15.114:3300/0 conn(0x7fd1200741f0 msgr2=0x7fd1200c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:38:15.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.947+0000 7fd125600640 1 --2- 172.21.15.114:0/1378108822 >> v2:172.21.15.114:3300/0 conn(0x7fd1200741f0 0x7fd1200c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:15.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.947+0000 7fd125600640 1 -- 172.21.15.114:0/1378108822 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1200700f0 con 0x7fd120074840 2024-05-27T09:38:15.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.947+0000 7fd117e00640 1 --2- 172.21.15.114:0/1378108822 >> v2:172.21.15.163:3300/0 conn(0x7fd1201547d0 0x7fd12015ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:38:15.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.948+0000 7fd125600640 1 --2- 172.21.15.114:0/1378108822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd120074840 0x7fd12006edf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd120071f60 tx=0x7fd11802a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c203bd017ecfbe6f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:15.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.948+0000 7fd11fe00640 1 -- 172.21.15.114:0/1378108822 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118036070 con 0x7fd120074840 2024-05-27T09:38:15.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.949+0000 7fd11fe00640 1 -- 172.21.15.114:0/1378108822 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fd118031600 con 0x7fd120074840 2024-05-27T09:38:15.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.949+0000 7fd11fe00640 1 -- 172.21.15.114:0/1378108822 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1180319f0 con 0x7fd120074840 2024-05-27T09:38:15.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.949+0000 7fd126000640 1 -- 172.21.15.114:0/1378108822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd120074840 msgr2=0x7fd12006edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:15.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.949+0000 7fd126000640 1 --2- 172.21.15.114:0/1378108822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd120074840 0x7fd12006edf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd120071f60 tx=0x7fd11802a750 comp rx=0 tx=0).stop 2024-05-27T09:38:15.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.949+0000 7fd126000640 1 -- 172.21.15.114:0/1378108822 shutdown_connections 2024-05-27T09:38:15.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.949+0000 7fd126000640 1 --2- 172.21.15.114:0/1378108822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd120074840 0x7fd12006edf0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:15.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.949+0000 7fd126000640 1 --2- 172.21.15.114:0/1378108822 >> v2:172.21.15.163:3300/0 conn(0x7fd1201547d0 0x7fd12015ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:15.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.949+0000 7fd126000640 1 --2- 172.21.15.114:0/1378108822 >> v2:172.21.15.114:3300/0 conn(0x7fd1200741f0 0x7fd1200c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:15.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.949+0000 7fd126000640 1 -- 172.21.15.114:0/1378108822 >> 172.21.15.114:0/1378108822 conn(0x7fd1200ca320 msgr2=0x7fd1200ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:15.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.950+0000 7fd126000640 1 -- 172.21.15.114:0/1378108822 shutdown_connections 2024-05-27T09:38:15.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.950+0000 7fd126000640 1 -- 172.21.15.114:0/1378108822 wait complete. 2024-05-27T09:38:15.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.952+0000 7fd126000640 1 Processor -- start 2024-05-27T09:38:15.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.952+0000 7fd126000640 1 -- start start 2024-05-27T09:38:15.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.953+0000 7fd126000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1200741f0 0x7fd120157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:15.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.953+0000 7fd126000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd120074840 0x7fd1201560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:15.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.953+0000 7fd126000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1201547d0 0x7fd12014d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:15.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.953+0000 7fd126000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd120073790 con 0x7fd1200741f0 2024-05-27T09:38:15.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.953+0000 7fd126000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd1200734b0 con 0x7fd120074840 2024-05-27T09:38:15.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.953+0000 7fd126000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fd120073cb0 con 0x7fd1201547d0 2024-05-27T09:38:15.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.954+0000 7fd124c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1200741f0 0x7fd120157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:15.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.954+0000 7fd125600640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1201547d0 0x7fd12014d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:15.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.954+0000 7fd117e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd120074840 0x7fd1201560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:15.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.954+0000 7fd124c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1200741f0 0x7fd120157150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49200/0 (socket says 172.21.15.114:49200) 2024-05-27T09:38:15.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.954+0000 7fd124c00640 1 -- 172.21.15.114:0/2577649819 learned_addr learned my addr 172.21.15.114:0/2577649819 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:38:15.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.954+0000 7fd125600640 1 -- 172.21.15.114:0/2577649819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd120074840 msgr2=0x7fd1201560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:15.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.954+0000 7fd125600640 1 --2- 172.21.15.114:0/2577649819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd120074840 0x7fd1201560d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:15.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.954+0000 7fd125600640 1 -- 172.21.15.114:0/2577649819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1200741f0 msgr2=0x7fd120157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:15.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.954+0000 7fd125600640 1 --2- 172.21.15.114:0/2577649819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1200741f0 0x7fd120157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:15.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.954+0000 7fd125600640 1 -- 172.21.15.114:0/2577649819 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd120157690 con 0x7fd1201547d0 2024-05-27T09:38:15.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.954+0000 7fd124c00640 1 --2- 172.21.15.114:0/2577649819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1200741f0 0x7fd120157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:38:15.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.955+0000 7fd125600640 1 --2- 172.21.15.114:0/2577649819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1201547d0 0x7fd12014d7c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd118030000 tx=0x7fd118004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:15.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.955+0000 7fd11d600640 1 -- 172.21.15.114:0/2577649819 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11803f020 con 0x7fd1201547d0 2024-05-27T09:38:15.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.955+0000 7fd11d600640 1 -- 172.21.15.114:0/2577649819 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fd118037040 con 0x7fd1201547d0 2024-05-27T09:38:15.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.955+0000 7fd126000640 1 -- 172.21.15.114:0/2577649819 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd120156610 con 0x7fd1201547d0 2024-05-27T09:38:15.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.955+0000 7fd11d600640 1 -- 172.21.15.114:0/2577649819 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118036070 con 0x7fd1201547d0 2024-05-27T09:38:15.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.955+0000 7fd126000640 1 -- 172.21.15.114:0/2577649819 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd12014ce90 con 0x7fd1201547d0 2024-05-27T09:38:15.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.956+0000 7fd126000640 1 -- 172.21.15.114:0/2577649819 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd120076a00 con 0x7fd1201547d0 2024-05-27T09:38:15.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.958+0000 7fd11d600640 1 -- 172.21.15.114:0/2577649819 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd118042020 con 0x7fd1201547d0 2024-05-27T09:38:15.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.959+0000 7fd11d600640 1 --2- 172.21.15.114:0/2577649819 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd0fc0762d0 0x7fd0fc0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:15.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.959+0000 7fd11d600640 1 -- 172.21.15.114:0/2577649819 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 6254+0+0 (secure 0 0 0) 0x7fd1180b8410 con 0x7fd1201547d0 2024-05-27T09:38:15.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.959+0000 7fd124c00640 1 --2- 172.21.15.114:0/2577649819 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd0fc0762d0 0x7fd0fc0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:15.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.960+0000 7fd124c00640 1 --2- 172.21.15.114:0/2577649819 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd0fc0762d0 0x7fd0fc0787e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd110009ed0 tx=0x7fd110009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:15.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:15.962+0000 7fd11d600640 1 -- 172.21.15.114:0/2577649819 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd118049030 con 0x7fd1201547d0 2024-05-27T09:38:16.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.210+0000 7fd126000640 1 -- 172.21.15.114:0/2577649819 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd120152d70 con 0x7fd1201547d0 2024-05-27T09:38:16.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.224+0000 7fd11d600640 1 -- 172.21.15.114:0/2577649819 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) ==== 161+0+0 (secure 0 0 0) 0x7fd118081450 con 0x7fd1201547d0 2024-05-27T09:38:16.226 INFO:teuthology.orchestra.run.smithi114.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-27T09:38:16.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.228+0000 7fd126000640 1 -- 172.21.15.114:0/2577649819 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd0fc0762d0 msgr2=0x7fd0fc0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:16.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.228+0000 7fd126000640 1 --2- 172.21.15.114:0/2577649819 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd0fc0762d0 0x7fd0fc0787e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd110009ed0 tx=0x7fd110009190 comp rx=0 tx=0).stop 2024-05-27T09:38:16.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.228+0000 7fd126000640 1 -- 172.21.15.114:0/2577649819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1201547d0 msgr2=0x7fd12014d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:16.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.228+0000 7fd126000640 1 --2- 172.21.15.114:0/2577649819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1201547d0 0x7fd12014d7c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd118030000 tx=0x7fd118004a40 comp rx=0 tx=0).stop 2024-05-27T09:38:16.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.228+0000 7fd126000640 1 -- 172.21.15.114:0/2577649819 shutdown_connections 2024-05-27T09:38:16.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.228+0000 7fd126000640 1 --2- 172.21.15.114:0/2577649819 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fd0fc0762d0 0x7fd0fc0787e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:16.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.229+0000 7fd126000640 1 --2- 172.21.15.114:0/2577649819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1201547d0 0x7fd12014d7c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:16.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.229+0000 7fd126000640 1 --2- 172.21.15.114:0/2577649819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd120074840 0x7fd1201560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:16.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.229+0000 7fd126000640 1 --2- 172.21.15.114:0/2577649819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1200741f0 0x7fd120157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:16.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.229+0000 7fd126000640 1 -- 172.21.15.114:0/2577649819 >> 172.21.15.114:0/2577649819 conn(0x7fd1200ca320 msgr2=0x7fd12015a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:16.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.229+0000 7fd126000640 1 -- 172.21.15.114:0/2577649819 shutdown_connections 2024-05-27T09:38:16.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.229+0000 7fd126000640 1 -- 172.21.15.114:0/2577649819 wait complete. 2024-05-27T09:38:16.256 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-27T09:38:16.277 INFO:tasks.ceph.mds.f.smithi163.stdout:starting mds.f at 2024-05-27T09:38:16.387 INFO:tasks.ceph.mds.e.smithi114.stdout:starting mds.e at 2024-05-27T09:38:16.404 INFO:tasks.ceph.mds.c.smithi114.stdout:starting mds.c at 2024-05-27T09:38:16.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.505+0000 7fe232a00640 1 Processor -- start 2024-05-27T09:38:16.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.505+0000 7fe232a00640 1 -- start start 2024-05-27T09:38:16.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.507+0000 7fe232a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe22c074a10 0x7fe22c154670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:16.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.507+0000 7fe232a00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe22c073260 0x7fe22c073670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:16.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.507+0000 7fe232a00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe22c0c9040 con 0x7fe22c15de20 2024-05-27T09:38:16.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.507+0000 7fe232a00640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fe22c0c9320 con 0x7fe22c073260 2024-05-27T09:38:16.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.507+0000 7fe232a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fe22c0c9600 con 0x7fe22c074a10 2024-05-27T09:38:16.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.507+0000 7fe22b400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe22c073260 0x7fe22c073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:16.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.507+0000 7fe22be00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe22c074a10 0x7fe22c154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:16.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.508+0000 7fe22b400640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe22c073260 0x7fe22c073670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.114:55394/0 (socket says 172.21.15.114:55394) 2024-05-27T09:38:16.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.508+0000 7fe22be00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe22c074a10 0x7fe22c154670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.114:46422/0 (socket says 172.21.15.114:46422) 2024-05-27T09:38:16.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.508+0000 7fe230a00640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe22c15de20 0x7fe22c15e230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.114:58374/0 (socket says 172.21.15.114:58374) 2024-05-27T09:38:16.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.508+0000 7fe22b400640 1 -- 172.21.15.114:0/267229569 learned_addr learned my addr 172.21.15.114:0/267229569 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:38:16.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.508+0000 7fe22b400640 1 -- 172.21.15.114:0/267229569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe22c074a10 msgr2=0x7fe22c154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:38:16.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.508+0000 7fe22b400640 1 --2- 172.21.15.114:0/267229569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe22c074a10 0x7fe22c154670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:16.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.508+0000 7fe22b400640 1 -- 172.21.15.114:0/267229569 >> v1:172.21.15.163:6789/0 conn(0x7fe22c15de20 legacy=0x7fe22c15e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:16.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.508+0000 7fe22aa00640 1 -- 172.21.15.114:0/267229569 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1077244025 0 0) 0x7fe22c0c9040 con 0x7fe22c15de20 2024-05-27T09:38:16.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.508+0000 7fe22b400640 1 -- 172.21.15.114:0/267229569 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22c0c98e0 con 0x7fe22c073260 2024-05-27T09:38:16.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.508+0000 7fe22be00640 1 --2- 172.21.15.114:0/267229569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe22c074a10 0x7fe22c154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:38:16.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.509+0000 7fe22b400640 1 --2- 172.21.15.114:0/267229569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe22c073260 0x7fe22c073670 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe218009870 tx=0x7fe21802a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8be2099f1a7a954 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:16.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.509+0000 7fe22aa00640 1 -- 172.21.15.114:0/267229569 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218037070 con 0x7fe22c073260 2024-05-27T09:38:16.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.509+0000 7fe22aa00640 1 -- 172.21.15.114:0/267229569 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe21802ae50 con 0x7fe22c073260 2024-05-27T09:38:16.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.509+0000 7fe22aa00640 1 -- 172.21.15.114:0/267229569 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218032880 con 0x7fe22c073260 2024-05-27T09:38:16.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.510+0000 7fe232a00640 1 -- 172.21.15.114:0/267229569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe22c073260 msgr2=0x7fe22c073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:16.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.510+0000 7fe232a00640 1 --2- 172.21.15.114:0/267229569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe22c073260 0x7fe22c073670 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe218009870 tx=0x7fe21802a300 comp rx=0 tx=0).stop 2024-05-27T09:38:16.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.510+0000 7fe232a00640 1 -- 172.21.15.114:0/267229569 shutdown_connections 2024-05-27T09:38:16.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.510+0000 7fe232a00640 1 --2- 172.21.15.114:0/267229569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe22c073260 0x7fe22c073670 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:16.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.510+0000 7fe232a00640 1 --2- 172.21.15.114:0/267229569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe22c074a10 0x7fe22c154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:16.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.510+0000 7fe232a00640 1 -- 172.21.15.114:0/267229569 >> 172.21.15.114:0/267229569 conn(0x7fe22c06f6e0 msgr2=0x7fe22c070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:16.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.510+0000 7fe232a00640 1 -- 172.21.15.114:0/267229569 shutdown_connections 2024-05-27T09:38:16.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.510+0000 7fe232a00640 1 -- 172.21.15.114:0/267229569 wait complete. 2024-05-27T09:38:16.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.513+0000 7fe232a00640 1 Processor -- start 2024-05-27T09:38:16.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.513+0000 7fe232a00640 1 -- start start 2024-05-27T09:38:16.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.514+0000 7fe232a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe22c073260 0x7fe22c1e5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:16.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.514+0000 7fe232a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe22c074a10 0x7fe22c15d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:16.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.514+0000 7fe232a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe22c15de20 0x7fe22c14eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:16.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.514+0000 7fe232a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c0ca240 con 0x7fe22c074a10 2024-05-27T09:38:16.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.514+0000 7fe232a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c0ca0c0 con 0x7fe22c15de20 2024-05-27T09:38:16.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.514+0000 7fe232a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fe22c0ca3c0 con 0x7fe22c073260 2024-05-27T09:38:16.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.514+0000 7fe22b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe22c074a10 0x7fe22c15d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:16.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.514+0000 7fe22b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe22c074a10 0x7fe22c15d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49246/0 (socket says 172.21.15.114:49246) 2024-05-27T09:38:16.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.514+0000 7fe22b400640 1 -- 172.21.15.114:0/1499389019 learned_addr learned my addr 172.21.15.114:0/1499389019 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:38:16.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.515+0000 7fe22be00640 1 --2- 172.21.15.114:0/1499389019 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe22c073260 0x7fe22c1e5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:16.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.515+0000 7fe230a00640 1 --2- 172.21.15.114:0/1499389019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe22c15de20 0x7fe22c14eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:16.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.515+0000 7fe22be00640 1 -- 172.21.15.114:0/1499389019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe22c15de20 msgr2=0x7fe22c14eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:16.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.515+0000 7fe22be00640 1 --2- 172.21.15.114:0/1499389019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe22c15de20 0x7fe22c14eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:16.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.515+0000 7fe22be00640 1 -- 172.21.15.114:0/1499389019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe22c074a10 msgr2=0x7fe22c15d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:16.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.515+0000 7fe22be00640 1 --2- 172.21.15.114:0/1499389019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe22c074a10 0x7fe22c15d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:16.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.515+0000 7fe22be00640 1 -- 172.21.15.114:0/1499389019 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22c14f080 con 0x7fe22c073260 2024-05-27T09:38:16.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.515+0000 7fe22be00640 1 --2- 172.21.15.114:0/1499389019 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe22c073260 0x7fe22c1e5e00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00e8a0 tx=0x7fe21c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:16.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.515+0000 7fe223e00640 1 -- 172.21.15.114:0/1499389019 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c017070 con 0x7fe22c073260 2024-05-27T09:38:16.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.515+0000 7fe223e00640 1 -- 172.21.15.114:0/1499389019 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe21c00c940 con 0x7fe22c073260 2024-05-27T09:38:16.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.516+0000 7fe232a00640 1 -- 172.21.15.114:0/1499389019 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22c1f0430 con 0x7fe22c073260 2024-05-27T09:38:16.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.516+0000 7fe223e00640 1 -- 172.21.15.114:0/1499389019 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c00cd50 con 0x7fe22c073260 2024-05-27T09:38:16.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.516+0000 7fe232a00640 1 -- 172.21.15.114:0/1499389019 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe22c1f09b0 con 0x7fe22c073260 2024-05-27T09:38:16.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.517+0000 7fe232a00640 1 -- 172.21.15.114:0/1499389019 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe22c06a380 con 0x7fe22c073260 2024-05-27T09:38:16.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.518+0000 7fe223e00640 1 -- 172.21.15.114:0/1499389019 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe21c01a070 con 0x7fe22c073260 2024-05-27T09:38:16.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.519+0000 7fe223e00640 1 --2- 172.21.15.114:0/1499389019 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe20c0762d0 0x7fe20c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:16.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.519+0000 7fe223e00640 1 -- 172.21.15.114:0/1499389019 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(51..51 src has 1..51) ==== 6512+0+0 (secure 0 0 0) 0x7fe21c097870 con 0x7fe22c073260 2024-05-27T09:38:16.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.519+0000 7fe22b400640 1 --2- 172.21.15.114:0/1499389019 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe20c0762d0 0x7fe20c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:16.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.520+0000 7fe22b400640 1 --2- 172.21.15.114:0/1499389019 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe20c0762d0 0x7fe20c0787e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe2180090b0 tx=0x7fe21802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:16.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.524+0000 7fe223e00640 1 -- 172.21.15.114:0/1499389019 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe21c060750 con 0x7fe22c073260 2024-05-27T09:38:16.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:16.754+0000 7fe232a00640 1 -- 172.21.15.114:0/1499389019 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe22c1581e0 con 0x7fe22c073260 2024-05-27T09:38:17.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.227+0000 7fe223e00640 1 -- 172.21.15.114:0/1499389019 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) ==== 104+0+0 (secure 0 0 0) 0x7fe21c095e20 con 0x7fe22c073260 2024-05-27T09:38:17.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.231+0000 7fe232a00640 1 -- 172.21.15.114:0/1499389019 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe20c0762d0 msgr2=0x7fe20c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:17.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.231+0000 7fe232a00640 1 --2- 172.21.15.114:0/1499389019 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe20c0762d0 0x7fe20c0787e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe2180090b0 tx=0x7fe21802e000 comp rx=0 tx=0).stop 2024-05-27T09:38:17.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.231+0000 7fe232a00640 1 -- 172.21.15.114:0/1499389019 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe22c073260 msgr2=0x7fe22c1e5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:17.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.231+0000 7fe232a00640 1 --2- 172.21.15.114:0/1499389019 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe22c073260 0x7fe22c1e5e00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00e8a0 tx=0x7fe21c00ed60 comp rx=0 tx=0).stop 2024-05-27T09:38:17.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.231+0000 7fe232a00640 1 -- 172.21.15.114:0/1499389019 shutdown_connections 2024-05-27T09:38:17.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.231+0000 7fe232a00640 1 --2- 172.21.15.114:0/1499389019 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe20c0762d0 0x7fe20c0787e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:17.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.231+0000 7fe232a00640 1 --2- 172.21.15.114:0/1499389019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe22c15de20 0x7fe22c14eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:17.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.231+0000 7fe232a00640 1 --2- 172.21.15.114:0/1499389019 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe22c074a10 0x7fe22c15d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:17.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.231+0000 7fe232a00640 1 --2- 172.21.15.114:0/1499389019 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe22c073260 0x7fe22c1e5e00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:17.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.231+0000 7fe232a00640 1 -- 172.21.15.114:0/1499389019 >> 172.21.15.114:0/1499389019 conn(0x7fe22c06f6e0 msgr2=0x7fe22c15a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:17.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.231+0000 7fe232a00640 1 -- 172.21.15.114:0/1499389019 shutdown_connections 2024-05-27T09:38:17.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.232+0000 7fe232a00640 1 -- 172.21.15.114:0/1499389019 wait complete. 2024-05-27T09:38:17.260 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-27T09:38:17.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.537+0000 7fe6b1000640 1 Processor -- start 2024-05-27T09:38:17.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.537+0000 7fe6b1000640 1 -- start start 2024-05-27T09:38:17.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.540+0000 7fe6b1000640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe6ac0c9830 0x7fe6ac06c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:17.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.540+0000 7fe6b1000640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe6ac073260 0x7fe6ac06cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:17.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.540+0000 7fe6b1000640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6ac074cb0 con 0x7fe6ac15a830 2024-05-27T09:38:17.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.540+0000 7fe6b1000640 1 -- --> v2:172.21.15.114:3300/0 -- mon_getmap magic: 0 -- 0x7fe6ac073ee0 con 0x7fe6ac0c9830 2024-05-27T09:38:17.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.540+0000 7fe6b1000640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 -- 0x7fe6ac074130 con 0x7fe6ac073260 2024-05-27T09:38:17.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.540+0000 7fe6a9e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe6ac0c9830 0x7fe6ac06c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:17.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.540+0000 7fe6a3e00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe6ac073260 0x7fe6ac06cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:17.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.540+0000 7fe6a9e00640 1 --2- >> v2:172.21.15.114:3300/0 conn(0x7fe6ac0c9830 0x7fe6ac06c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49248/0 (socket says 172.21.15.114:49248) 2024-05-27T09:38:17.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.540+0000 7fe6a9e00640 1 -- 172.21.15.114:0/3037207624 learned_addr learned my addr 172.21.15.114:0/3037207624 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:38:17.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.541+0000 7fe6a9400640 1 -- 172.21.15.114:0/3037207624 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2590167602 0 0) 0x7fe6ac074cb0 con 0x7fe6ac15a830 2024-05-27T09:38:17.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.541+0000 7fe6a9400640 1 -- 172.21.15.114:0/3037207624 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe690003520 con 0x7fe6ac15a830 2024-05-27T09:38:17.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.541+0000 7fe6a3e00640 1 -- 172.21.15.114:0/3037207624 >> v2:172.21.15.114:3300/0 conn(0x7fe6ac0c9830 msgr2=0x7fe6ac06c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T09:38:17.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.541+0000 7fe6a3e00640 1 --2- 172.21.15.114:0/3037207624 >> v2:172.21.15.114:3300/0 conn(0x7fe6ac0c9830 0x7fe6ac06c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:17.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.541+0000 7fe6a3e00640 1 -- 172.21.15.114:0/3037207624 >> v1:172.21.15.163:6789/0 conn(0x7fe6ac15a830 legacy=0x7fe6ac15ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:17.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.541+0000 7fe6a3e00640 1 -- 172.21.15.114:0/3037207624 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ac074410 con 0x7fe6ac073260 2024-05-27T09:38:17.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.542+0000 7fe6a3e00640 1 --2- 172.21.15.114:0/3037207624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6ac073260 0x7fe6ac06cbd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0c5350 tx=0x7fe69402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31062ad5d3b49972 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:17.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.542+0000 7fe6a9400640 1 -- 172.21.15.114:0/3037207624 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe694037070 con 0x7fe6ac073260 2024-05-27T09:38:17.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.542+0000 7fe6a9400640 1 -- 172.21.15.114:0/3037207624 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe694032440 con 0x7fe6ac073260 2024-05-27T09:38:17.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.542+0000 7fe6a9400640 1 -- 172.21.15.114:0/3037207624 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe694032830 con 0x7fe6ac073260 2024-05-27T09:38:17.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.543+0000 7fe6b1000640 1 -- 172.21.15.114:0/3037207624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6ac073260 msgr2=0x7fe6ac06cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:17.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.543+0000 7fe6b1000640 1 --2- 172.21.15.114:0/3037207624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6ac073260 0x7fe6ac06cbd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0c5350 tx=0x7fe69402a250 comp rx=0 tx=0).stop 2024-05-27T09:38:17.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.543+0000 7fe6b1000640 1 -- 172.21.15.114:0/3037207624 shutdown_connections 2024-05-27T09:38:17.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.543+0000 7fe6b1000640 1 --2- 172.21.15.114:0/3037207624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6ac073260 0x7fe6ac06cbd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:17.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.543+0000 7fe6b1000640 1 --2- 172.21.15.114:0/3037207624 >> v2:172.21.15.114:3300/0 conn(0x7fe6ac0c9830 0x7fe6ac06c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:17.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.543+0000 7fe6b1000640 1 -- 172.21.15.114:0/3037207624 >> 172.21.15.114:0/3037207624 conn(0x7fe6ac0cd9e0 msgr2=0x7fe6ac0cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:17.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.543+0000 7fe6b1000640 1 -- 172.21.15.114:0/3037207624 shutdown_connections 2024-05-27T09:38:17.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.544+0000 7fe6b1000640 1 -- 172.21.15.114:0/3037207624 wait complete. 2024-05-27T09:38:17.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.546+0000 7fe6b1000640 1 Processor -- start 2024-05-27T09:38:17.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.546+0000 7fe6b1000640 1 -- start start 2024-05-27T09:38:17.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.547+0000 7fe6b1000640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe6ac073260 0x7fe6ac0c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:17.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.547+0000 7fe6b1000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6ac0c9830 0x7fe6ac0cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:17.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.547+0000 7fe6b1000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6ac15a830 0x7fe6ac0ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:17.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.547+0000 7fe6b1000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac0706c0 con 0x7fe6ac15a830 2024-05-27T09:38:17.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.547+0000 7fe6b1000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac0749d0 con 0x7fe6ac0c9830 2024-05-27T09:38:17.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.547+0000 7fe6b1000640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7fe6ac073940 con 0x7fe6ac073260 2024-05-27T09:38:17.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.548+0000 7fe6aa800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6ac15a830 0x7fe6ac0ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:17.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.548+0000 7fe6aa800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6ac15a830 0x7fe6ac0ccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49260/0 (socket says 172.21.15.114:49260) 2024-05-27T09:38:17.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.548+0000 7fe6aa800640 1 -- 172.21.15.114:0/1561772146 learned_addr learned my addr 172.21.15.114:0/1561772146 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:38:17.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.548+0000 7fe6a9e00640 1 --2- 172.21.15.114:0/1561772146 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe6ac073260 0x7fe6ac0c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:17.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.548+0000 7fe6a3e00640 1 --2- 172.21.15.114:0/1561772146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6ac0c9830 0x7fe6ac0cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:17.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.548+0000 7fe6aa800640 1 -- 172.21.15.114:0/1561772146 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe6ac073260 msgr2=0x7fe6ac0c3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:17.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.548+0000 7fe6aa800640 1 --2- 172.21.15.114:0/1561772146 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe6ac073260 0x7fe6ac0c3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:17.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.548+0000 7fe6aa800640 1 -- 172.21.15.114:0/1561772146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6ac0c9830 msgr2=0x7fe6ac0cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:17.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.548+0000 7fe6aa800640 1 --2- 172.21.15.114:0/1561772146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6ac0c9830 0x7fe6ac0cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:17.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.548+0000 7fe6aa800640 1 -- 172.21.15.114:0/1561772146 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ac1589e0 con 0x7fe6ac15a830 2024-05-27T09:38:17.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.548+0000 7fe6a3e00640 1 --2- 172.21.15.114:0/1561772146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6ac0c9830 0x7fe6ac0cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:38:17.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.549+0000 7fe6aa800640 1 --2- 172.21.15.114:0/1561772146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6ac15a830 0x7fe6ac0ccea0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400c730 tx=0x7fe6a400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:17.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.549+0000 7fe6a2000640 1 -- 172.21.15.114:0/1561772146 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4003b70 con 0x7fe6ac15a830 2024-05-27T09:38:17.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.549+0000 7fe6a2000640 1 -- 172.21.15.114:0/1561772146 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe6a4003d30 con 0x7fe6ac15a830 2024-05-27T09:38:17.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.549+0000 7fe6b1000640 1 -- 172.21.15.114:0/1561772146 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6ac1f3710 con 0x7fe6ac15a830 2024-05-27T09:38:17.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.549+0000 7fe6a2000640 1 -- 172.21.15.114:0/1561772146 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a400aab0 con 0x7fe6ac15a830 2024-05-27T09:38:17.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.549+0000 7fe6b1000640 1 -- 172.21.15.114:0/1561772146 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6ac1f2c30 con 0x7fe6ac15a830 2024-05-27T09:38:17.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.550+0000 7fe6b1000640 1 -- 172.21.15.114:0/1561772146 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6ac06a380 con 0x7fe6ac15a830 2024-05-27T09:38:17.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.551+0000 7fe6a2000640 1 -- 172.21.15.114:0/1561772146 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe6a400ac50 con 0x7fe6ac15a830 2024-05-27T09:38:17.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.552+0000 7fe6a2000640 1 --2- 172.21.15.114:0/1561772146 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe6880762d0 0x7fe6880787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:17.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.552+0000 7fe6a2000640 1 -- 172.21.15.114:0/1561772146 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6512+0+0 (secure 0 0 0) 0x7fe6a40977e0 con 0x7fe6ac15a830 2024-05-27T09:38:17.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.553+0000 7fe6a9e00640 1 --2- 172.21.15.114:0/1561772146 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe6880762d0 0x7fe6880787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:17.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.553+0000 7fe6a9e00640 1 --2- 172.21.15.114:0/1561772146 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe6880762d0 0x7fe6880787e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe69c009b30 tx=0x7fe69c009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:17.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.557+0000 7fe6a2000640 1 -- 172.21.15.114:0/1561772146 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe6a40606c0 con 0x7fe6ac15a830 2024-05-27T09:38:17.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:17.767+0000 7fe6b1000640 1 -- 172.21.15.114:0/1561772146 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe6ac158770 con 0x7fe6ac15a830 2024-05-27T09:38:18.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.227+0000 7fe6a2000640 1 -- 172.21.15.114:0/1561772146 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v52) ==== 181+0+0 (secure 0 0 0) 0x7fe6a40649c0 con 0x7fe6ac15a830 2024-05-27T09:38:18.227 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_metadata' removed 2024-05-27T09:38:18.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.231+0000 7fe6b1000640 1 -- 172.21.15.114:0/1561772146 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe6880762d0 msgr2=0x7fe6880787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:18.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.231+0000 7fe6b1000640 1 --2- 172.21.15.114:0/1561772146 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe6880762d0 0x7fe6880787e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe69c009b30 tx=0x7fe69c009240 comp rx=0 tx=0).stop 2024-05-27T09:38:18.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.231+0000 7fe6b1000640 1 -- 172.21.15.114:0/1561772146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6ac15a830 msgr2=0x7fe6ac0ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:18.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.231+0000 7fe6b1000640 1 --2- 172.21.15.114:0/1561772146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6ac15a830 0x7fe6ac0ccea0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400c730 tx=0x7fe6a400cbf0 comp rx=0 tx=0).stop 2024-05-27T09:38:18.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.232+0000 7fe6b1000640 1 -- 172.21.15.114:0/1561772146 shutdown_connections 2024-05-27T09:38:18.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.232+0000 7fe6b1000640 1 --2- 172.21.15.114:0/1561772146 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7fe6880762d0 0x7fe6880787e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:18.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.232+0000 7fe6b1000640 1 --2- 172.21.15.114:0/1561772146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe6ac15a830 0x7fe6ac0ccea0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:18.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.232+0000 7fe6b1000640 1 --2- 172.21.15.114:0/1561772146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6ac0c9830 0x7fe6ac0cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:18.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.232+0000 7fe6b1000640 1 --2- 172.21.15.114:0/1561772146 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe6ac073260 0x7fe6ac0c3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:18.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.232+0000 7fe6b1000640 1 -- 172.21.15.114:0/1561772146 >> 172.21.15.114:0/1561772146 conn(0x7fe6ac0cd9e0 msgr2=0x7fe6ac153ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:18.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.237+0000 7fe6b1000640 1 -- 172.21.15.114:0/1561772146 shutdown_connections 2024-05-27T09:38:18.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.237+0000 7fe6b1000640 1 -- 172.21.15.114:0/1561772146 wait complete. 2024-05-27T09:38:18.284 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-27T09:38:18.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.530+0000 7ff938a00640 1 Processor -- start 2024-05-27T09:38:18.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.530+0000 7ff938a00640 1 -- start start 2024-05-27T09:38:18.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.532+0000 7ff938a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9340b37a0 0x7ff93415c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:18.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.532+0000 7ff938a00640 1 -- --> v1:172.21.15.114:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9340c84a0 con 0x7ff9340c7700 2024-05-27T09:38:18.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.532+0000 7ff938a00640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9340c8780 con 0x7ff9340c71b0 2024-05-27T09:38:18.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.532+0000 7ff938a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7ff9340c3020 con 0x7ff9340b37a0 2024-05-27T09:38:18.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.532+0000 7ff92be00640 1 --1- >> v1:172.21.15.114:6789/0 conn(0x7ff9340c7700 0x7ff934161bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.114:6789/0 says I am v1:172.21.15.114:38158/0 (socket says 172.21.15.114:38158) 2024-05-27T09:38:18.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.532+0000 7ff92be00640 1 -- 172.21.15.114:0/2663377765 learned_addr learned my addr 172.21.15.114:0/2663377765 (peer_addr_for_me v1:172.21.15.114:0/0) 2024-05-27T09:38:18.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.532+0000 7ff933200640 1 --2- 172.21.15.114:0/2663377765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9340b37a0 0x7ff93415c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:18.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.532+0000 7ff931e00640 1 -- 172.21.15.114:0/2663377765 <== mon.0 v1:172.21.15.114:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2011186983 0 0) 0x7ff9340c84a0 con 0x7ff9340c7700 2024-05-27T09:38:18.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.532+0000 7ff931e00640 1 -- 172.21.15.114:0/2663377765 --> v1:172.21.15.114:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff918005730 con 0x7ff9340c7700 2024-05-27T09:38:18.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.532+0000 7ff931e00640 1 -- 172.21.15.114:0/2663377765 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2731834100 0 0) 0x7ff9340c8780 con 0x7ff9340c71b0 2024-05-27T09:38:18.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.533+0000 7ff931e00640 1 -- 172.21.15.114:0/2663377765 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9340c84a0 con 0x7ff9340c71b0 2024-05-27T09:38:18.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.533+0000 7ff933200640 1 -- 172.21.15.114:0/2663377765 >> v1:172.21.15.163:6789/0 conn(0x7ff9340c71b0 legacy=0x7ff93406fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:18.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.533+0000 7ff933200640 1 -- 172.21.15.114:0/2663377765 >> v1:172.21.15.114:6789/0 conn(0x7ff9340c7700 legacy=0x7ff934161bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:18.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.533+0000 7ff933200640 1 -- 172.21.15.114:0/2663377765 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9340c8a60 con 0x7ff9340b37a0 2024-05-27T09:38:18.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.534+0000 7ff933200640 1 --2- 172.21.15.114:0/2663377765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9340b37a0 0x7ff93415c890 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff9340c31a0 tx=0x7ff92c02a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=319a62ae5720881f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:18.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.534+0000 7ff931e00640 1 -- 172.21.15.114:0/2663377765 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c007740 con 0x7ff9340b37a0 2024-05-27T09:38:18.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.534+0000 7ff931e00640 1 -- 172.21.15.114:0/2663377765 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7ff92c02ae50 con 0x7ff9340b37a0 2024-05-27T09:38:18.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.534+0000 7ff931e00640 1 -- 172.21.15.114:0/2663377765 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c007b20 con 0x7ff9340b37a0 2024-05-27T09:38:18.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.535+0000 7ff938a00640 1 -- 172.21.15.114:0/2663377765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9340b37a0 msgr2=0x7ff93415c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:18.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.535+0000 7ff938a00640 1 --2- 172.21.15.114:0/2663377765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9340b37a0 0x7ff93415c890 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff9340c31a0 tx=0x7ff92c02a630 comp rx=0 tx=0).stop 2024-05-27T09:38:18.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.535+0000 7ff938a00640 1 -- 172.21.15.114:0/2663377765 shutdown_connections 2024-05-27T09:38:18.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.535+0000 7ff938a00640 1 --2- 172.21.15.114:0/2663377765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9340b37a0 0x7ff93415c890 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:18.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.535+0000 7ff938a00640 1 -- 172.21.15.114:0/2663377765 >> 172.21.15.114:0/2663377765 conn(0x7ff93406c690 msgr2=0x7ff93406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:18.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.535+0000 7ff938a00640 1 -- 172.21.15.114:0/2663377765 shutdown_connections 2024-05-27T09:38:18.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.535+0000 7ff938a00640 1 -- 172.21.15.114:0/2663377765 wait complete. 2024-05-27T09:38:18.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.537+0000 7ff938a00640 1 Processor -- start 2024-05-27T09:38:18.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.537+0000 7ff938a00640 1 -- start start 2024-05-27T09:38:18.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.539+0000 7ff938a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9340b37a0 0x7ff9341f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:18.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.539+0000 7ff938a00640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9340c71b0 0x7ff93414d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:18.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.539+0000 7ff938a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9340c7700 0x7ff93414d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:18.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.539+0000 7ff938a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff9340c2990 con 0x7ff9340c7700 2024-05-27T09:38:18.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.539+0000 7ff938a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff9340c35d0 con 0x7ff9340b37a0 2024-05-27T09:38:18.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.539+0000 7ff938a00640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 -- 0x7ff9340c2b10 con 0x7ff9340c71b0 2024-05-27T09:38:18.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.539+0000 7ff933200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9340c7700 0x7ff93414d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:18.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.539+0000 7ff933200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9340c7700 0x7ff93414d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49272/0 (socket says 172.21.15.114:49272) 2024-05-27T09:38:18.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.539+0000 7ff933200640 1 -- 172.21.15.114:0/3733833785 learned_addr learned my addr 172.21.15.114:0/3733833785 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-05-27T09:38:18.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.539+0000 7ff92be00640 1 --2- 172.21.15.114:0/3733833785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9340c71b0 0x7ff93414d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:18.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.539+0000 7ff932800640 1 --2- 172.21.15.114:0/3733833785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9340b37a0 0x7ff9341f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:18.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.540+0000 7ff933200640 1 -- 172.21.15.114:0/3733833785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9340c71b0 msgr2=0x7ff93414d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:18.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.540+0000 7ff933200640 1 --2- 172.21.15.114:0/3733833785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9340c71b0 0x7ff93414d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:18.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.540+0000 7ff933200640 1 -- 172.21.15.114:0/3733833785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9340b37a0 msgr2=0x7ff9341f1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:18.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.540+0000 7ff933200640 1 --2- 172.21.15.114:0/3733833785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9340b37a0 0x7ff9341f1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:18.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.540+0000 7ff933200640 1 -- 172.21.15.114:0/3733833785 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9341596f0 con 0x7ff9340c7700 2024-05-27T09:38:18.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.540+0000 7ff933200640 1 --2- 172.21.15.114:0/3733833785 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9340c7700 0x7ff93414d750 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009eb0 tx=0x7ff92c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:18.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.540+0000 7ff92aa00640 1 -- 172.21.15.114:0/3733833785 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c03f070 con 0x7ff9340c7700 2024-05-27T09:38:18.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.540+0000 7ff92aa00640 1 -- 172.21.15.114:0/3733833785 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7ff92c03b820 con 0x7ff9340c7700 2024-05-27T09:38:18.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.540+0000 7ff92aa00640 1 -- 172.21.15.114:0/3733833785 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c03bbf0 con 0x7ff9340c7700 2024-05-27T09:38:18.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.540+0000 7ff938a00640 1 -- 172.21.15.114:0/3733833785 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff934158690 con 0x7ff9340c7700 2024-05-27T09:38:18.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.540+0000 7ff938a00640 1 -- 172.21.15.114:0/3733833785 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff934158c10 con 0x7ff9340c7700 2024-05-27T09:38:18.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.542+0000 7ff938a00640 1 -- 172.21.15.114:0/3733833785 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff93406a380 con 0x7ff9340c7700 2024-05-27T09:38:18.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.543+0000 7ff92aa00640 1 -- 172.21.15.114:0/3733833785 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff92c005d10 con 0x7ff9340c7700 2024-05-27T09:38:18.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.544+0000 7ff92aa00640 1 --2- 172.21.15.114:0/3733833785 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff9100762d0 0x7ff9100787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:18.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.544+0000 7ff92aa00640 1 -- 172.21.15.114:0/3733833785 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6074+0+0 (secure 0 0 0) 0x7ff92c0b7ff0 con 0x7ff9340c7700 2024-05-27T09:38:18.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.544+0000 7ff932800640 1 --2- 172.21.15.114:0/3733833785 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff9100762d0 0x7ff9100787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:18.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.545+0000 7ff932800640 1 --2- 172.21.15.114:0/3733833785 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff9100762d0 0x7ff9100787e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff924000f60 tx=0x7ff924009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:18.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.548+0000 7ff92aa00640 1 -- 172.21.15.114:0/3733833785 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff92c04a020 con 0x7ff9340c7700 2024-05-27T09:38:18.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:18.755+0000 7ff938a00640 1 -- 172.21.15.114:0/3733833785 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff9341f2790 con 0x7ff9340c7700 2024-05-27T09:38:19.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:19.254+0000 7ff92aa00640 1 -- 172.21.15.114:0/3733833785 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v53) ==== 169+0+0 (secure 0 0 0) 0x7ff92c0810a0 con 0x7ff9340c7700 2024-05-27T09:38:19.255 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_data' removed 2024-05-27T09:38:19.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:19.258+0000 7ff938a00640 1 -- 172.21.15.114:0/3733833785 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff9100762d0 msgr2=0x7ff9100787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:19.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:19.258+0000 7ff938a00640 1 --2- 172.21.15.114:0/3733833785 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff9100762d0 0x7ff9100787e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff924000f60 tx=0x7ff924009040 comp rx=0 tx=0).stop 2024-05-27T09:38:19.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:19.258+0000 7ff938a00640 1 -- 172.21.15.114:0/3733833785 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9340c7700 msgr2=0x7ff93414d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:19.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:19.258+0000 7ff938a00640 1 --2- 172.21.15.114:0/3733833785 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9340c7700 0x7ff93414d750 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009eb0 tx=0x7ff92c004a40 comp rx=0 tx=0).stop 2024-05-27T09:38:19.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:19.259+0000 7ff938a00640 1 -- 172.21.15.114:0/3733833785 shutdown_connections 2024-05-27T09:38:19.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:19.259+0000 7ff938a00640 1 --2- 172.21.15.114:0/3733833785 >> [v2:172.21.15.163:6832/1122018204,v1:172.21.15.163:6833/1122018204] conn(0x7ff9100762d0 0x7ff9100787e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:19.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:19.259+0000 7ff938a00640 1 --2- 172.21.15.114:0/3733833785 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9340c7700 0x7ff93414d750 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:19.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:19.259+0000 7ff938a00640 1 --2- 172.21.15.114:0/3733833785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9340c71b0 0x7ff93414d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:19.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:19.259+0000 7ff938a00640 1 --2- 172.21.15.114:0/3733833785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9340b37a0 0x7ff9341f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:19.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:19.259+0000 7ff938a00640 1 -- 172.21.15.114:0/3733833785 >> 172.21.15.114:0/3733833785 conn(0x7ff93406c690 msgr2=0x7ff9340c9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:19.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:19.262+0000 7ff938a00640 1 -- 172.21.15.114:0/3733833785 shutdown_connections 2024-05-27T09:38:19.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-05-27T09:38:19.262+0000 7ff938a00640 1 -- 172.21.15.114:0/3733833785 wait complete. 2024-05-27T09:38:19.305 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-27T09:38:19.305 ERROR:tasks.ceph.mds.a:tried to stop a non-running daemon 2024-05-27T09:38:19.311 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-27T09:38:19.311 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:38:19.358 INFO:tasks.ceph.mds.c:Stopped 2024-05-27T09:38:19.359 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-05-27T09:38:19.359 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:38:19.412 INFO:tasks.ceph.mds.e:Stopped 2024-05-27T09:38:19.412 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-27T09:38:19.412 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:38:19.427 INFO:tasks.ceph.mds.b:Stopped 2024-05-27T09:38:19.427 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-27T09:38:19.427 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:38:19.481 INFO:tasks.ceph.mds.d:Stopped 2024-05-27T09:38:19.482 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-05-27T09:38:19.482 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:38:19.532 INFO:tasks.ceph.mds.f:Stopped 2024-05-27T09:38:19.533 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-27T09:38:19.533 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-27T09:38:19.533 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:38:21.525 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:38:21.533 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:38:25.635 INFO:tasks.ceph.osd.0:Stopped 2024-05-27T09:38:25.636 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-27T09:38:25.636 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:38:25.707 INFO:tasks.ceph.osd.1:Stopped 2024-05-27T09:38:25.708 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-27T09:38:25.708 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:38:25.983 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-05-27T09:38:31.810 INFO:tasks.ceph.osd.2:Stopped 2024-05-27T09:38:31.810 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-27T09:38:31.810 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:38:31.885 INFO:tasks.ceph.osd.3:Stopped 2024-05-27T09:38:31.885 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-27T09:38:31.886 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:38:31.918 INFO:tasks.ceph.osd.4:Stopped 2024-05-27T09:38:31.918 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-27T09:38:31.918 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:38:31.993 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-05-27T09:38:31.993 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-05-27T09:38:32.004 INFO:tasks.ceph.osd.5:Stopped 2024-05-27T09:38:32.004 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-27T09:38:32.004 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:38:32.040 INFO:tasks.ceph.osd.6:Stopped 2024-05-27T09:38:32.040 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-27T09:38:32.040 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:38:32.116 INFO:tasks.ceph.osd.7:Stopped 2024-05-27T09:38:32.116 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-27T09:38:32.116 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-27T09:38:32.116 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:38:37.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-05-27T09:38:37.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-05-27T09:38:38.218 INFO:tasks.ceph.mgr.y:Stopped 2024-05-27T09:38:38.219 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-27T09:38:38.219 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:38:38.249 INFO:tasks.ceph.mgr.x:Stopped 2024-05-27T09:38:38.250 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-27T09:38:38.250 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-27T09:38:38.251 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:38:38.325 INFO:tasks.ceph.mon.a:Stopped 2024-05-27T09:38:38.325 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-27T09:38:38.325 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:38:38.338 INFO:tasks.ceph.mon.b:Stopped 2024-05-27T09:38:38.339 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-27T09:38:38.339 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T09:38:38.392 INFO:tasks.ceph.mon.c:Stopped 2024-05-27T09:38:38.393 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-27T09:38:38.393 DEBUG:teuthology.orchestra.run.smithi114:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v 'Replacing daemon mds' | head -n 1 2024-05-27T09:38:38.517 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi114.front.sepia.ceph.com 2024-05-27T09:38:38.518 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-27T09:38:39.057 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi114.front.sepia.ceph.com 2024-05-27T09:38:39.058 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-27T09:38:39.204 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi114.front.sepia.ceph.com 2024-05-27T09:38:39.204 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-27T09:38:39.333 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi114.front.sepia.ceph.com 2024-05-27T09:38:39.333 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-27T09:38:39.481 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi163.front.sepia.ceph.com 2024-05-27T09:38:39.482 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-27T09:38:40.105 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi163.front.sepia.ceph.com 2024-05-27T09:38:40.106 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-27T09:38:40.192 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi163.front.sepia.ceph.com 2024-05-27T09:38:40.192 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-27T09:38:40.256 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi163.front.sepia.ceph.com 2024-05-27T09:38:40.256 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-27T09:38:40.308 INFO:tasks.ceph:Archiving mon data... 2024-05-27T09:38:40.310 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727681/data/mon.a.tgz 2024-05-27T09:38:40.310 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2024-05-27T09:38:40.366 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.FtoY5GTqk3 2024-05-27T09:38:40.367 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.FtoY5GTqk3 2024-05-27T09:38:40.537 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.FtoY5GTqk3 2024-05-27T09:38:40.677 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.FtoY5GTqk3 is 339KB 2024-05-27T09:38:40.789 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.FtoY5GTqk3 2024-05-27T09:38:40.846 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727681/data/mon.b.tgz 2024-05-27T09:38:40.846 DEBUG:teuthology.orchestra.run.smithi163:> mktemp 2024-05-27T09:38:40.865 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.61A5iinEgA 2024-05-27T09:38:40.865 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.61A5iinEgA 2024-05-27T09:38:41.015 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.61A5iinEgA 2024-05-27T09:38:41.104 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.61A5iinEgA is 353KB 2024-05-27T09:38:41.167 DEBUG:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.61A5iinEgA 2024-05-27T09:38:41.183 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727681/data/mon.c.tgz 2024-05-27T09:38:41.183 DEBUG:teuthology.orchestra.run.smithi163:> mktemp 2024-05-27T09:38:41.241 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.VcRbeCVW0t 2024-05-27T09:38:41.242 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.VcRbeCVW0t 2024-05-27T09:38:41.375 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.VcRbeCVW0t 2024-05-27T09:38:41.461 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.VcRbeCVW0t is 353KB 2024-05-27T09:38:41.500 DEBUG:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.VcRbeCVW0t 2024-05-27T09:38:41.516 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-27T09:38:41.516 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-27T09:38:41.524 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-27T09:38:41.661 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-27T09:38:41.677 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-27T09:38:41.677 INFO:tasks.ceph:Archiving crash dumps... 2024-05-27T09:38:41.679 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727681/remote/smithi114/crash 2024-05-27T09:38:41.680 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-27T09:38:41.762 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727681/remote/smithi163/crash 2024-05-27T09:38:41.763 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-27T09:38:41.786 INFO:tasks.ceph:Shutting down logrotate 2024-05-27T09:38:41.786 DEBUG:teuthology.orchestra.run.smithi114:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:38:41.792 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-27T09:38:41.915 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-27T09:38:42.000 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-27T09:38:42.024 INFO:tasks.ceph:Compressing logs... 2024-05-27T09:38:42.024 DEBUG:teuthology.orchestra.run.smithi114:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-27T09:38:42.032 DEBUG:teuthology.orchestra.run.smithi163:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-27T09:38:42.092 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-27T09:38:42.092 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38131.log 2024-05-27T09:38:42.092 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-05-27T09:38:42.093 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.38131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38260.log 2024-05-27T09:38:42.093 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-mds.f.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38131.log.gz 2024-05-27T09:38:42.093 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-27T09:38:42.093 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-27T09:38:42.093 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.38260.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-27T09:38:42.094 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38260.log.gz 2024-05-27T09:38:42.094 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-27T09:38:42.094 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-27T09:38:42.094 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38180.log 2024-05-27T09:38:42.094 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-27T09:38:42.094 INFO:teuthology.orchestra.run.smithi163.stderr: 94.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-27T09:38:42.094 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.38180.log: 92.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-27T09:38:42.095 INFO:teuthology.orchestra.run.smithi163.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.38180.log.gz 2024-05-27T09:38:42.095 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-27T09:38:42.095 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-27T09:38:42.095 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-mon.c.log: 93.6% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-27T09:38:42.096 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38220.log 2024-05-27T09:38:42.108 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44983.log 2024-05-27T09:38:42.109 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49499.log 2024-05-27T09:38:42.110 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38300.log 2024-05-27T09:38:42.110 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.38220.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38220.log.gz 2024-05-27T09:38:42.111 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50205.log 2024-05-27T09:38:42.111 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44983.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44983.log.gz 2024-05-27T09:38:42.112 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51452.log 2024-05-27T09:38:42.112 INFO:teuthology.orchestra.run.smithi114.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49499.log.gz 2024-05-27T09:38:42.113 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.50205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43604.log 2024-05-27T09:38:42.113 INFO:teuthology.orchestra.run.smithi114.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50205.log.gz 2024-05-27T09:38:42.114 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.51452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45306.log 2024-05-27T09:38:42.114 INFO:teuthology.orchestra.run.smithi114.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51452.log.gz 2024-05-27T09:38:42.114 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.0.43604.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43604.log.gz 2024-05-27T09:38:42.115 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46693.log 2024-05-27T09:38:42.116 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44077.log 2024-05-27T09:38:42.116 INFO:teuthology.orchestra.run.smithi114.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45306.log.gz 2024-05-27T09:38:42.117 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42864.log 2024-05-27T09:38:42.117 INFO:teuthology.orchestra.run.smithi114.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46693.log.gz 2024-05-27T09:38:42.118 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43623.log 2024-05-27T09:38:42.118 INFO:teuthology.orchestra.run.smithi114.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44077.log.gz 2024-05-27T09:38:42.119 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49756.log 2024-05-27T09:38:42.119 INFO:teuthology.orchestra.run.smithi114.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42864.log.gz 2024-05-27T09:38:42.120 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.0.43623.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.0.43623.log.gz --verbose 2024-05-27T09:38:42.120 INFO:teuthology.orchestra.run.smithi114.stderr: -- /var/log/ceph/ceph-client.admin.51635.log 2024-05-27T09:38:42.121 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48487.log 2024-05-27T09:38:42.122 INFO:teuthology.orchestra.run.smithi114.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49756.log.gz 2024-05-27T09:38:42.123 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38380.log 2024-05-27T09:38:42.123 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.51635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53281.log 2024-05-27T09:38:42.123 INFO:teuthology.orchestra.run.smithi114.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51635.log.gz 2024-05-27T09:38:42.124 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41296.log 2024-05-27T09:38:42.124 INFO:teuthology.orchestra.run.smithi114.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48487.log.gz 2024-05-27T09:38:42.125 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.53281.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53281.log.gz 2024-05-27T09:38:42.125 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46255.log 2024-05-27T09:38:42.131 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.38300.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38300.log.gz 2024-05-27T09:38:42.131 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38089.log 2024-05-27T09:38:42.132 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.38380.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38380.log.gz 2024-05-27T09:38:42.132 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-27T09:38:42.132 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.38089.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38089.log.gz 2024-05-27T09:38:42.132 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-27T09:38:42.133 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-mds.d.log: 93.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-27T09:38:42.133 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38340.log 2024-05-27T09:38:42.138 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43035.log 2024-05-27T09:38:42.138 INFO:teuthology.orchestra.run.smithi114.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41296.log.gz 2024-05-27T09:38:42.138 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48182.log 2024-05-27T09:38:42.138 INFO:teuthology.orchestra.run.smithi114.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46255.log.gz 2024-05-27T09:38:42.138 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50844.log 2024-05-27T09:38:42.138 INFO:teuthology.orchestra.run.smithi114.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.43035.log.gz 2024-05-27T09:38:42.138 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45033.log 2024-05-27T09:38:42.139 INFO:teuthology.orchestra.run.smithi114.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48182.log.gz 2024-05-27T09:38:42.139 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.50844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48224.log 2024-05-27T09:38:42.139 INFO:teuthology.orchestra.run.smithi114.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50844.log.gz 2024-05-27T09:38:42.139 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45033.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45033.log.gz 2024-05-27T09:38:42.139 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45075.log 2024-05-27T09:38:42.139 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48673.log 2024-05-27T09:38:42.139 INFO:teuthology.orchestra.run.smithi114.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48224.log.gz 2024-05-27T09:38:42.139 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51325.log 2024-05-27T09:38:42.139 INFO:teuthology.orchestra.run.smithi114.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45075.log.gz 2024-05-27T09:38:42.140 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48673.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48673.log.gz 2024-05-27T09:38:42.140 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51533.log 2024-05-27T09:38:42.140 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.51325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50558.log 2024-05-27T09:38:42.140 INFO:teuthology.orchestra.run.smithi114.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51325.log.gz 2024-05-27T09:38:42.140 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.51533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45708.log 2024-05-27T09:38:42.140 INFO:teuthology.orchestra.run.smithi114.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51533.log.gz 2024-05-27T09:38:42.140 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.50558.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50558.log.gz 2024-05-27T09:38:42.140 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50714.log 2024-05-27T09:38:42.142 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48782.log 2024-05-27T09:38:42.142 INFO:teuthology.orchestra.run.smithi114.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45708.log.gz 2024-05-27T09:38:42.142 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.50714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50121.log 2024-05-27T09:38:42.143 INFO:teuthology.orchestra.run.smithi114.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50714.log.gz 2024-05-27T09:38:42.143 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48782.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48782.log.gz 2024-05-27T09:38:42.143 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43101.log 2024-05-27T09:38:42.143 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.50121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51929.log 2024-05-27T09:38:42.143 INFO:teuthology.orchestra.run.smithi114.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50121.log.gz 2024-05-27T09:38:42.144 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43924.log 2024-05-27T09:38:42.144 INFO:teuthology.orchestra.run.smithi114.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.43101.log.gz 2024-05-27T09:38:42.145 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.51929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44617.log 2024-05-27T09:38:42.145 INFO:teuthology.orchestra.run.smithi114.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51929.log.gz 2024-05-27T09:38:42.145 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43924.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43924.log.gz 2024-05-27T09:38:42.146 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48529.log 2024-05-27T09:38:42.147 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53084.log 2024-05-27T09:38:42.147 INFO:teuthology.orchestra.run.smithi114.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44617.log.gz 2024-05-27T09:38:42.148 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48529.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-27T09:38:42.148 INFO:teuthology.orchestra.run.smithi114.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48529.log.gz 2024-05-27T09:38:42.149 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.38340.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38340.log.gz 2024-05-27T09:38:42.149 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.53084.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53084.log.gz 2024-05-27T09:38:42.149 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44899.log 2024-05-27T09:38:42.150 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48613.log 2024-05-27T09:38:42.151 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50289.log 2024-05-27T09:38:42.152 INFO:teuthology.orchestra.run.smithi114.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44899.log.gz 2024-05-27T09:38:42.152 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48613.log: 86.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-27T09:38:42.152 INFO:teuthology.orchestra.run.smithi114.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48613.log.gz 2024-05-27T09:38:42.153 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42201.log 2024-05-27T09:38:42.154 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.50289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48400.log 2024-05-27T09:38:42.154 INFO:teuthology.orchestra.run.smithi114.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50289.log.gz 2024-05-27T09:38:42.155 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42201.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-27T09:38:42.155 INFO:teuthology.orchestra.run.smithi114.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42201.log.gz 2024-05-27T09:38:42.156 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48400.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48400.log.gz 2024-05-27T09:38:42.157 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42177.log 2024-05-27T09:38:42.157 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42738.log 2024-05-27T09:38:42.158 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43798.log 2024-05-27T09:38:42.159 INFO:teuthology.orchestra.run.smithi114.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42177.log.gz 2024-05-27T09:38:42.160 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42951.log 2024-05-27T09:38:42.160 INFO:teuthology.orchestra.run.smithi114.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42738.log.gz 2024-05-27T09:38:42.161 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45433.log 2024-05-27T09:38:42.161 INFO:teuthology.orchestra.run.smithi114.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43798.log.gz 2024-05-27T09:38:42.162 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42951.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42951.log.gz 2024-05-27T09:38:42.162 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49798.log 2024-05-27T09:38:42.163 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36992.log 2024-05-27T09:38:42.164 INFO:teuthology.orchestra.run.smithi114.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45433.log.gz 2024-05-27T09:38:42.164 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51096.log 2024-05-27T09:38:42.164 INFO:teuthology.orchestra.run.smithi114.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49798.log.gz 2024-05-27T09:38:42.165 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45349.log 2024-05-27T09:38:42.166 INFO:teuthology.orchestra.run.smithi114.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36992.log.gz 2024-05-27T09:38:42.167 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.51096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40930.log 2024-05-27T09:38:42.167 INFO:teuthology.orchestra.run.smithi114.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51096.log.gz 2024-05-27T09:38:42.168 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51677.log 2024-05-27T09:38:42.168 INFO:teuthology.orchestra.run.smithi114.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45349.log.gz/var/log/ceph/ceph-client.admin.40930.log: 2024-05-27T09:38:42.169 INFO:teuthology.orchestra.run.smithi114.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40930.log.gz 2024-05-27T09:38:42.169 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52706.log 2024-05-27T09:38:42.170 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.51677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49848.log 2024-05-27T09:38:42.170 INFO:teuthology.orchestra.run.smithi114.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51677.log.gz 2024-05-27T09:38:42.171 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.52706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48358.log 2024-05-27T09:38:42.171 INFO:teuthology.orchestra.run.smithi114.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52706.log.gz 2024-05-27T09:38:42.172 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50886.log 2024-05-27T09:38:42.173 INFO:teuthology.orchestra.run.smithi114.stderr: 87.6%/var/log/ceph/ceph-client.admin.48358.log: -- replaced with /var/log/ceph/ceph-client.admin.49848.log.gz 2024-05-27T09:38:42.174 INFO:teuthology.orchestra.run.smithi114.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48358.log.gz 2024-05-27T09:38:42.174 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42527.log 2024-05-27T09:38:42.174 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.50886.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-27T09:38:42.175 INFO:teuthology.orchestra.run.smithi114.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50886.log.gz 2024-05-27T09:38:42.176 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42527.log: gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.42527.log.gz --verbose 2024-05-27T09:38:42.176 INFO:teuthology.orchestra.run.smithi114.stderr: -- /var/log/ceph/ceph-client.admin.42181.log 2024-05-27T09:38:42.177 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42542.log 2024-05-27T09:38:42.178 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42181.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42181.log.gz 2024-05-27T09:38:42.178 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44027.log 2024-05-27T09:38:42.179 INFO:teuthology.orchestra.run.smithi114.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-27T09:38:42.179 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42542.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42542.log.gz 2024-05-27T09:38:42.180 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49890.log 2024-05-27T09:38:42.180 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43269.log 2024-05-27T09:38:42.181 INFO:teuthology.orchestra.run.smithi114.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44027.log.gz 2024-05-27T09:38:42.182 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50928.log 2024-05-27T09:38:42.182 INFO:teuthology.orchestra.run.smithi114.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49890.log.gz 2024-05-27T09:38:42.183 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52622.log 2024-05-27T09:38:42.184 INFO:teuthology.orchestra.run.smithi114.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43269.log.gz 2024-05-27T09:38:42.184 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.50928.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50928.log.gz 2024-05-27T09:38:42.184 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41855.log 2024-05-27T09:38:42.185 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.52622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40673.log 2024-05-27T09:38:42.186 INFO:teuthology.orchestra.run.smithi114.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52622.log.gz 2024-05-27T09:38:42.187 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41855.log: gzip -5 --verbose 88.6% -- -- replaced with /var/log/ceph/ceph-client.admin.41855.log.gz 2024-05-27T09:38:42.187 INFO:teuthology.orchestra.run.smithi114.stderr: /var/log/ceph/ceph-client.admin.52874.log 2024-05-27T09:38:42.187 INFO:teuthology.orchestra.run.smithi163.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-05-27T09:38:42.188 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51198.log 2024-05-27T09:38:42.189 INFO:teuthology.orchestra.run.smithi114.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40673.log.gz 2024-05-27T09:38:42.189 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.52874.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52874.log.gz 2024-05-27T09:38:42.189 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52454.log 2024-05-27T09:38:42.190 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.51198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50247.log 2024-05-27T09:38:42.190 INFO:teuthology.orchestra.run.smithi114.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51198.log.gz 2024-05-27T09:38:42.191 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.52454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41865.log 2024-05-27T09:38:42.191 INFO:teuthology.orchestra.run.smithi114.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52454.log.gz 2024-05-27T09:38:42.192 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.50247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41254.log 2024-05-27T09:38:42.193 INFO:teuthology.orchestra.run.smithi114.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50247.log.gz 2024-05-27T09:38:42.193 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41865.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41865.log.gz 2024-05-27T09:38:42.194 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53574.log 2024-05-27T09:38:42.195 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41254.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-27T09:38:42.195 INFO:teuthology.orchestra.run.smithi114.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41254.log.gz 2024-05-27T09:38:42.196 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.53574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46357.log 2024-05-27T09:38:42.196 INFO:teuthology.orchestra.run.smithi114.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53574.log.gz 2024-05-27T09:38:42.197 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41120.log 2024-05-27T09:38:42.199 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46357.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46357.log.gz 2024-05-27T09:38:42.199 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46819.log 2024-05-27T09:38:42.200 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41120.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41120.log.gz 2024-05-27T09:38:42.200 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47984.log 2024-05-27T09:38:42.201 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46819.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46819.log.gz 2024-05-27T09:38:42.201 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42536.log 2024-05-27T09:38:42.202 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44164.log 2024-05-27T09:38:42.202 INFO:teuthology.orchestra.run.smithi114.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47984.log.gz 2024-05-27T09:38:42.203 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42536.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42536.log.gz 2024-05-27T09:38:42.204 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41515.log 2024-05-27T09:38:42.205 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44164.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44164.log.gz 2024-05-27T09:38:42.205 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49397.log 2024-05-27T09:38:42.206 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41515.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41515.log.gz 2024-05-27T09:38:42.206 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46154.log 2024-05-27T09:38:42.207 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52916.log 2024-05-27T09:38:42.207 INFO:teuthology.orchestra.run.smithi114.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49397.log.gz 2024-05-27T09:38:42.208 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46154.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46154.log.gz 2024-05-27T09:38:42.208 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46861.log 2024-05-27T09:38:42.209 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.52916.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52916.log.gz 2024-05-27T09:38:42.210 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41860.log 2024-05-27T09:38:42.211 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46861.log: gzip -5 88.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.46861.log.gz -- 2024-05-27T09:38:42.211 INFO:teuthology.orchestra.run.smithi114.stderr: /var/log/ceph/ceph-client.admin.44121.log 2024-05-27T09:38:42.211 INFO:teuthology.orchestra.run.smithi163.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-27T09:38:42.212 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48860.log 2024-05-27T09:38:42.212 INFO:teuthology.orchestra.run.smithi114.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41860.log.gz 2024-05-27T09:38:42.213 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44121.log: gzip -5 --verbose -- 88.1% /var/log/ceph/ceph-client.admin.43882.log -- replaced with /var/log/ceph/ceph-client.admin.44121.log.gz 2024-05-27T09:38:42.213 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-05-27T09:38:42.214 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48860.log: gzip -5 --verbose -- 88.4% /var/log/ceph/ceph-client.admin.42780.log -- replaced with /var/log/ceph/ceph-client.admin.48860.log.gz 2024-05-27T09:38:42.214 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-05-27T09:38:42.215 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43882.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-27T09:38:42.215 INFO:teuthology.orchestra.run.smithi114.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43882.log.gz 2024-05-27T09:38:42.216 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41851.log 2024-05-27T09:38:42.216 INFO:teuthology.orchestra.run.smithi114.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42780.log.gz 2024-05-27T09:38:42.217 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50802.log 2024-05-27T09:38:42.219 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41851.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41851.log.gz 2024-05-27T09:38:42.219 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41641.log 2024-05-27T09:38:42.220 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.50802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51575.log 2024-05-27T09:38:42.220 INFO:teuthology.orchestra.run.smithi114.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50802.log.gz 2024-05-27T09:38:42.221 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42475.log 2024-05-27T09:38:42.222 INFO:teuthology.orchestra.run.smithi114.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41641.log.gz 2024-05-27T09:38:42.223 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.51575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42822.log 2024-05-27T09:38:42.223 INFO:teuthology.orchestra.run.smithi114.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51575.log.gz 2024-05-27T09:38:42.224 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41058.log 2024-05-27T09:38:42.224 INFO:teuthology.orchestra.run.smithi114.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42475.log.gz 2024-05-27T09:38:42.225 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42822.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-27T09:38:42.226 INFO:teuthology.orchestra.run.smithi114.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42822.log.gz 2024-05-27T09:38:42.227 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41058.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41058.log.gz 2024-05-27T09:38:42.227 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53406.log 2024-05-27T09:38:42.228 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-27T09:38:42.229 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.53406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42523.log 2024-05-27T09:38:42.230 INFO:teuthology.orchestra.run.smithi114.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53406.log.gz 2024-05-27T09:38:42.230 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51138.log 2024-05-27T09:38:42.232 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45222.log 2024-05-27T09:38:42.233 INFO:teuthology.orchestra.run.smithi114.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42523.log.gz 2024-05-27T09:38:42.234 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.51138.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51138.log.gz 2024-05-27T09:38:42.234 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43966.log 2024-05-27T09:38:42.235 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45222.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45222.log.gz 2024-05-27T09:38:42.235 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43167.log 2024-05-27T09:38:42.237 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43966.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-27T09:38:42.237 INFO:teuthology.orchestra.run.smithi114.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43966.log.gz 2024-05-27T09:38:42.239 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50979.log 2024-05-27T09:38:42.239 INFO:teuthology.orchestra.run.smithi114.stderr: 91.2%/var/log/ceph/ceph-osd.0.log: -- replaced with /var/log/ceph/ceph-client.admin.43167.log.gz 2024-05-27T09:38:42.240 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52115.log 2024-05-27T09:38:42.242 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.50979.log: 94.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-27T09:38:42.242 INFO:teuthology.orchestra.run.smithi114.stderr: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49714.log 2024-05-27T09:38:42.242 INFO:teuthology.orchestra.run.smithi114.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50979.log.gz 2024-05-27T09:38:42.243 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.52115.log: gzip -5 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52115.log.gz 2024-05-27T09:38:42.244 INFO:teuthology.orchestra.run.smithi114.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50412.log 2024-05-27T09:38:42.244 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49714.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49714.log.gz 2024-05-27T09:38:42.244 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50163.log 2024-05-27T09:38:42.246 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48068.log 2024-05-27T09:38:42.246 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.50412.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50412.log.gz 2024-05-27T09:38:42.247 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.50163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49977.log 2024-05-27T09:38:42.247 INFO:teuthology.orchestra.run.smithi114.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50163.log.gz 2024-05-27T09:38:42.248 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48068.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48068.log.gz 2024-05-27T09:38:42.248 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46609.log 2024-05-27T09:38:42.250 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51410.log 2024-05-27T09:38:42.250 INFO:teuthology.orchestra.run.smithi114.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49977.log.gz 2024-05-27T09:38:42.251 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46609.log: gzip -5 --verbose -- 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46609.log.gz 2024-05-27T09:38:42.251 INFO:teuthology.orchestra.run.smithi114.stderr: /var/log/ceph/ceph-client.admin.52315.log 2024-05-27T09:38:42.252 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.51410.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51410.log.gz 2024-05-27T09:38:42.252 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48026.log 2024-05-27T09:38:42.253 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.52315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51971.log 2024-05-27T09:38:42.254 INFO:teuthology.orchestra.run.smithi114.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52315.log.gz 2024-05-27T09:38:42.255 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46651.log 2024-05-27T09:38:42.255 INFO:teuthology.orchestra.run.smithi114.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48026.log.gz 2024-05-27T09:38:42.256 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.51971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41212.log 2024-05-27T09:38:42.256 INFO:teuthology.orchestra.run.smithi114.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51971.log.gz 2024-05-27T09:38:42.257 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46651.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46651.log.gz 2024-05-27T09:38:42.258 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52790.log 2024-05-27T09:38:42.259 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41212.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41212.log.gz 2024-05-27T09:38:42.259 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53000.log 2024-05-27T09:38:42.260 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.52790.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.52790.log.gz -5 2024-05-27T09:38:42.261 INFO:teuthology.orchestra.run.smithi114.stderr: --verbose -- /var/log/ceph/ceph.tmp-client.admin.30712.log 2024-05-27T09:38:42.262 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.53000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43756.log 2024-05-27T09:38:42.262 INFO:teuthology.orchestra.run.smithi114.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53000.log.gz 2024-05-27T09:38:42.263 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph.tmp-client.admin.30712.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30712.log.gz 2024-05-27T09:38:42.263 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36727.log 2024-05-27T09:38:42.264 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43756.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43756.log.gz 2024-05-27T09:38:42.264 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41473.log 2024-05-27T09:38:42.265 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46297.log 2024-05-27T09:38:42.266 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36727.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36727.log.gz 2024-05-27T09:38:42.267 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41473.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-27T09:38:42.267 INFO:teuthology.orchestra.run.smithi114.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41473.log.gz 2024-05-27T09:38:42.268 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53490.log 2024-05-27T09:38:42.268 INFO:teuthology.orchestra.run.smithi114.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46297.log.gz 2024-05-27T09:38:42.269 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52664.log 2024-05-27T09:38:42.271 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.53490.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53490.log.gz 2024-05-27T09:38:42.272 INFO:teuthology.orchestra.run.smithi114.stderr:gzip/var/log/ceph/ceph-client.admin.52664.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42193.log 2024-05-27T09:38:42.272 INFO:teuthology.orchestra.run.smithi114.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52664.log.gz 2024-05-27T09:38:42.273 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49672.log 2024-05-27T09:38:42.275 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42193.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42193.log.gz 2024-05-27T09:38:42.276 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48721.log/var/log/ceph/ceph-client.admin.49672.log: 2024-05-27T09:38:42.276 INFO:teuthology.orchestra.run.smithi114.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49672.log.gz 2024-05-27T09:38:42.277 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51719.log 2024-05-27T09:38:42.279 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48721.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48721.log.gz 2024-05-27T09:38:42.280 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44941.log 2024-05-27T09:38:42.280 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.51719.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51719.log.gz 2024-05-27T09:38:42.282 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52580.log 2024-05-27T09:38:42.282 INFO:teuthology.orchestra.run.smithi114.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44941.log.gz 2024-05-27T09:38:42.283 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48137.log 2024-05-27T09:38:42.285 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.52580.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52580.log.gz 2024-05-27T09:38:42.286 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41864.log 2024-05-27T09:38:42.286 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48137.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48137.log.gz 2024-05-27T09:38:42.287 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53324.log 2024-05-27T09:38:42.288 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41864.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41864.log.gz 2024-05-27T09:38:42.289 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46982.log 2024-05-27T09:38:42.290 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.53324.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53324.log.gz 2024-05-27T09:38:42.291 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46483.log 2024-05-27T09:38:42.292 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46982.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46982.log.gz 2024-05-27T09:38:42.293 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53738.log 2024-05-27T09:38:42.295 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46483.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46483.log.gz 2024-05-27T09:38:42.295 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51803.log 2024-05-27T09:38:42.297 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.53738.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53738.log.gz 2024-05-27T09:38:42.297 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37095.log 2024-05-27T09:38:42.298 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.51803.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51803.log.gz 2024-05-27T09:38:42.299 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53169.log 2024-05-27T09:38:42.300 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37095.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37095.log.gz 2024-05-27T09:38:42.301 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48571.log 2024-05-27T09:38:42.302 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.53169.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53169.log.gz 2024-05-27T09:38:42.303 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53366.log 2024-05-27T09:38:42.304 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48571.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48571.log.gz 2024-05-27T09:38:42.305 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53127.log 2024-05-27T09:38:42.306 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.53366.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53366.log.gz 2024-05-27T09:38:42.307 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52496.log/var/log/ceph/ceph-client.admin.53127.log: 2024-05-27T09:38:42.308 INFO:teuthology.orchestra.run.smithi114.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53127.log.gz 2024-05-27T09:38:42.309 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46921.log 2024-05-27T09:38:42.310 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.52496.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52496.log.gz 2024-05-27T09:38:42.311 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46441.log 2024-05-27T09:38:42.312 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46921.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46921.log.gz 2024-05-27T09:38:42.313 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53219.log 2024-05-27T09:38:42.314 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46441.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46441.log.gz 2024-05-27T09:38:42.315 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44857.log 2024-05-27T09:38:42.316 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.53219.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53219.log.gz 2024-05-27T09:38:42.317 INFO:teuthology.orchestra.run.smithi114.stderr:gzip/var/log/ceph/ceph-client.admin.44857.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41431.log 2024-05-27T09:38:42.318 INFO:teuthology.orchestra.run.smithi114.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44857.log.gz 2024-05-27T09:38:42.318 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52241.log 2024-05-27T09:38:42.320 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41431.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41431.log.gz 2024-05-27T09:38:42.321 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.52241.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-05-27T09:38:42.321 INFO:teuthology.orchestra.run.smithi114.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52241.log.gz 2024-05-27T09:38:42.322 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42194.log 2024-05-27T09:38:42.325 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-mds.e.log: /var/log/ceph/ceph-client.admin.42194.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42194.log.gz 2024-05-27T09:38:42.326 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50019.log 2024-05-27T09:38:42.328 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.50019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53042.log 2024-05-27T09:38:42.330 INFO:teuthology.orchestra.run.smithi114.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50019.log.gz 2024-05-27T09:38:42.331 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48266.log 2024-05-27T09:38:42.334 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.53042.log: /var/log/ceph/ceph-client.admin.48266.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53042.log.gz 2024-05-27T09:38:42.335 INFO:teuthology.orchestra.run.smithi114.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42176.log 2024-05-27T09:38:42.335 INFO:teuthology.orchestra.run.smithi114.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48266.log.gz 2024-05-27T09:38:42.337 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50516.log 2024-05-27T09:38:42.338 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51845.log 2024-05-27T09:38:42.341 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42176.log: /var/log/ceph/ceph-client.admin.51845.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42176.log.gz 2024-05-27T09:38:42.342 INFO:teuthology.orchestra.run.smithi114.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51845.log.gz 2024-05-27T09:38:42.344 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.50516.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44719.log 2024-05-27T09:38:42.344 INFO:teuthology.orchestra.run.smithi114.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50516.log.gz 2024-05-27T09:38:42.345 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41557.log 2024-05-27T09:38:42.347 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52958.log 2024-05-27T09:38:42.350 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44719.log: 88.3%/var/log/ceph/ceph-client.admin.52958.log: -- replaced with /var/log/ceph/ceph-client.admin.44719.log.gz 2024-05-27T09:38:42.351 INFO:teuthology.orchestra.run.smithi114.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52958.log.gz 2024-05-27T09:38:42.352 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41557.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41557.log.gz 2024-05-27T09:38:42.353 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43840.log 2024-05-27T09:38:42.355 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41863.log 2024-05-27T09:38:42.357 INFO:teuthology.orchestra.run.smithi114.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43840.log.gz 2024-05-27T09:38:42.358 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44815.log 2024-05-27T09:38:42.363 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44815.log: /var/log/ceph/ceph-client.admin.41863.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52157.log 2024-05-27T09:38:42.364 INFO:teuthology.orchestra.run.smithi114.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41863.log.gz 2024-05-27T09:38:42.364 INFO:teuthology.orchestra.run.smithi114.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44815.log.gz 2024-05-27T09:38:42.366 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52031.log 2024-05-27T09:38:42.369 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.52031.log: /var/log/ceph/ceph-client.admin.52157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41857.log 2024-05-27T09:38:42.371 INFO:teuthology.orchestra.run.smithi114.stderr: 88.5% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52157.log.gz 2024-05-27T09:38:42.372 INFO:teuthology.orchestra.run.smithi114.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52031.log.gz 2024-05-27T09:38:42.374 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49935.log 2024-05-27T09:38:42.377 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49935.log: /var/log/ceph/ceph-client.admin.41857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40788.log 2024-05-27T09:38:42.379 INFO:teuthology.orchestra.run.smithi114.stderr: 87.4% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41857.log.gz 2024-05-27T09:38:42.380 INFO:teuthology.orchestra.run.smithi114.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49935.log.gz 2024-05-27T09:38:42.381 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52538.log 2024-05-27T09:38:42.383 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.40788.log: -- /var/log/ceph/ceph-client.admin.46525.log 2024-05-27T09:38:42.386 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.52538.log: 88.2% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40788.log.gz 2024-05-27T09:38:42.387 INFO:teuthology.orchestra.run.smithi114.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52538.log.gz 2024-05-27T09:38:42.388 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48445.log 2024-05-27T09:38:42.390 INFO:teuthology.orchestra.run.smithi163.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-27T09:38:42.392 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48445.log: /var/log/ceph/ceph-client.admin.46525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42195.log 2024-05-27T09:38:42.394 INFO:teuthology.orchestra.run.smithi114.stderr: 88.6% 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46525.log.gz 2024-05-27T09:38:42.394 INFO:teuthology.orchestra.run.smithi114.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48445.log.gz 2024-05-27T09:38:42.396 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50061.log 2024-05-27T09:38:42.397 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45264.log 2024-05-27T09:38:42.398 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-05-27T09:38:42.400 INFO:teuthology.orchestra.run.smithi114.stderr: 88.4%/var/log/ceph/ceph-client.admin.50061.log: /var/log/ceph/ceph-client.admin.45264.log: -- replaced with /var/log/ceph/ceph-client.admin.42195.log.gz 2024-05-27T09:38:42.401 INFO:teuthology.orchestra.run.smithi114.stderr: 88.2% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50061.log.gz 2024-05-27T09:38:42.401 INFO:teuthology.orchestra.run.smithi114.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45264.log.gz 2024-05-27T09:38:42.402 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52073.log 2024-05-27T09:38:42.405 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.52073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45391.log 2024-05-27T09:38:42.406 INFO:teuthology.orchestra.run.smithi114.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52073.log.gz 2024-05-27T09:38:42.407 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45180.log 2024-05-27T09:38:42.412 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45391.log: /var/log/ceph/ceph-client.admin.45180.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50672.log 2024-05-27T09:38:42.412 INFO:teuthology.orchestra.run.smithi114.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45391.log.gz 2024-05-27T09:38:42.414 INFO:teuthology.orchestra.run.smithi114.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45180.log.gz 2024-05-27T09:38:42.417 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.50672.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50672.log.gz 2024-05-27T09:38:42.417 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48316.log 2024-05-27T09:38:42.418 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48316.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48316.log.gz 2024-05-27T09:38:42.419 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41338.log 2024-05-27T09:38:42.421 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49457.log 2024-05-27T09:38:42.422 INFO:teuthology.orchestra.run.smithi114.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41338.log.gz 2024-05-27T09:38:42.424 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46777.log 2024-05-27T09:38:42.428 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46777.log: /var/log/ceph/ceph-client.admin.49457.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42537.log 2024-05-27T09:38:42.430 INFO:teuthology.orchestra.run.smithi114.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49457.log.gz 88.0% 2024-05-27T09:38:42.430 INFO:teuthology.orchestra.run.smithi114.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46777.log.gz 2024-05-27T09:38:42.431 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-27T09:38:42.435 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.42537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51887.log 2024-05-27T09:38:42.437 INFO:teuthology.orchestra.run.smithi114.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42993.log 2024-05-27T09:38:42.438 INFO:teuthology.orchestra.run.smithi114.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42537.log.gz 2024-05-27T09:38:42.439 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51368.log 2024-05-27T09:38:42.443 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42993.log: /var/log/ceph/ceph-client.admin.51887.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51887.log.gz 2024-05-27T09:38:42.444 INFO:teuthology.orchestra.run.smithi114.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42993.log.gz/var/log/ceph/ceph-client.admin.51368.log: 2024-05-27T09:38:42.444 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40972.log 2024-05-27T09:38:42.450 INFO:teuthology.orchestra.run.smithi114.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51761.log 2024-05-27T09:38:42.451 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40972.log: -- replaced with /var/log/ceph/ceph-client.admin.51368.log.gz 2024-05-27T09:38:42.451 INFO:teuthology.orchestra.run.smithi114.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40972.log.gz 2024-05-27T09:38:42.451 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50350.log 2024-05-27T09:38:42.452 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.51761.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51761.log.gz 2024-05-27T09:38:42.452 INFO:teuthology.orchestra.run.smithi163.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-27T09:38:42.454 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.50350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46567.log 2024-05-27T09:38:42.455 INFO:teuthology.orchestra.run.smithi114.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50350.log.gz 2024-05-27T09:38:42.457 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41170.log 2024-05-27T09:38:42.459 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46567.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.46567.log.gz 2024-05-27T09:38:42.462 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46399.log 2024-05-27T09:38:42.463 INFO:teuthology.orchestra.run.smithi114.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41170.log.gz 2024-05-27T09:38:42.464 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53532.log 2024-05-27T09:38:42.465 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51283.log 2024-05-27T09:38:42.466 INFO:teuthology.orchestra.run.smithi114.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46399.log.gz 2024-05-27T09:38:42.467 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.53532.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53532.log.gz 2024-05-27T09:38:42.472 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42906.log 2024-05-27T09:38:42.473 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.51283.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51283.log.gz 2024-05-27T09:38:42.475 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41380.log 2024-05-27T09:38:42.476 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42906.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42906.log.gz 2024-05-27T09:38:42.481 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41014.log 2024-05-27T09:38:42.483 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41380.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41380.log.gz 2024-05-27T09:38:42.484 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51038.log 2024-05-27T09:38:42.488 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41014.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41014.log.gz 2024-05-27T09:38:42.488 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50760.log 2024-05-27T09:38:42.490 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.51038.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51038.log.gz 2024-05-27T09:38:42.493 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41599.log 2024-05-27T09:38:42.494 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.50760.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50760.log.gz 2024-05-27T09:38:42.496 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52832.log 2024-05-27T09:38:42.496 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41599.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41599.log.gz 2024-05-27T09:38:42.500 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42538.log 2024-05-27T09:38:42.502 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.52832.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52832.log.gz 2024-05-27T09:38:42.506 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41853.log 2024-05-27T09:38:42.507 INFO:teuthology.orchestra.run.smithi114.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42538.log.gz 2024-05-27T09:38:42.508 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52748.log 2024-05-27T09:38:42.510 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41853.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41853.log.gz 2024-05-27T09:38:42.513 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53616.log 2024-05-27T09:38:42.515 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.52748.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52748.log.gz 2024-05-27T09:38:42.515 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52199.log 2024-05-27T09:38:42.517 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.53616.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53616.log.gz 2024-05-27T09:38:42.517 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51240.log 2024-05-27T09:38:42.519 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.52199.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52199.log.gz 2024-05-27T09:38:42.523 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.51240.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51240.log.gz 2024-05-27T09:38:42.523 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53696.log 2024-05-27T09:38:42.528 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45135.log 2024-05-27T09:38:42.530 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.53696.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53696.log.gz 2024-05-27T09:38:42.531 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49628.log 2024-05-27T09:38:42.533 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45135.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45135.log.gz 2024-05-27T09:38:42.534 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42190.log 2024-05-27T09:38:42.536 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49628.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49628.log.gz 2024-05-27T09:38:42.539 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46735.log 2024-05-27T09:38:42.541 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42190.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42190.log.gz 2024-05-27T09:38:42.544 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41683.log 2024-05-27T09:38:42.544 INFO:teuthology.orchestra.run.smithi114.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46735.log.gz 2024-05-27T09:38:42.545 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53448.log 2024-05-27T09:38:42.548 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42531.log 2024-05-27T09:38:42.549 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.53448.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53448.log.gz 2024-05-27T09:38:42.550 INFO:teuthology.orchestra.run.smithi114.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41683.log.gz 2024-05-27T09:38:42.553 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42531.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42531.log.gz 2024-05-27T09:38:42.598 INFO:teuthology.orchestra.run.smithi114.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-27T09:38:42.933 INFO:teuthology.orchestra.run.smithi114.stderr: 90.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-27T09:38:45.079 INFO:teuthology.orchestra.run.smithi163.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-27T09:38:45.680 INFO:teuthology.orchestra.run.smithi163.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-27T09:38:45.965 INFO:teuthology.orchestra.run.smithi114.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-27T09:38:45.980 INFO:teuthology.orchestra.run.smithi163.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-27T09:38:46.895 INFO:teuthology.orchestra.run.smithi114.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-27T09:38:47.080 INFO:teuthology.orchestra.run.smithi163.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-27T09:38:47.082 INFO:teuthology.orchestra.run.smithi163.stderr: 2024-05-27T09:38:47.082 INFO:teuthology.orchestra.run.smithi163.stderr:real 0m5.004s 2024-05-27T09:38:47.083 INFO:teuthology.orchestra.run.smithi163.stderr:user 0m15.769s 2024-05-27T09:38:47.083 INFO:teuthology.orchestra.run.smithi163.stderr:sys 0m0.545s 2024-05-27T09:38:49.234 INFO:teuthology.orchestra.run.smithi114.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-27T09:38:49.756 INFO:teuthology.orchestra.run.smithi114.stderr: 92.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-27T09:39:00.622 INFO:teuthology.orchestra.run.smithi114.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-05-27T09:39:02.881 INFO:teuthology.orchestra.run.smithi114.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-27T09:39:02.882 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-05-27T09:39:02.883 INFO:teuthology.orchestra.run.smithi114.stderr:real 0m20.806s 2024-05-27T09:39:02.883 INFO:teuthology.orchestra.run.smithi114.stderr:user 0m58.555s 2024-05-27T09:39:02.883 INFO:teuthology.orchestra.run.smithi114.stderr:sys 0m4.869s 2024-05-27T09:39:02.884 INFO:tasks.ceph:Archiving logs... 2024-05-27T09:39:02.884 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727681/remote/smithi114/log 2024-05-27T09:39:02.885 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-27T09:39:11.134 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727681/remote/smithi163/log 2024-05-27T09:39:11.195 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-27T09:39:15.965 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-27T09:39:16.551 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-27T09:39:16.552 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-27T09:39:16.563 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-27T09:39:16.698 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-27T09:39:16.739 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-27T09:39:16.739 DEBUG:teuthology.orchestra.run.smithi114:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T09:39:16.750 DEBUG:teuthology.orchestra.run.smithi163:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T09:39:16.767 INFO:teuthology.orchestra.run.smithi163.stderr:bash: line 1: ntpq: command not found 2024-05-27T09:39:16.772 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T09:39:16.772 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2024-05-27T09:39:16.772 INFO:teuthology.orchestra.run.smithi163.stdout:^* hv01.front.sepia.ceph.com 3 6 377 63 +1081us[+1120us] +/- 64ms 2024-05-27T09:39:16.772 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 61 -1425us[-1425us] +/- 53ms 2024-05-27T09:39:16.772 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 +608us[ +608us] +/- 61ms 2024-05-27T09:39:16.772 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:39:16.797 INFO:teuthology.orchestra.run.smithi114.stderr:bash: line 1: ntpq: command not found 2024-05-27T09:39:16.807 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T09:39:16.807 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2024-05-27T09:39:16.807 INFO:teuthology.orchestra.run.smithi114.stdout:^* hv01.front.sepia.ceph.com 3 6 377 62 +1048us[+1079us] +/- 64ms 2024-05-27T09:39:16.807 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 62 -1476us[-1476us] +/- 53ms 2024-05-27T09:39:16.807 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 +576us[ +576us] +/- 61ms 2024-05-27T09:39:16.807 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:39:16.808 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-27T09:39:16.821 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-27T09:39:16.822 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-27T09:39:16.836 DEBUG:teuthology.orchestra.run.smithi114:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-27T09:39:17.038 DEBUG:teuthology.orchestra.run.smithi163:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-27T09:39:17.103 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T09:39:17.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:39:17.188 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T09:39:17.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:39:17.216 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-27T09:39:17.226 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-27T09:39:17.234 INFO:teuthology.task.internal:Duration was 1299.141271 seconds 2024-05-27T09:39:17.235 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-27T09:39:17.244 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-27T09:39:17.244 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-27T09:39:17.252 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-27T09:39:17.296 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T09:39:17.390 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T09:39:17.644 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-27T09:39:17.644 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi114.front.sepia.ceph.com 2024-05-27T09:39:17.644 DEBUG:teuthology.orchestra.run.smithi114:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-27T09:39:17.707 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi163.front.sepia.ceph.com 2024-05-27T09:39:17.708 DEBUG:teuthology.orchestra.run.smithi163:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-27T09:39:17.729 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-27T09:39:17.729 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-27T09:39:17.734 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-27T09:39:17.810 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-27T09:39:17.810 DEBUG:teuthology.orchestra.run.smithi114:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-27T09:39:17.815 DEBUG:teuthology.orchestra.run.smithi163:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-27T09:39:17.992 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-27T09:39:18.002 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-27T09:39:18.002 DEBUG:teuthology.orchestra.run.smithi114:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-27T09:39:18.085 DEBUG:teuthology.orchestra.run.smithi163:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-27T09:39:18.109 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-27T09:39:18.118 DEBUG:teuthology.orchestra.run.smithi114:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-27T09:39:18.126 DEBUG:teuthology.orchestra.run.smithi163:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-27T09:39:18.171 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = core 2024-05-27T09:39:18.213 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = core 2024-05-27T09:39:18.253 DEBUG:teuthology.orchestra.run.smithi114:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-27T09:39:18.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:39:18.298 DEBUG:teuthology.orchestra.run.smithi163:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-27T09:39:18.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:39:18.312 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-27T09:39:18.325 INFO:teuthology.task.internal:Transferring archived files... 2024-05-27T09:39:18.326 DEBUG:teuthology.misc:Transferring archived files from smithi114:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727681/remote/smithi114 2024-05-27T09:39:18.326 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-27T09:39:18.433 DEBUG:teuthology.misc:Transferring archived files from smithi163:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727681/remote/smithi163 2024-05-27T09:39:18.433 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-27T09:39:18.465 INFO:teuthology.task.internal:Removing archive directory... 2024-05-27T09:39:18.465 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-27T09:39:18.473 DEBUG:teuthology.orchestra.run.smithi163:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-27T09:39:18.522 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-27T09:39:18.533 INFO:teuthology.task.internal:Not uploading archives. 2024-05-27T09:39:18.534 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-27T09:39:18.544 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-27T09:39:18.544 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-27T09:39:18.550 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-27T09:39:18.570 INFO:teuthology.orchestra.run.smithi163.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 27 09:39 /home/ubuntu/cephtest 2024-05-27T09:39:18.598 INFO:teuthology.orchestra.run.smithi114.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 27 09:39 /home/ubuntu/cephtest 2024-05-27T09:39:18.600 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-27T09:39:18.611 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-27T09:39:18.639 INFO:teuthology.run:Summary data: client.0-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} duration: 1299.1412708759308 failure_reason: 'Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi114 with status 128: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6507388dd5057528934822c0163b0c347ef1d5d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh''' flavor: default owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=9e7a4c8b869448ea9bdb01b3774aa0ae status: fail success: false 2024-05-27T09:39:18.639 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-27T09:39:18.722 INFO:teuthology.run:FAIL